JP2016110585A - Sales processing system and commodity popularity analysis method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform popularity analysis on a commodity that has an individual difference in terms of appearance.SOLUTION: A sales processing system 1 includes a CPU 101 that performs control of storing commodity individual information, which is based on an individual difference of a commodity in terms of appearance acquired through image recognition when the type of commodity is identified through the image recognition in order to perform sales processing on the commodity, in association with a sale time situation concerning a sale time based on the sales of the commodity in a memory unit 104, that references the sale time situation of each of plural commodities of the same type, which are stored in the memory unit 104, so as to discriminate the popularity of each commodity, and that outputs commodity individual information on each commodity associated with the discriminated popularity or feature information based on the commodity individual information.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a sales processing apparatus and object popularity analysis method using object recognition.

  At a bakery that sells handmade bread, customers use the tongs to pick up the desired bread from the bread display stand and place the bread on the tray, then pay the price at the POS terminal (cash register). Receive. Here, POS is an abbreviation for point of sale system. In such a bakery, handmade bread is often displayed as it is on a display stand without being packed in a bag. As a result, the fresh bread can be provided to the customer without sacrificing the taste and texture, and the man-hours for bread packaging can be omitted. On the other hand, there is a problem that bread cannot be managed with a barcode by not packing the bread.

  There are many types of handmade bread, and the appearance of the same type of product is not always the same. Conventionally, a bakery clerk has been required to accurately identify the type of bread on the tray and manually input it to the POS terminal. Therefore, a predetermined training period for the store clerk is necessary.

  For this reason, in recent years, a POS terminal for a bakery that automates product registration by photographing the bread on the tray with a camera and recognizing (object recognition) the type and quantity of bread on the tray from this image information has appeared. doing. Thereby, the training period of the bakery clerk can be shortened.

  In Patent Document 1 as an example of such a conventional technique, a problem is described that “a type of bread is accurately identified by image recognition”, and “a pan color image is a color image of an inner region” is described as a configuration. And the color image of the outer region, the feature value related to the outline of the pan, the feature value related to the texture of the pan, the feature value in the color space of the inner region of the pan, and the feature value of the outer region of the pan. The feature amount in the color space is obtained and the type of bread is identified ".

  Conventionally, sales analysis has been carried out in a merchandise store such as a bakery for the purpose of marketing and the like by counting and analyzing the number of sales and the sales time for each product. According to this sales analysis, it is possible to know how many products are sold in what time zone, and to identify so-called best-selling products.

JP 2011-170745 A

  However, in the above-mentioned conventional sales analysis, what is the individual difference in sales for products that are the same type of product but have individual differences in appearance (for example, handmade bread, cakes, Japanese sweets, stuffed animals, kokeshi dolls, etc.) Cannot be analyzed. For example, in the case of handmade bread, even if it is a product of the same type, there are individual differences in the size, shape, color, topping (such as sesame of anpan), etc. depending on the bread, but what kind of bread is sold It is easy to analyze what kind of bread is hard to sell. If the impact on sales (consumer's preference) due to individual differences can be analyzed, it can be used as reference for handmade the product from the next time, and sales and customer satisfaction can be improved.

  Therefore, an object of the present invention is to enable a popularity analysis of products having individual differences in appearance.

In order to achieve the above object, the present invention
A sales processing device that performs product sales processing by specifying a product type by image recognition,
When performing the sales process of the product by specifying the product type by the image recognition, product individual information based on individual differences in the appearance of the product acquired by the image recognition, the product type, Storage control means for controlling to store in the storage means in association with the sales point situation regarding the sales point based on the sales of the product;
With respect to a plurality of products of the same type stored in the storage unit, a determination unit that determines the degree of popularity of each individual product by referring to the sales point situation of the individual product;
Output means for outputting the product individual information of the individual product corresponding to the degree of popularity determined by the determination unit, or feature information based on the product individual information;
A sales processing apparatus characterized by comprising:

In order to achieve the above object, the present invention
It is a product popularity analysis method for performing product sales processing by specifying a product type by image recognition,
When performing the sales process of the product by specifying the product type by the image recognition, product individual information based on individual differences in the appearance of the product acquired by the image recognition, the product type, A storage step for storing in the storage means in association with the sales point situation regarding the sales point based on the sales of the product;
For a plurality of products of the same type stored in the storage means, a determination step of determining the popularity of each product by referring to the sales point situation of each product;
An output step for outputting the product individual information of the individual product corresponding to the degree of popularity determined in the determination step, or feature information based on the product individual information;
It is a commodity popularity analysis method characterized by including.

  According to the present invention, it is possible to analyze the popularity of products having individual differences in appearance.

It is an external appearance perspective view which shows the sales processing apparatus in 1st Embodiment. 1 is a schematic configuration diagram illustrating a sales processing apparatus according to a first embodiment. It is a logical block diagram which shows the sales processing apparatus in 1st Embodiment. It is a figure which shows the example of a mode that the bread is put on the display stand of bakery in 1st Embodiment. It is a figure which shows the example of the screen transition in 1st Embodiment. It is an activity diagram which shows operation | movement of the sales processing apparatus in 1st Embodiment. It is a figure which shows the example of the data for object recognition in 1st Embodiment. It is a figure which shows the example of the product content data in 1st Embodiment. It is a figure which shows the example of the sales master in 1st Embodiment. It is a flowchart which shows the goods recognition process in 1st Embodiment. It is a flowchart which shows the popularity analysis process of the goods in 1st Embodiment. It is a figure which shows the example of the feature-value coordinate plane of an bread in 1st Embodiment. It is a figure which shows the example of the screen which displayed the representative image for every group of bread | pans with popularity in 1st Embodiment. It is a flowchart which shows the popularity analysis process of goods in 2nd Embodiment. It is a figure which shows the example of the feature-value coordinate plane of an bread in 2nd Embodiment. In the second embodiment, a screen displaying a representative image of anpan group sold before the start of the time sale and a representative image of three child groups of anpan group sold after the start of the time sale. It is a figure which shows an example.

  Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings. In addition, in several drawing, the same code | symbol is provided to the same element and the overlapping description is abbreviate | omitted suitably. In the first embodiment, a case where a sales processing apparatus is introduced into a bakery will be described as an example.

FIG. 1 is an external perspective view showing a sales processing device 1 according to the first embodiment.
As shown in FIG. 1, the sales processing apparatus 1 includes a customer display 11, a touch display 12, a cash drawer 13, a printer 14, a photographing device 15, and a photographing stand 16, and a product at a bakery. It is installed on the counter stand 2 that performs It should be noted that an operator (store clerk) who handles the sales processing apparatus 1 stands on the near side of the counter table 2 when drawing the merchandise. The customer stands on the back side of the counter table 2 in the drawing.

  The sales processing apparatus 1 functions as a POS terminal, specifies the type and number of bread as a product by image recognition, displays total price, etc., and executes calculation / input / output such as sales management / sales processing.

  The customer display 11 is, for example, a liquid crystal display device, and faces the back side of the drawing, which is the customer side. The customer display 11 displays information on merchandise accounting (product name, amount, etc.) to the customer.

  The touch display 12 has a touch panel 12B (FIG. 2) laminated on the surface of a display 12A (FIG. 2) which is a liquid crystal display device, for example, and faces the front side of the drawing which is an operator side. The touch display 12 displays a photographed image and various information (product name, amount, etc.) to the operator and accepts a touch operation input by the operator.

The cash drawer 13 is a drawer that stores banknotes, money, cash vouchers, etc., which are handled at the time of product accounting, and is located directly below the touch display 12.
The printer 14 is located at the lower left of the touch display 12 and prints the contents of accounting (product name, amount, etc.) on the receipt when the product is accounted for.

  The imaging device 15 is a tray 3 and a product in the imaging area 151 that is an area that can be imaged by the imaging device 15 on the imaging platform 16 (hereinafter, accessories and products such as the tray 3 are collectively referred to as “product etc.”). Take a picture from right above. The photographing stand 16 is a stand on which the customer who purchases the product places the tray 3 on which the product is placed at the time of payment. An illumination device (not shown) is attached adjacent to the photographing device 15 and illuminates a photographing region 151 by the photographing device 15. The product is, for example, handmade bread.

  When the photographing device 15 performs photographing, the photographing region 151 is irradiated with illumination light from the illumination device, and the backlight from the backlight light source 17 is irradiated upward from below the tray 3. Although the tray 3 is not transparent, it is translucent to transmit light in the vertical direction and is formed in a single color having no pattern. The color of the tray 3 is preferably a translucent white or light color. Furthermore, it is desirable that the tray 3 has a fine matte process on its upper surface. Thereby, reflection of the illumination light from an illuminating device can be suppressed.

  The customer places an arbitrary number of breads (for example, bun 5) on the tray 3 and places the tray 3 on the imaging table 16. In the example of FIG. 1, one bun 5 is placed on the tray 3.

  The backlight light source 17 is accommodated in the photographing stand 16 and is backlit from the bottom of the tray 3 so that the photographed image becomes clearer when photographing the product in the photographing region 151 by the photographing device 15. Irradiate upward. The backlight light source 17 can be realized by, for example, an LED (Light Emitting Diode), but is not limited thereto.

  The tray 3 is translucent as described above. When the pan on the tray 3 is photographed by the photographing device 15, the backlight source 17 irradiates the back side behind the tray 3 with the backlight. Thereby, the shadow formed in the circumference | surroundings of bread with the illumination light from an illuminating device can be eliminated as much as possible. In order to obtain a state in which the backlight is irradiated from the backlight light source 17 when photographing with the photographing device 15, for example, the backlight light source 17 is always turned on. However, the present invention is not limited to this, and lighting of the backlight light source 17 and photographing of the photographing device 15 may be synchronized. In order to realize this, the sales processing device 1 may control the photographing device 15 and the backlight light source 17 so that the backlight light source 17 is turned on in synchronization with the photographing device 15 when photographing.

  The sales processing device 1 identifies the product on the tray 3 by processing the image photographed by the photographing device 15. Then, the sales processing device 1 registers (sales registration) the sales information of the product in the sales master 107 (FIG. 3).

FIG. 2 is a schematic configuration diagram illustrating the sales processing device 1 according to the first embodiment.
In addition to the configuration shown in FIG. 1, the sales processing apparatus 1 includes a CPU (Central Processing Unit) 101, a RAM (Random Access Memory) 102, a ROM (Read Only Memory) 103, a storage unit 104, a speaker 18, and the like. It has. Each component is connected to be communicable with each other via an internal bus and each input / output circuit (not shown).

The CPU 101 is a central control device and is a control means that controls the entire sales processing device 1.
The RAM 102 is temporary storage means used by the CPU 101, and temporarily stores image data and various variables related to a program executed by the CPU 101.
The ROM 103 is a non-volatile storage unit and stores a program executed by the CPU 101.

The customer display 11 is a display unit that is controlled by the CPU 101 and displays information (product name, amount, etc.) relating to a photographed image of the product and the accounting for the product to the customer.
The display 12A is a display unit that is controlled by the CPU 101 and displays a photographed image of a product and information (product name, price, etc.) related to product accounting to an operator.
The touch panel 12B receives an operator's touch operation input for information displayed on the display 12A.

  The storage unit 104 is configured by, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like, and is a storage unit that stores various programs and various files. Various programs and various files stored in the storage unit 104 are all or partially copied to the RAM 102 and executed by the CPU 101 when the sales processing apparatus 1 is activated. The storage unit 104 stores various files, but stores at least sales information of products having individual differences in appearance so that the sales order can be identified.

  The photographing device 15 includes a color CCD (Charge Coupled Device) image sensor, a color CMOS (Complementary Metal Oxide Semiconductor) image sensor, and the like. The imaging device 15 is an imaging unit that performs imaging under the control of the CPU 101. The imaging device 15 captures a moving image of 30 fps (frame per second), for example. Frame images (captured images) sequentially captured by the image capturing device 15 at a predetermined frame rate are stored in the RAM 102.

  Under the control of the CPU 101, the backlight light source 17 faces the backlight upward from the bottom of the tray 3 so that the photographed image becomes clearer when the photographing device 15 photographs the product on the tray 3 or the tray 3. Irradiate. Thereby, the shadow in the imaging | photography area | region 151 formed with the illumination light from an illuminating device or the other illumination in a shop can be made thin, and the precision of image processing can be improved.

The cash drawer 13 is controlled to open according to an instruction from the CPU 101.
The printer 14 is a thermal transfer printer, for example, and prints a receipt. Specifically, the printer 14 prints the contents of the accounting on the receipt according to an instruction from the CPU 101 at the time of accounting for the product.
The speaker 18 generates a preset notification sound or the like. The speaker 18 performs warning sound or voice notification under the control of the CPU 101.

FIG. 3 is a logical block diagram showing the sales processing device 1 in the first embodiment.
The CPU 101 (FIG. 2) of the sales processing apparatus 1 embodies the processing unit 9 by executing a program (not shown) stored in the ROM 103 (FIG. 2). The processing unit 9 includes an image storage unit 91, an object recognition processing unit 92, a confirmation notification unit 93, a candidate product presentation unit 94, an input acquisition unit 95, a sales registration unit 96, an information output unit 97, a popularity An analysis processing unit 98 and an object recognition data update unit 99 are included. The object recognition processing unit 92 includes an object detection unit 921, a similarity calculation unit 922, and a similarity determination unit 923.

  The processing unit 9 performs various processes with reference to the object recognition data 105, the product content data 106, and the sales master 107 stored in the storage unit 104. For example, when the processing unit 9 specifies a product type by image recognition and performs sales processing of the product, the processing unit 9 obtains product individual information based on individual differences in appearance of the product acquired by the image recognition. The sales data is stored in the sales master 107 of the storage unit 104 in association with the type of merchandise and the sales time status regarding the sales time based on the sales of the product.

  Here, as the individual product information based on the individual difference in appearance of the product (Anpan 5), for example, the individual feature amount calculated from the recognition image of the product, the recognition image itself, or the product type of the recognition image. The difference information with respect to the reference image can be mentioned, but is not limited thereto. In the first embodiment, an individual feature amount (for example, the overall shape of the top of pancake 5 or the arrangement area of sesame as topping) is adopted as the individual product information.

  Further, the point-of-sale status regarding the point of sale based on the sales of the product (Anpan 5) is information on the order of sale and the time of sale of the product. For example, the numerical value ( Information corresponding to the sales order) and the sales date and time column 107b. In the first embodiment, the numerical value of the customer number column 107a is adopted as the situation at the time of sale.

  The object recognition data 105 includes template information that combines feature quantities modeled for each type of product (such as bread). These pieces of information are registered in advance in the object recognition data 105. The object recognition data 105 is a data file in which an association between a product name or ID (IDentifier) and a feature amount of each product is set for each product displayed and sold in a store. Function as dictionary data (described later in FIG. 7).

  The product content data 106 is a data file in which product content information is set. In the product content data 106, the product name or ID, the unit price, etc. are set as product content information (described later in FIG. 8).

  The sales master 107 is a file for recording the sales registration of the product, and records the sales date and time, the sales product, the sales amount, the individual feature amount, etc. so that the sales order can be identified (described later in FIG. 9).

The image storage unit 91 sequentially captures and stores frame images (color digital images) photographed by the photographing device 15.
The object detection unit 921 uses edge detection or the like to separate an image as a commodity candidate in the captured frame image from the background, that is, to cut out and detect only the object to be identified. Specifically, when the customer places the tray 3 on the imaging table 16 and the operator instructs imaging, the processing unit 9 images the imaging area 151 on the imaging table 16 with the imaging device 15. The object detection unit 921 binarizes the captured frame image and extracts a contour line. Next, the object detection unit 921 compares the contour line extracted from the previous frame image with the contour line extracted from the current frame image, and separates the image into each region surrounded by the contour line, Detect an object.

  The similarity calculation unit 922 identifies the type of each product based on the detected separated image of each object. The similarity calculation unit 922 calculates, for each separated image, at least one state among a size, a shape, a color, a pattern, and the like as a feature amount. The similarity calculation unit 922 further compares each feature value (individual feature value) of each separated image with each feature value of the product recorded in the object recognition data 105, thereby each separated image and each object recognition object. The degree of similarity with the product recorded in the data 105 is calculated.

  Here, the similarity refers to the feature amount of each separated image when the feature amount of an object (image) assumed as a reference for the product recorded in the object recognition data 105 is 100% similarity. It shows whether they are similar to a certain extent. Note that, when there are a plurality of types of feature amounts, the similarity calculation unit 922 performs comprehensive evaluation based on the plurality of feature amounts, but at that time, the feature amount may be weighted for each feature amount.

Recognizing an object included in an image in this way is called generic object recognition. For example, in the following "Current status and future of general object recognition" written by Keiji Yanai, a survey of general object recognition research is added to methods, data sets, evaluation benchmarks, and the future is prospected.
Keiji Yanai, “Current Status and Future of General Object Recognition”, [online], Journal of Information Processing Society of Japan, November 15, 2007, Vol. 48, No. SIG16, pp. 1-24, [October 31, 2014 Day search], Internet <URL: http: //mm.cs.uec.ac.jp/IPSJ-TCVIM-Yanai.pdf>

A technique for performing general object recognition by dividing an image into regions for each object (object) is described in, for example, the following documents.
Jamie Shotton et al., “Semantic Texton Forests for Image Categorization and Segmentation”, Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on, [October 31, 2014 search], Internet <URL: http: // citeseerx .ist.psu.edu / viewdoc / download? doi = 10.1.1.145.3036 & rep = rep1 & type = pdf>

  Note that the method for calculating the degree of similarity between the feature amount of the photographed image and the feature amount of the product recorded in the object recognition data 105 is not particularly limited. For example, the similarity between the feature amount of the photographed image and the feature amount of each product recorded in the object recognition data 105 may be calculated as an absolute evaluation or may be calculated as a relative evaluation.

  When the similarity is calculated as an absolute evaluation, the feature amount of the separated image and the feature amount of the product stored as the object recognition data 105 are compared one-to-one, and the similarity calculated as a result of this comparison (0 to 100%) may be adopted as it is.

  Further, when calculating the similarity as a relative evaluation, the total of the similarities with each product is calculated to be 1.0 (100%). For example, it is assumed that the feature quantities of the products A and B are stored as the object recognition data 105. At this time, the separation image is calculated such that, for example, the similarity to the product A is 0.95 and the similarity to the product B is 0.05.

For each separated image of the object, the similarity determination unit 923 determines, for example, the following three options for the corresponding product based on the similarity calculated by the similarity calculation unit 922 using a predetermined algorithm.
(1) A product corresponding to the separated image is uniquely determined.
(2) There are one or more product candidates corresponding to the separated image.
(3) It is determined that there is no product corresponding to the separated image.

  As conditions for this determination, for example, the storage unit 104 stores conditions X and Y described below in advance. Hereinafter, a case where the similarity calculation method is absolute evaluation will be described as an example.

  Condition X is a condition for determining (1) above if satisfied. The condition X is, for example, “the similarity of the product with the highest similarity is 90% or more” and “the similarity of the product with the highest similarity and the similarity of the product with the second highest similarity. The difference is 20% or more. Specifically, for example, for an object in a certain separated image, it is assumed that the product having the highest similarity is the product A having a similarity of 95%, and the next is the product B having a similarity of 60%. Since the condition X is satisfied at this time, the product A corresponding to the separated image is uniquely determined.

If the condition X is not satisfied, the condition Y is used.
Condition Y is a condition for determining (2) above if satisfied. The condition Y is, for example, “there is at least one product having a similarity of 60% or more”. Specifically, for example, for an object in a certain separated image, the product with the highest similarity is the product A with the similarity 80%, the second is the product B with the similarity 75%, and the third is the similarity 65%. And the fourth is a product D having a similarity of 55%. Since the condition Y is satisfied at this time, it is determined that the products A, B, and C that are targets with a similarity of 60% or more exist as candidates corresponding to the separated image.

When both the conditions X and Y are not satisfied, it is determined as (3) above. The above-described conditions X and Y are examples, and are not limited to these.
Also, when the similarity calculation method is relative evaluation, the conditions may be set in the same manner.

The confirmation notifying unit 93 displays, on the display 12A and the customer display 11, and the sound output from the speaker 18 that the product is uniquely confirmed for the object of the separated image determined by the similarity determination unit 923 as (1). Etc. to inform the operator and customer.
More specifically, the confirmation notifying unit 93 displays the separated image determined by the similarity determining unit 923 as the above (1) on, for example, the customer display 11 or the display 12A together with the green frame line, thereby separating the separated image. Indicates that the product corresponding to is uniquely determined.

The candidate product presentation unit 94 displays the separated image determined by the similarity determination unit 923 as the above (2) on the display 12A or the customer display 11 together with the yellow frame line, for example, so that the product corresponding to the separated image is displayed. Indicates that one or more candidates exist. Further, when the operator touches the separated image via the touch panel 12B, the display 12A displays the photographic images and product names that are product candidates in descending order of similarity.
At this time, the candidate product presentation unit 94 reads out a photographic image and a product name of a product satisfying the condition Y from the object recognition data 105 and the product content data 106, and from the one with the high similarity calculated by the similarity calculation unit 922, The images are sequentially displayed on the display 12A.

  Note that even if photographic images of product candidates are displayed on the display 12A, if there is no selection operation for these product candidates, photographing by the photographing device 15, image storage processing by the image storage unit 91, and object The object detection process by the detection unit 921 and the similarity calculation process by the similarity calculation unit 922 are continued.

  The input acquisition unit 95 receives various input operations corresponding to the display on the display 12A via the touch panel 12B. For example, when it is determined as (2) above and a separated image is displayed on the display 12A together with the yellow frame line, the input acquisition unit 95 receives a touch input operation for selecting the separated image using the touch panel 12B by the operator. Furthermore, when one or more images of product candidates are displayed on the display 12A, a touch input operation for product selection using the touch panel 12B by the operator is accepted.

  The sales registration unit 96 refers to the object recognition data 105 and the product content data 106 based on the name or ID of the product, and registers the sales of the product. Specifically, the sales registration unit 96 temporarily records the individual feature amount calculated from the image information in the RAM 102 or the like in addition to the sales date and time, the sales product, the sales amount, etc., and performs temporary registration of sales. . The sales registration unit 96 further records the provisional registration information of sales recorded in the RAM 102 or the like in the sales master 107 or the like and performs sales registration.

  The information output unit 97 transmits the product ID of the confirmed product and the like confirmed as described above and the corresponding product classification, product name, unit price, number of sales, etc. to the customer display 11 and the display 12A for display. Various kinds of output such as transmission to the printer 14 and printing (printing).

The popularity analysis processing unit 98 determines the degree of popularity of each product or group by referring to the sales point situation of each product in the sales master 107 for a plurality of products of the same type (FIG. 11, FIG. 14 later).
Further, the popularity analysis processing unit 98 uses the information output unit 97 to output the product individual information of the individual product corresponding to the determined degree of popularity or the feature information based on the product individual information.

  Here, as the feature information based on the product individual information, for example, a representative image of a group to which the product belongs (details will be described later), or a comparison image with a reference image or an unpopular product image of the product type in the recognition image However, the present invention is not limited to these. In the first embodiment, a group representative image is employed as the feature information.

  Further, as the output of the individual product information using the information output unit 97 by the popularity analysis processing unit 98 or the feature information based on the individual product information, such information is transmitted to the customer display 11 or the touch display 12, for example. Display the information, send the information to the printer 14 or another printer for printing, or send the information as a result of analysis to a computer device other than the sales processing device 1 (for example, a personal computer at the head office). (Hereinafter, referred to as “output (display, printing, transmission, etc.)”), but is not limited thereto. Note that audio data may be included in the information to be output and output by the destination device.

  The object recognition data update unit 99 updates the object recognition data 105 (described later in FIGS. 11 and 14).

  FIG. 4 is a diagram illustrating an example of a state in which bread rolls are placed on a bakery display stand in the first embodiment. In FIG. 4, for the sake of simplicity of illustration and description, four types of bread rolls are shown with respect to two points: the overall shape in top view and the arrangement area of sesame as topping.

Anpan 5A (5) has a circular shape with a high overall roundness (hereinafter simply referred to as “roundness”) in a top view, and has a large sesame arrangement area.
Anpan 5B (5) has an elliptical shape with low roundness and a small arrangement area of sesame.
Anpan 5C (5) has an elliptical shape with low roundness and a large sesame arrangement area.
Anpan 5D (5) has a circular shape with high roundness and a small arrangement area of sesame.

  In this way, when four buns 5A to 5D that are products of the same type but have individual differences in appearance are placed on the display stand, when the customer purchases the buns 5, Based on this, the breads 5A to 5D are selected.

FIG. 5 is a diagram illustrating an example of screen transition in the first embodiment. This screen is displayed on the touch display 12 (FIG. 2).
FIG. 5A is a view showing an example of a screen when the tray 3 on which the bread 5 (any one of the breads 5A to 5D) is placed is placed on the photographing stand 16.
First, a through image by the photographing device 15 is displayed on the touch display 12. The customer places the bread pan 5 as a product on the tray 3 and places the tray 3 on the photographing table 16. When the operator starts shooting by touching the shooting button at the lower right of the screen, the screen shown in FIG. 5B is displayed on the touch display 12.

  In the screen of FIG. 5B, the bread pan 5 is given a green outline (shown by a broken line) to indicate that a product corresponding to the separated image of the bread pan 5 is uniquely determined. In addition, a window of “Anpan 150 yen” indicating the contents of the product is displayed corresponding to the Anpan 5.

  Thereafter, the operator (clerk) touches the confirmation button at the lower right of the screen. Thereby, the screen of FIG. 5C is displayed on the touch display 12. In the screen of FIG. 5C, the product name and unit price of the sales product are displayed, and the total price is further displayed. When the operator touches the settlement button at the lower right of the screen and executes settlement, information related to the sale of these products is registered in the sales master 107 together with the individual feature amounts.

FIG. 6 is an activity diagram showing the operation of the sales processing apparatus 1 in the first embodiment.
When the power is turned on, the sales processing device 1 transitions to the through image display mode of mode M10.
Mode M10 is a through image display mode. At this time, the sales processing device 1 displays the through image of the photographing device 15 on the touch display 12 as it is. Specifically, this through image is the screen illustrated in FIG. When the shooting button on the screen is touched in mode M10, the sales processing apparatus 1 transitions to mode M11.

  Mode M11 is a product etc. recognition processing mode. At this time, the sales processing device 1 performs object recognition processing on the photographed image by the photographing device 15 to recognize the product. Details of the product etc. recognition process will be described in detail with reference to FIG. When the product etc. recognition process is completed in mode M11, the sales processing apparatus 1 transitions to mode M12.

  Mode M12 is a recognition result display mode. At this time, the sales processing device 1 displays the result of the product etc. recognition processing. Specifically, the display of the result of the product etc. recognition process is the screen illustrated in FIG. In the modes M11 and M12, each process when the similarity determination unit 923 determines that “(2) one or more product candidates corresponding to the separated image exists”, that is, the separated image is displayed. The display surrounded by a yellow frame, the display of a candidate photographic image, etc., the selection input reception, and the like are also performed. When the confirmation button on the screen is touched in mode M12, the sales processing device 1 transitions to mode M13.

  Mode M13 is a settlement screen display mode. At this time, the sales processing apparatus 1 displays a settlement screen and prompts the operator to confirm. Specifically, the settlement screen display is the screen illustrated in FIG. When the settlement button on the screen is touched in mode M13, the sales processing apparatus 1 transitions to mode M14.

Mode M14 is a sales master registration processing mode. At this time, the sales processing apparatus 1 registers the individual feature amount in the sales master 107 in addition to the sales date and time, the sales product, the sales amount, and the like.
After the mode M14, the sales processing apparatus 1 ends a series of processes.

FIG. 7 is a diagram illustrating an example of the object recognition data 105 according to the first embodiment.
The object recognition data 105 includes a feature quantity column 105a related to a product, an image link column 105b, a type column 105c, and a corresponding product name column 105d. Note that the object recognition data 105 is not limited to these columns, and may include an arbitrary data column related to object recognition.

The feature amount column 105a stores a feature amount modeled for each type of product (eg bread).
The image link column 105b stores a link of a reference image related to this product. The feature amount stored in the feature amount column 105a is information indicating the size, shape, color, picture state, etc. of the reference image. The similarity determination unit 923 can determine which product has a high similarity to the object related to the separated image by referring to the feature amount column 105a.

The type column 105c stores the type of whether the product is a product itself or an accessory (coaster, spoon, etc.).
The product name column 105d stores the name of this product. When “product” is stored in the type column 105c, the product name of the product is stored in the product name column 105d.

FIG. 8 is a diagram illustrating an example of the product content data 106 according to the first embodiment.
The product content data 106 includes a product name field 106a, an image link field 106b, and a unit price field 106c. The product content data 106 is not limited to these columns, and may include a column of arbitrary content data related to the product.

The product name column 106a stores the product name of the product.
Similar to the image link field 105b of the object recognition data 105, the image link field 106b stores an image link of a product.
The unit price of the product is stored in the unit price column 106c. The sales registration unit 96 of the sales processing apparatus 1 calculates sales with reference to the unit price column 106c.

FIG. 9 is a diagram illustrating an example of the sales master 107 according to the first embodiment.
The sales master 107 includes a customer number column 107a, a sales date / time column 107b, a sales product column 107c, a sales amount column 107d, and a feature amount column 107e. Note that the sales master 107 is not limited to these columns, and may include columns of arbitrary data related to sales.

The customer number column 107a stores a serial number (sales order information) for each customer who purchased the product.
The sale date / time column 107b stores the sale date / time of the product. For example, when a time sale (half price of a product, etc.) is started at the time of the store, whether the product was sold before or after the time sale (sales time zone) is set in the sales date / time column 107b. It can be specified by reference.
The name of the sold product is stored in the sales product column 107c.
The sales amount column 107d stores the sales amount.
The feature amount column 107e stores individual feature amounts of sales products. The sales registration unit 96 of the sales processing apparatus 1 registers the individual feature amount calculated from the image information of the sales product in the feature amount column 107e.

FIG. 10 is a flowchart showing a product etc. recognition process (mode M11 in FIG. 6) in the first embodiment.
When the customer places the tray 3 on the photographing stand 16 and the operator touches the photographing button on the touch display 12 (FIG. 5A), the processing unit 9 starts a series of processes.
In step S <b> 1, the processing unit 9 outputs a shooting-on signal to the shooting device 15 and starts capturing (shooting) an image. The image storage unit 91 stores a frame image (captured image) captured by the imaging device 15 in the RAM 102.

  In step S <b> 2, the object recognition processing unit 92 performs a series of object recognition processes on the frame image stored in the image storage unit 91. In this process, the object detection unit 921 tries to recognize (detect) a separated image of an object that is a product. The similarity calculation unit 922 calculates a feature amount from the detected separated image and calculates a similarity with each product. Further, the similarity determination unit 923 determines a corresponding product using the predetermined algorithm.

  Specifically, when the bread pan 5 is placed on the tray 3 (FIG. 5B), the object detection unit 921 detects a separated image of the bread pan 5. The similarity calculation unit 922 calculates a feature amount (individual feature amount) from the separated image of the bread roll 5 and calculates a similarity with each product. The similarity determination unit 923 determines that the object of the separated image of the bread 5 is a bread according to the predetermined algorithm.

  Next, the processing unit 9 repeats the processes of steps S3 to S17 for the recognized objects. Through this repeated process, the product sales information and the individual feature amount are provisionally registered. Here, the provisional registration of the product sales information and the individual feature amount means that the information is temporarily stored in an arbitrary storage area (for example, the RAM 102) other than the sales master 107.

  In step S4, the sales registration unit 96 determines whether or not a product has been detected. If Yes, the process proceeds to step S5. If No, the process proceeds to step S7. Specifically, the sales registration unit 96 refers to the type column 105c (FIG. 7) of the object recognition data 105, and the “type” of the recognized object pan 5 is “product” (“ Yes "), the process proceeds to step S5.

  In step S5, the sales registration unit 96 searches for products. Specifically, the sales registration unit 96 searches the product name field 105d (FIG. 7) of the object recognition data 105 and identifies that the product corresponding to the bread 5 is “an bread”. After step S5, the process proceeds to step S6.

  In step S6, the sales registration unit 96 temporarily registers this product in sales. Specifically, the sales registration unit 96 temporarily records in the RAM 102 or the like that the sold product is “Anpan” and performs temporary registration of sales.

The processing unit 9 ends the process after repeating the processes of steps S3 to S7 for the recognized objects.
As can be seen from the activity diagram of FIG. 6, the sales product information and the individual feature quantities temporarily registered in the mode M11 of FIG. 10 are finally sold by the sales registration unit 96 in the sales master registration processing mode of the mode M14. Reflected (registered) in the master 107.

  FIG. 11 is a flowchart showing the popularity analysis process for merchandise according to the first embodiment. For example, when the operator performs an operation for starting the popularity analysis process on the touch display 12 after the bakery is closed, the popularity analysis processing unit 98 of the processing unit 9 starts a series of processes. Here, it is assumed that the popularity analysis process of the bread 5 is performed.

  In step S11, the popularity analysis processing unit 98 obtains the individual feature amount (feature amount column 107e) of the bread bun 5 that is the target sale product from the sales master 107 (FIG. 9), and sales order information (customer No. column 107a). Together with the numerical value). Here, the numerical value in the customer number column 107a is the serial number for each customer who purchased the product as described above, and can be used as sales order information (sales point-in-time situation).

  Next, in step S <b> 12, the popularity analysis processing unit 98 classifies the bread buns 5 that are target sales products into a plurality of groups having common characteristics based on the individual feature amounts. An example of this classification will be described with reference to FIG.

  FIG. 12 is a diagram illustrating an example of the feature amount coordinate plane of the bread pan in the first embodiment. Here, for the sake of simplicity of illustration and explanation, attention is paid to two feature amounts of roundness (horizontal axis) and sesame arrangement area (vertical axis) among the plurality of feature amounts in the bread roll 5. explain.

As shown in FIG. 12, when a plurality of sold bread rolls 5 are distributed as shown in the feature amount coordinate plane, for example, they can be classified into four groups G1 to G4.
Group G1 is a group of Anpan 5 having a high roundness and a large sesame arrangement area.
Group G2 is a group of bread rolls 5 having a high roundness and a small sesame arrangement area.
Group G3 is a group of Anpan 5 having a low roundness and a large sesame arrangement area.
Group G4 is a group of bread rolls 5 having a low roundness and a small arrangement area of sesame.

  Returning to FIG. 11, next, in step S13, the popularity analysis processing unit 98 calculates the popularity for each of the groups G1 to G4. For example, the popularity may be calculated by calculating an average sales order for each group and determining that the higher the average sales order is, the higher the popularity is.

  For example, for the bread bun 5 that is the target sales product, first, the sales order of the bread bun 5 alone is specified based on the read value in the customer number column 107a of the sales master 107. Specifically, for example, when 10 pieces of anpan 5 are sold, the sales order including bread other than anpan 5 is 3, 5, 8, 10, 12, 16, 20, 22, 25, 30. If it is the second, they are converted into the sales order 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 of the bread 5 only.

For example, for the group G1, if the number of sales is three and the sales order is the first, third and fifth, the average sales ranking is (1 + 3 + 5) th / 3th = third.
Further, regarding the group G2, if the number of sales is 3 and the sales order is the 2nd, 6th, and 7th, the average sales ranking is (2 + 6 + 7) / 3rd = 5th.
Further, regarding the group G3, if the number of sales is 2 and the sales order is 4th and 9th, the average sales ranking is (4 + 9) /2/2=6.5.
Further, regarding the group G4, if the number of sales is 2 and the sales order is the 8th and 10th, the average sales ranking is (8 + 10) th / 2 pieces = 9th.
Then, it can be understood that the groups G1 to G4 were the groups G1, G2, G3, and G4 in descending order of popularity from the average sales ranks.

The method for calculating the popularity is not limited to this. For example, in the above example, the sales order of all products of the target sales product was converted to the sales order of only that product, but without such conversion, the average sales order was calculated in the original sales order. Also good.
Further, the popularity may be calculated based on the average sales order, that is, not the average value of the sales order but the median value of the sales order.

  The popularity may be calculated based on the sales time (sales date / time column 107b of the sales master 107 in FIG. 9) (sales point-in-time situation) instead of the sales order. Specifically, for example, first, a sales time chart for each group is created, that is, each group is plotted on the horizontal axis and the sales time is plotted on the vertical axis, and the sales time of each product is plotted within the same group. Then, if the upper part of the vertical axis is the time elapsed from the sales start time, it can be determined that the more the plot is below, the higher the popularity of the group. For example, by calculating an average value (or median value) of the sales times of plots for each group and comparing the average values of the groups, the popularity ranking of each group can be known.

  In addition, when there are a plurality of lots (baked) like a bakery, for example, the popularity may be calculated based on the sales order and the sales time for each lot time zone.

  Next, in step S14, the popularity analysis processing unit 98 displays the representative images (characteristic information based on the individual product information) for each of the groups G1 to G4 together with the degree of popularity. An example of this display will be described with reference to FIG. In addition, what is necessary is just to use the image corresponding to the gravity center part of each area | region of FIG. 12, for example as a representative image for every group G1-G4.

  FIG. 13 is a diagram illustrating an example of a screen on which representative images for the groups G1 to G4 of the bread 5 are displayed together with the popularity degree in the first embodiment. As shown in FIG. 13, the representative images of the groups G1, G2, G3, and G4 are displayed on the display areas 21, 22, 23, and 24 on the touch display 12 in descending order of popularity.

  By looking at this screen, the operator can know how individual differences in the appearance of Anpan 5 have an effect on sales. Improve sales and customer satisfaction. In the example of FIG. 13, it can be seen that the higher the roundness, the higher the popularity. It can also be seen that the larger the sesame arrangement area, the higher the popularity. Further, from the representative images of the display areas 22 and 23, the group G2 having a high roundness and a small sesame arrangement area is more popular than the group G3 having a low roundness and a large sesame arrangement area. I understand that.

  Returning to FIG. 11, next, in step S <b> 15, the processing unit 9 accepts an end button (FIG. 13) on the touch display 12 by the operator or a display touch of the representative image, and ends the processing when the end button is touched. If the representative image is touched, the process proceeds to step S16. Here, for example, when the operator wants to use the representative image of the pancake 5 having the highest popularity (representative image displayed in the display area 21 in FIG. 13) as new object recognition data, the operator uses the display area 21. touch.

  In step S16, the object recognition data update unit 99 changes the feature amount column 105a corresponding to the pan 5 of the object recognition data 105 (FIG. 7) to the feature amount corresponding to the representative image displayed in the display area 21. By replacing, the object recognition data 105 is updated, and the process is terminated.

  As described above, according to the sales processing device 1 of the first embodiment, by performing popularity analysis of products having individual differences in appearance, in a merchandise store such as a bakery, marketing is strategically performed, Improve sales and customer satisfaction. Specifically, in the example of FIGS. 11 to 13, it is important to make a homemade bread bun 5 (an bread bun 5 in the display area 21 in FIG. 13) having a high popularity after the next time. 5 sales and customer satisfaction can be improved.

(Second Embodiment)
Next, the sales processing apparatus 1 of 2nd Embodiment is demonstrated. The second embodiment is an example where the sold bread rolls 5 are grouped into two time zones before and after the start of the time sale (a plurality of groups divided by the sales time zones). 1 to 10 are the same as those in the first embodiment, and a description thereof will be omitted.

  FIG. 14 is a flowchart showing a popularity analysis process for merchandise according to the second embodiment. For example, when the operator performs an operation for starting the popularity analysis process on the touch display 12 after the bakery is closed, the popularity analysis processing unit 98 of the processing unit 9 starts a series of processes. Here, as in the first embodiment, it is assumed that the popularity analysis processing of the bread 5 is performed. In the bakery, it is assumed that the time sale starts from a certain time in the evening, and the price of the bread 5 is reduced to half price thereafter. Therefore, the bread roll 5 sold before the start of the time sale is high in popularity, and the bread roll 5 sold after the start of the time sale is considered low in popularity.

  In step S21, the popularity analysis processing unit 98 obtains, from the sales master 107 (FIG. 9), the individual feature amount (feature amount column 107e) of the bread bun 5 before the start of the time sale for the bread bun 5 that is the target sales product. The individual feature amount (feature amount column 107e) of the bread roll 5 after the start of the time sale is read out.

  Next, the popularity analysis processing unit 98 displays the representative image of the bread bun 5 sold before the start of the time sale in step S22, and the representative image of the bread bun 5 sold after the start of the time sale in step S23. indicate. Examples of these displays will be described with reference to FIGS.

  FIG. 15 is a diagram illustrating an example of the feature amount coordinate plane of the bread pan in the second embodiment. Here, in order to simplify the illustration and description, among the plurality of feature amounts in the bread pan 5, two of the overall color density (horizontal axis) and the degree of deviation from the center of sesame (vertical axis) are shown. Description will be made by paying attention to the feature amount.

  As shown in FIG. 15, when the sold bread rolls 5 are distributed on the feature amount coordinate plane as shown in the figure, the group G11 is a group of bread rolls 5 sold before the start of the time sale. Moreover, the group of bread rolls 5 sold after the start of the time sale can be further classified into a plurality of child groups G12, G13, and G14 having characteristics common to the respective child groups. As for the classification into child groups, assuming that the group of bread rolls 5 sold after the start of the time sale is single, the representative image thereof is close to the representative image of the bread roll group 5 sold before the start of the time sale. This is to avoid the situation.

Group G11 is a group of Anpan 5 sold before the start of the time sale, which has a medium color density and a low degree of sesame bias. An image corresponding to P11 in the center of gravity can be used as the representative image.
Child group G12 is one of the child groups of Anpan 5 sold after the start of the time sale, and has a dark color and a low degree of sesame bias. An image corresponding to P12 in the center of gravity can be used as the representative image.

The child group G13 is one of the child groups of the Anpan 5 sold after the start of the time sale, and has a light color and a low degree of sesame bias. An image corresponding to P13 in the center of gravity can be used as a representative image.
Child group G14 is one of the child groups of Anpan 5 sold after the start of the time sale, and has a medium color density and a high degree of sesame bias. Then, an image corresponding to P14 in the center of gravity can be used as a representative image.

  FIG. 16 shows a representative image of the bread bun group sold before the start of the time sale and a representative image of three child groups of the bread bun sold after the time sale start in the second embodiment. It is a figure which shows the example of the displayed screen.

  As shown in FIG. 16, on the touch display 12, a representative image of the group of bread rolls 5 sold before the start of the time sale is displayed in the upper display area 31, and time is displayed in the lower display areas 32, 33, and 34, respectively. Representative images of child groups G12, G13, and G14 of Anpan 5 sold after the sale starts are displayed.

  By looking at this screen, the operator can know how individual differences in the appearance of Anpan 5 have an effect on sales. Improve sales and customer satisfaction. In the example of FIG. 16, it can be seen that the bun 5 (display area 31) having a medium color density and a low degree of sesame bias is highly popular. In addition, by looking at the bread buns 5 in the display areas 32, 33, and 34, it is possible to recognize the three types of bread buns 5 with low popularity sold after the start of the time sale.

  Returning to FIG. 11, next, in step S <b> 24, the processing unit 9 accepts an end button (FIG. 16) on the touch display 12 by the operator or a display touch of the representative image, and ends the processing when the end button is touched. If the representative image is touched, the process proceeds to step S25. Here, for example, when the operator wants to use the representative image (an display area 31 in FIG. 16) of the bread roll 5 sold before the start of the time sale as new object recognition data, the operator touches the display area 31.

  In step S25, the object recognition data update unit 99 corresponds the feature quantity in the feature quantity column 105a corresponding to the pan 5 of the object recognition data 105 (FIG. 7) to the representative image displayed in the display area 31. By replacing with the feature value, the object recognition data 105 is updated, and the process is terminated.

  As described above, according to the sales processing device 1 of the second embodiment, by performing popularity analysis of products having individual differences in appearance, in a merchandise store such as a bakery, strategic marketing is performed, Improve sales and customer satisfaction. Specifically, in the examples shown in FIGS. 14 to 16, by taking care to make the appearance of the bread 5 sold before the start of the time sale from the next time, the sales of the bread 5 and customers Satisfaction etc. can be improved. In addition, by displaying the bread roll 5 (unpopular product) sold after the start of the time sale, it is possible to take care not to make the bread bread 5 having such an appearance.

(Modification)
The present invention is not limited to the above-described embodiment, and can be modified without departing from the spirit of the present invention. For example, there are the following (a) to (i).

(A) In the above embodiment, the product is bread. However, in addition to those, for example, articles such as handmade cakes, Japanese confectionery, stuffed animals, kokeshi dolls having individual differences in appearance may be used as products.

(B) In the above-described embodiment, shooting is performed by illuminating a pan or the like on a translucent tray with a backlight. However, the present invention is not limited to this, and an opaque tray may be used to shoot a pan or the like on the tray by illumination from above.

(C) In the above embodiment, the feature amount of the product (anpan 5) has been described as two-dimensional in order to simplify the illustration and description. However, the present invention is not limited to this, and the feature amount of the product may be three-dimensional or more.

(D) In the above embodiment, the individual feature amount is calculated based on the photographed image of the product (Anpan 5), stored, and used. However, the present invention is not limited to this, and a captured image of the product (Anpan 5) may be compared with a reference image of the product type, and an individual feature amount may be calculated based on the difference, stored, and used. .

(E) In the above embodiment, the popularity analysis was performed only on the sold product (Anpan 5). However, the present invention is not limited thereto, and a representative image may be displayed as a less popular product by photographing unsold products, recognizing the objects, calculating individual feature amounts, storing them, and using them. .

(F) In the second embodiment, the group of products (anpan 5) sold after the start of the time sale is further classified into a plurality of child groups. However, the present invention is not limited to this, and a group of products sold after the start of the time sale may be handled as a single unit, and a representative image of the group may be displayed.

(G) In the second embodiment, the popularity of the product (Anpan 5) was analyzed by dividing the time zone into two before and after the start of the time sale. However, the present invention is not limited to this, and the popularity analysis of products may be performed by dividing the time zone into three or more, for example, by dividing every hour.

(H) In the above-described embodiment, the individual feature amount is calculated and stored as individual product information for each individual product that has been image-recognized (the feature amount column 107e of the sales master 107 in FIG. 9). However, the present invention is not limited to this, and instead of storing the individual feature amount, the recognition image itself may be stored as information corresponding to each product as product individual information. In that case, for example, in the popular analysis process, the analysis result in the form of comparing the recognition image of the product with the earlier sales point in the same type as the feature image of the popular product with the recognition image of the product with the later sales point. Output (display, printing, transmission, etc.).

(I) In the above embodiment, an image corresponding to the center of gravity of the area of the group on the feature amount coordinate plane (FIGS. 12 and 15) is used as the representative image of the product of the group with high popularity. However, the present invention is not limited to this, and the image of the most popular product (that is, the product sold first) may be used as the representative image of the group. When displaying the image of the most popular product, if the images of the unpopular product are displayed side by side, the features of the most popular product can be more easily recognized.

The invention described in the scope of claims attached to the application of this application will be added below. The item numbers of the claims described in the appendix are as set forth in the claims attached to the application of this application.
[Appendix]
<Claim 1>
A sales processing device that performs product sales processing by specifying a product type by image recognition,
When performing the sales process of the product by specifying the product type by the image recognition, product individual information based on individual differences in the appearance of the product acquired by the image recognition, the product type, Storage control means for controlling to store in the storage means in association with the sales point situation regarding the sales point based on the sales of the product;
With respect to a plurality of products of the same type stored in the storage unit, a determination unit that determines the degree of popularity of each individual product by referring to the sales point situation of the individual product;
Output means for outputting the product individual information of the individual product corresponding to the degree of popularity determined by the determination unit, or feature information based on the product individual information;
A sales processing apparatus comprising:
<Claim 2>
The storage control unit controls the storage unit to store the individual feature amount of the product calculated from the recognized image as the product individual information.
The determination unit classifies a plurality of products of the same type stored in the storage unit into a plurality of groups having features common to each group based on the individual feature amount,
By referring to the situation at the point of sale, determine the popularity for each of the plurality of groups,
The output means outputs the representative image for each of the plurality of groups as feature information based on the product individual information together with the popularity degree determined by the determination means.
The sales processing apparatus according to claim 1, wherein:
<Claim 3>
The determination unit classifies a plurality of products of the same type stored in the storage unit into a plurality of groups divided by a sales time zone based on the sales point situation,
The output means outputs the representative image for each of the plurality of groups as feature information based on the product individual information, together with information that the popularity is higher as the sales time zone is earlier,
The sales processing apparatus according to claim 1, wherein:
<Claim 4>
It is a product popularity analysis method for performing product sales processing by specifying a product type by image recognition,
When performing the sales process of the product by specifying the product type by the image recognition, product individual information based on individual differences in the appearance of the product acquired by the image recognition, the product type, A storage step for storing in the storage means in association with the sales point situation regarding the sales point based on the sales of the product;
For a plurality of products of the same type stored in the storage means, a determination step of determining the popularity of each product by referring to the sales point situation of each product;
An output step for outputting the product individual information of the individual product corresponding to the degree of popularity determined in the determination step, or feature information based on the product individual information;
A product popularity analysis method characterized by including:

1 Sales processing device 101 CPU (an example of control means and storage control means)
102 RAM
103 ROM
104 storage unit (an example of storage means)
105 Object Recognition Data 106 Product Content Data 107 Sales Master 11 Customer Display 12 Touch Display 12A Display 12B Touch Panel 13 Cash Drawer 14 Printer 15 Imaging Device (Example of Imaging Unit)
151 Shooting area 16 Shooting stand 17 Backlight light source 18 Speaker 2 Counter stand 3 Tray 5, 5A to 5D Anpan (an example of a product)
6 Display stand 9 Processing unit 91 Image storage unit 92 Object recognition processing unit 921 Object detection unit 922 Similarity calculation unit 923 Similarity determination unit 93 Confirmation notification unit 94 Candidate product presentation unit 95 Input acquisition unit 96 Sales registration unit 97 Information output unit (Example of output means)
98 popularity analysis processing part (an example of determination means)
99 Data update unit for object recognition

Claims (4)

A sales processing device that performs product sales processing by specifying a product type by image recognition,
When performing the sales process of the product by specifying the product type by the image recognition, product individual information based on individual differences in the appearance of the product acquired by the image recognition, the product type, Storage control means for controlling to store in the storage means in association with the sales point situation regarding the sales point based on the sales of the product;
With respect to a plurality of products of the same type stored in the storage unit, a determination unit that determines the degree of popularity of each individual product by referring to the sales point situation of the individual product;
Output means for outputting the product individual information of the individual product corresponding to the degree of popularity determined by the determination unit, or feature information based on the product individual information;
A sales processing apparatus comprising: The storage control unit controls the storage unit to store the individual feature amount of the product calculated from the recognized image as the product individual information.
The determination unit classifies a plurality of products of the same type stored in the storage unit into a plurality of groups having features common to each group based on the individual feature amount,
By referring to the situation at the point of sale, determine the popularity for each of the plurality of groups,
The output means outputs the representative image for each of the plurality of groups as feature information based on the product individual information together with the popularity degree determined by the determination means.
The sales processing apparatus according to claim 1, wherein: The determination unit classifies a plurality of products of the same type stored in the storage unit into a plurality of groups divided by a sales time zone based on the sales point situation,
The output means outputs the representative image for each of the plurality of groups as feature information based on the product individual information, together with information that the popularity is higher as the sales time zone is earlier,
The sales processing apparatus according to claim 1, wherein: It is a product popularity analysis method for performing product sales processing by specifying a product type by image recognition,
When performing the sales process of the product by specifying the product type by the image recognition, product individual information based on individual differences in the appearance of the product acquired by the image recognition, the product type, A storage step for storing in the storage means in association with the sales point situation regarding the sales point based on the sales of the product;
For a plurality of products of the same type stored in the storage means, a determination step of determining the popularity of each product by referring to the sales point situation of each product;
An output step for outputting the product individual information of the individual product corresponding to the degree of popularity determined in the determination step, or feature information based on the product individual information;
A product popularity analysis method characterized by including:

Images