JP6695454B1 - Information processing apparatus, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve recognition accuracy when recognizing a product from a product image. A product recognition process is performed based on a first model generated by learning using a plurality of product images labeled with product identification information of each product and group identification information of a group including a plurality of similar products. And a second recognition processing unit that performs recognition processing of similar products based on a second model generated by learning using product images of a plurality of similar products, and a first product image is acquired. If the image acquisition unit and the recognized product are similar products, the group identification information of the group including the similar product is acquired. If the recognized product is not the similar product, the product identification information of the recognized product is acquired. A first acquisition unit that acquires the product identification information of the recognized product from the second recognition processing unit when the group identification information is acquired; and an output unit that outputs the product identification information, Equipped with. [Selection diagram] Fig. 5

Description

  The present disclosure relates to an information processing device, an information processing method, and a program.

  In recent years, a system for recognizing a product using a product image has been developed. For example, Patent Document 1 discloses a technique of recognizing whether or not a product identification code is attached to a product image, and suspending recognition of the product if the product identification code is not attached.

JP, 2018-45494, A

  However, even if the product identification code is assigned to all products, for example, when a large number of products are handled, the recognition accuracy is lowered due to the image capturing angle of the product image and obstacles (for example, background color and hands). There is a problem that it will end up.

  The present invention has been made in view of the circumstances described above, and provides an information processing device, an information processing method, and a program capable of improving recognition accuracy when a product is recognized from a product image. One of the purposes.

  An information processing apparatus according to an embodiment of the present disclosure includes a first model for product recognition generated by learning using a plurality of product images labeled with product identification information of each product, and a plurality of similar products. A first recognition processing unit having one or a plurality of first recognition units that perform product recognition processing based on the group identification information of the included group, and similar product recognition generated by learning using product images of the plurality of similar products. A second recognition processing unit having one or a plurality of second recognition units for performing recognition processing of similar products, an image acquisition unit for acquiring a first product image to be recognized, and the first recognition unit. If the recognized product is the similar product, the group identification information of the group including the similar product is acquired and recognized from the first recognition processing unit that has recognized the product of the first product image using the model. If the purchased product is not the similar product, a first acquisition unit that acquires product identification information of the recognized product, and if the group identification information is acquired, the first product image using the second model Second acquisition unit that acquires the product identification information of the recognized product from the second recognition processing unit that has recognized the product, and the product identification information acquired by the first acquisition unit or the second acquisition unit. And an output unit for outputting.

  According to the disclosed technology, it is possible to improve the recognition accuracy when recognizing a product from a product image.

It is a figure which shows schematic structure of the recognition system 1 which concerns on this embodiment. It is a figure which shows an example of the hardware constitutions of the information processing apparatus 20 which concerns on this embodiment. It is a figure which shows an example of a functional structure of the learning device 104 which concerns on this embodiment. It is a figure which shows an example of image correction. It is a figure which shows an example of a functional structure of the recognition apparatus 106 in this embodiment. It is a figure for showing a learning target part of a similar product. It is a flow chart which shows an example of the whole processing of the system in an embodiment. It is a flow chart which shows an example of learning processing in an embodiment. It is a flow chart which shows an example of recognition processing in an embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The same elements will be denoted by the same reference symbols, without redundant description.

A. This Embodiment In the present embodiment, a system for recognizing a product from a product image is assumed in a large-scale mass retailer, a supermarket, or the like, but it is also applicable to any store that sells the product (a convenience store, a specialty store, etc.). Further, the invention is not limited to recognition of merchandise and can be applied to recognition of an object included in a captured image. Hereinafter, the object will be described by taking a product as an example, but the object is not limited to this.

(1) Configuration <system configuration>
FIG. 1 is a diagram showing a schematic configuration of a recognition system 1 according to this embodiment. As shown in FIG. 1, the recognition system 1 includes a first imaging device 102, a learning device 104, a recognition device 106, and a second imaging device 108. Further, all or part of the above-mentioned devices are connected to each other via a communication network so that they can communicate with each other. The communication network is, for example, the Internet, a LAN, a dedicated line, a telephone line, a corporate network, a mobile communication network, Bluetooth (registered trademark), WiFi (Wireless Fidelity), any other communication line, or a combination thereof. However, it does not matter whether it is wired or wireless. The recognition system 1 may be configured such that the first imaging device 102 and the learning device 104 are the first system in the learning phase, and the recognition device 106 and the second imaging device 108 are the second system in the recognition phase.

  The first imaging device 102 is a device that can image a product from an arbitrary angle. The captured product image is transmitted to the learning device 104. Further, the first image pickup device 102 may not be able to take an image from an arbitrary angle, but the user may take an image from an arbitrary angle. The first imaging device 102 images the product from various angles and acquires a large number of captured images so that the learning device 104 can create a three-dimensional model that can recognize the product from any angle. You can For example, when the learning device 104 generates a three-dimensional model, as an example, about 500 captured images are captured for each product.

  The learning device 104 acquires captured images (commodity images) of a plurality of products from the first imaging device 102, performs learning on the plurality of captured images, and generates a learning model for product recognition. In the learning of the present embodiment, for example, it is preferable to use deep learning (deep learning) in order to accurately learn a large number of products, but the learning is not limited to this. The learning device 104 has a learning unit that generates a learning model in units of a predetermined number of products. For example, a learning model for 10,000 products is generated by one learning unit. The learning unit can be configured by a GPU (Graphics Processing Unit) or the like. Further, in the learned product image, product identification information (hereinafter, also referred to as “product ID”) is labeled as a correct answer for each product, and the image is corrected as necessary. The product ID is, for example, a JAN code.

  The learning device 104 also creates a similar product list in which similar products are grouped. In the similar product list, product IDs of similar products are grouped, and group identification information (hereinafter, also referred to as “group ID”) is given to each group. Similar products can be automatically specified by using, for example, that the similarity between images of products is equal to or more than a threshold value. The learning device 104 outputs the generated learning model and the similar product list to the recognition device 106 for each learned unit. Further, the learning device 104 performs machine learning, preferably deep learning, on the similar portion of the similar product to generate a learning model for the similar product. Detailed functions of the learning device 104 will be described with reference to FIG.

  The recognition device 106 has a recognition unit for each learning unit of the learning device 104, and each recognition unit stores a learning model and a similar product list. For example, the recognition unit can be configured by a combination of GPU and CPU (Central Processing Unit). The recognition device 106 performs a primary recognition process on the product image of the target product acquired from the second imaging device 108. The recognition device 106 acquires the product ID of the product recognized by using the plurality of recognition units. If the acquired product ID is included in the similar product list, the recognition device 106 performs secondary recognition processing for similar products, and performs more accurate product recognition. Detailed functions of the recognition device 106 will be described with reference to FIG.

  The second imaging device 108 is, for example, a camera, images a product, and outputs the captured image to the recognition device 106. The second imaging device 108 is used, for example, to image a product at a cash register or to manage a product displayed on a shelf.

<Hardware configuration>
FIG. 2 is a diagram illustrating an example of a hardware configuration of the information processing device 20 according to the present embodiment. The information processing device 20 can be implemented as hardware of the learning device 104 or the recognition device 106. As shown in FIG. 2, the information processing device 20 includes a processor 202, a memory 204, a storage 206, an input / output interface (input / output I / F) 208, and a communication interface (communication I / F) 210. . The respective components of the HW of the information processing device 200 are connected to each other via a bus B, for example.

  The information processing apparatus 20 realizes the functions and / or methods described in the present embodiment by the cooperation of the processor 202, the memory 204, the storage 206, the input / output I / F 208, and the communication I / F 210. To do.

  The processor 202 executes the function and / or the method realized by the code or instruction included in the program stored in the storage 206. The processor 202 is, for example, a central processing unit (CPU), MPU (Micro Processing Unit), GPU, microprocessor (microprocessor), processor core (processor core), multiprocessor (multiprocessor), ASIC (Application-Specific Integrated Circuit), Includes FPGA (Field Programmable Gate Array), etc.

  The memory 204 temporarily stores the program loaded from the storage 206 and provides a work area to the processor 202. Various data generated while the processor 202 executes the program is also temporarily stored in the memory 204. The memory 204 includes, for example, a RAM (Random Access Memory) and a ROM (Read Only Memory).

  The storage 206 stores a program executed by the processor 202 and the like. The storage 206 includes, for example, a HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, and the like.

  The input / output I / F 208 includes an input device that inputs various operations to the information processing device 20, and an output device that outputs a processing result processed by the information processing device 20. In the input / output I / F 208, the input device and the output device may be integrated, or the input device and the output device may be separated.

  The input device is realized by any of all types of devices capable of receiving an input from a user and transmitting information related to the input to the processor 202, or a combination thereof. The input device includes, for example, a hardware key such as a touch panel, a touch display, and a keyboard, a pointing device such as a mouse, a camera (operation input via an image), and a microphone (operation input by voice).

  The output device is realized by any of all types of devices capable of outputting the processing result processed by the processor 202, or a combination thereof. In the case of outputting the processing result as a video and / or a moving image, the output device may be any one or all of the devices capable of displaying the display data according to the display data written in the frame buffer. It is realized by combination. The output device includes, for example, a device capable of displaying images, text information, and the like on a touch panel, a touch display, a monitor (for example, a liquid crystal display, an OELD (Organic Electroluminescence Display), etc.), a speaker (voice output), and the like.

  The communication I / F 210 transmits and receives various data via the network. The communication may be executed by wire or wireless, and any communication protocol may be used as long as mutual communication can be executed. The communication I / F 210 has a function of executing communication with another information processing device via a network. The communication I / F 210 transmits various data to another information processing device according to an instruction from the processor 202. Further, the communication I / F 210 receives various data transmitted from another information processing device, and transmits the data to the processor 202.

  The program of this embodiment may be provided in a state of being stored in a computer-readable storage medium. The storage medium can store the program on a “non-transitory tangible medium”. The programs include, for example, software programs and computer programs.

  At least a part of the processing in the information processing device 20 may be realized by cloud computing including one or more computers. At least part of the processing in the information processing device 20 may be configured to be performed by another information processing device. In this case, at least part of the processing of each functional unit realized by the processor 202 may be configured to be performed by another information processing device.

<Functional configuration of learning device>
FIG. 3 is a diagram showing an example of a functional configuration of the learning device 104 according to the present embodiment. In the example illustrated in FIG. 3, the learning device 104 includes an image correction unit 302 and a deep learning unit 304. The image correction unit 302 and the deep learning unit 304 can be realized by, for example, the processor 202 shown in FIG. 2 and the memory 204 as a work area.

  The image correction unit 302 performs correction processing on the product image acquired from the first imaging device 102 as necessary. The correction process includes cutting out an image of a product portion from the product image and labeling the partial image of the product with product identification information such as a JAN code as a correct answer for learning. In addition, the correction process may include correcting the background, adding an image of a hand, combining with another product, and the like. The image correction unit 302 outputs the corrected image to the deep learning unit 304.

(Example of image correction)
Here, an example of image correction in the present embodiment will be described. FIG. 4 is a diagram showing an example of image correction. The product image A1 is a partial image (hereinafter also referred to as “partial image A1”) in which a product part is cut out from the captured product image.

  Images C1 to C3 are product images obtained by combining the partial image A1 with different background colors. For example, the image C1 is an example in which the partial image A1 is combined with a black background color. The image C2 is an example in which the red background color is combined with the partial image A1. The image C3 is an example in which the green background color is combined with the partial image A1. Accordingly, even if the background of the captured product image is an arbitrary color, the product can be appropriately recognized, and the recognition accuracy can be improved.

  Images H1 to H3 are product images obtained by combining the partial image A1 with the image of the hand. For example, the image H1 is an example in which the hand holds the lower portion of the partial image A1. The image H2 is an example in which the hand holds the middle of the partial image A1. The image H3 is an example in which the hand holds the upper portion of the partial image A1. With this, by combining the captured product image with the image of the hand, the product can be properly recognized even when the product is captured while the user holds the product in the hand, and the recognition accuracy is improved. Can be planned.

  Images P1 to P3 are product images obtained by combining the partial image A1 with other product images. For example, the image P1 is an example in which the partial image A1 is combined with two other products (for example, “product M”). The image P2 is an example in which the partial image A1 is combined with two other products (hereinafter, referred to as “product N”). The image P3 is an example in which the partial image A1 is combined with one product M1 and one product N1. Accordingly, even if the captured product image includes another product image, the product can be appropriately recognized, and the recognition accuracy can be improved.

  As described above, since the image correction unit 302 corrects the image, it is possible to create a learning model in advance by assuming various scenes at the time of imaging. This makes it possible to improve the recognition accuracy of the product even when the product is captured in different scenes.

  Returning to FIG. 3, the deep learning unit 304 performs deep learning (deep learning) on the product image acquired from the image correction unit 302 to generate a learning model for product recognition. The deep learning unit 304 includes, for example, a GPU control unit 310 and a learning processing unit 312 including a plurality of learning units (1, 2, 3, ...). Each learning unit can be realized by, for example, a GPU.

The GPU control unit 310 controls the learning of each learning unit. As described above, in each learning unit, for example, product images of 10,000 products are input, and the learning model (hereinafter, also referred to as “first model”).
) Are generated respectively. A similar product list is attached to the generated learning model. The similar product list can be generated by the deep learning unit 304 by discriminating similar products using the degree of similarity between product images, grouping similar products, and assigning a group ID to the group.

  In addition, the GPU control unit 310 causes the learning unit to perform deep learning by using a portion of the product image of the similar product that is similar to another product. The learning processing unit 312 uses the partial image of the similar product to generate a learning model for the similar product (hereinafter, also referred to as “second model”). A product ID is labeled on the partial image of the similar product. The GPU control unit 310 outputs the first model generated by each learning unit or the second model generated by the learning processing unit 312 to the recognition device 106.

<Functional configuration of recognition device>
FIG. 5 is a diagram showing an example of a functional configuration of the recognition device 106 in the present embodiment. In the example illustrated in FIG. 5, the recognition device 106 includes a first control unit 502, a first recognition processing unit 504, and a second recognition processing unit 506. The first control unit 502, the first recognition processing unit 504, and the second recognition processing unit 506 can be realized by, for example, the processor 202 shown in FIG. 2 and the memory 204 as a work area.

  The first control unit 502 includes an image acquisition unit 512, a first acquisition unit 514, a determination unit 516, a second acquisition unit 518, and an output unit 520. The image acquisition unit 512 acquires the first product image to be recognized from the second imaging device 108. The image acquisition unit 512 outputs the acquired first product image to the first recognition processing unit 504.

  The first recognition processing unit 504 includes a first model for product recognition generated by learning using a plurality of product images labeled with product identification information of each product, and a group of groups including a plurality of similar products. It has a plurality of recognition units that perform product recognition processing based on the ID. Each recognition unit may be realized by a combination of GPU and CPU. Further, each recognition unit may be realized by software, and in this case, it is possible to allow one GPU to execute a plurality of recognition units, and to increase the processing capacity by increasing the number of GPUs. You can Also, the number of recognition units to be executed can be controlled.

  It should be noted that each recognition unit performs recognition processing (primary recognition processing) on the acquired first product image using the first model held by itself. Each recognition unit acquires, as a recognition result, the product ID of the product closest to the first product image and the similarity thereof. At this time, when the product ID is included in the similar product list, the group ID of the group including the product ID is included in the recognition result. As the similarity, a known similarity such as a square error between images may be used. The first recognition processing unit 504 outputs the product ID or group ID having the highest degree of similarity to the first acquisition unit 514.

  The first acquisition unit 514 acquires the group ID of the similar product from the first recognition processing unit 504 if the recognized product is a similar product, and if the recognized product is not a similar product, the recognized product. To acquire the product ID. For example, the first acquisition unit 514 acquires, from the first recognition processing unit 504, the product ID or the group ID having the highest degree of similarity in the recognition result output by each recognition unit.

  The determination unit 516 determines whether the ID acquired by the first acquisition unit 514 is a product ID or a group ID. If the determination result is the product ID, the determination unit 516 outputs the product ID to the output unit 520. Further, the determination unit 516 outputs the product image to the second recognition processing unit 506 if the determination result is the group ID.

  The second recognition processing unit 506 performs second product recognition processing (secondary recognition processing) based on a second model for similar product recognition generated by learning using product images of a plurality of similar products. It has one or more units 534 (recognition units X1, X2, ...). The second recognition processing unit 506 also includes a second control unit 532 that controls the recognition unit for similar products. The second control unit 532 acquires the recognition result from each recognition unit. The recognition result includes a product ID and information indicating an error when the product recognition is not properly performed (hereinafter, also referred to as “error information”).

(Learning target for similar products)
Here, FIG. 6 is a diagram for showing a learning target portion of the similar product. In the example shown in FIG. 6, the products S1 to S3 are substantially the same except for the partial images R102, R104, and R106. These products S1 to S3 are assigned to one group as similar products and are given a group ID. The 2nd recognition part 534 has a learning model learned using each partial image R102, R104, and R106, for example.

  At this time, the second control unit 532 specifies which part of the product is similar to the first product image to be recognized from the group ID and cuts out the partial image of the similar part. The second recognition unit 534 performs product recognition using the partial image to be recognized and the second model, and includes the product ID of the product having the highest similarity and the similarity in the recognition result. As a result, the second recognition unit 534 for similar products can perform the recognition process using the partial image, that is, can perform the recognition process using the portion in which the feature appears, and further improve the recognition accuracy. be able to.

  Returning to FIG. 5, the second control unit 532 outputs error information to the second acquisition unit 518 if the degree of similarity is less than the predetermined threshold value, and matches if the degree of similarity is greater than or equal to the predetermined threshold value. The product ID is output to the second acquisition unit 518. For example, the case where the degree of similarity is less than a predetermined threshold value is a case where the partial image is hidden by a hand or the like and the degree of similarity is reduced without performing appropriate recognition processing. At this time, since the appropriate recognition process is not executed, the second control unit 532 returns the error information.

  The second acquisition unit 518 acquires the recognition result of the similar product from the second recognition processing unit 506. The recognition result includes the product ID or error information. The second acquisition unit 518 outputs the product ID or the error information to the output unit 520.

  The output unit 520 outputs the product ID or the error information acquired by the first acquisition unit 514 or the second acquisition unit 518. For example, when the recognition device 106 is a cashier device, the output unit 520 outputs the product ID to the settlement unit (not shown) so that the amount corresponding to the product ID is displayed or added to the payment amount of the user. It will be done. Further, the output unit 520 may output error information to an output device and call an attendant. This allows the attendant to recognize the product that cannot be recognized by using the scanning device or the like as usual.

(2) Description of Operation Next, the operation of the recognition system 1 according to the embodiment will be described.
<Overall processing of system>
FIG. 7 is a flowchart showing an example of the overall processing of the system in the embodiment. In the example illustrated in FIG. 7, in the recognition system 1, in step S102, the first imaging device 102 captures a product image of a product to be learned. A large amount of products may be imaged from various angles so that a three-dimensional model can be created in learning, which is the latter process.

  In step S104, the learning device 104 performs a learning process, for example, deep learning, on many product images acquired from the first imaging device 102.

  In step S106, the recognition device 106 uses the model learned by the learning device 104 on the product image captured by the second imaging device 108 to perform recognition processing for recognizing the product. As a result, the product can be identified from the captured product image using the model generated by deep learning.

<Learning process>
Next, the learning process of the learning device 104 in this embodiment will be described. FIG. 8 is a flowchart showing an example of the learning process in the embodiment.

  In step S202, the image correction unit 302 acquires a product image from the first imaging device 102.

  In step S204, the image correction unit 302 performs a correction process on the acquired product image. The correction process includes, for example, cutting out an image of a product portion from the product image and labeling the partial image of the product with product identification information such as a JAN code. In addition, the correction process may include correcting the background, adding an image of a hand, combining with another product, and the like.

  In step S206, the deep learning unit 304 performs deep learning (deep learning) on the product image acquired from the image correction unit 302 to generate a learning model for product recognition. The deep learning unit 304 executes the primary learning for the corrected image and the secondary learning for the partial image of the similar product.

  In step S207, the deep learning unit 304 includes a similar product list in which similar products are grouped in the learned model.

  Thus, by performing deep learning on the corrected image, it is possible to deal with various imaging scenes and improve the recognition accuracy. In addition, for similar products, by cutting out partial images that show features and performing deep learning using the partial images, it is possible to generate a learning model that focuses on different parts. Can be improved.

<Recognition processing>
Next, the recognition process of the recognition device 106 in this embodiment will be described. FIG. 9 is a flowchart showing an example of the recognition process in the embodiment.

  In step S302, the second imaging device 108 images the product to be recognized and generates a first product image.

  In step S304, the image acquisition unit 512 acquires the first product image to be recognized from the second imaging device 108.

  In step S306, the first recognition processing unit 504 outputs the first product image to each recognition unit (1, 2, 3, ...) And acquires the recognition result from each recognition unit. The recognition result includes the product ID of the product having the highest degree of similarity and the degree of similarity. Further, when the product ID of the product having the highest degree of similarity is included in the similar group, each recognition unit includes the group ID of this group in the recognition result. In addition, the first recognition processing unit 504 identifies the product ID or the group ID having the highest similarity among the similarities included in each recognition result.

  In step S308, the first acquisition unit 514 acquires the recognition result from the first recognition processing unit 504. The recognition result includes the product ID or the group ID.

  In step S310, the determination unit 516 determines whether the ID acquired by the first acquisition unit 514 is a product ID or a group ID. If the determination result is the product ID (step S310-NO), the process proceeds to step S318. If the determination result is the group ID (step S310-YES), the process proceeds to step S312.

  In step S312, the second recognition processing unit 506 performs recognition processing of similar products using the second model for identifying the products from the product images of the plurality of similar products. At this time, a portion where the feature appears may be cut out as a partial image from the first product image. The portion in which the feature appears may be held in the second recognition processing unit 506 in association with the group ID.

  In step S314, the second acquisition unit 518 acquires the recognition result of the similar product from the second recognition processing unit 506.

  In step S316, the second acquisition unit 518 determines whether the information included in the acquired recognition result is the product ID or the error information. If the acquired information is the product ID (step S316-YES), the process proceeds to step S318, and if the acquired information is error information (step S316-NO), the process proceeds to step S320.

  In step S318, the output unit 520 outputs the product ID acquired by the first acquisition unit 514 or the second acquisition unit 518.

  In step S320, the output unit 520 outputs the error information.

  Accordingly, it is possible to perform a two-step recognition process including a normal product recognition process and a product recognition process using a partial image in which a characteristic portion appears, and it is possible to improve product recognition accuracy. ..

  As described above, according to the recognition system 1 according to the present embodiment, in the learning stage, products having similar product images are grouped in advance, and learning processing is performed using partial images showing features. You can keep running. At the recognition stage, the recognition accuracy is improved by performing a primary recognition process using a product image and a secondary recognition process using a partial image of a similar product when the primary recognition process cannot appropriately recognize the product. Can be improved.

  Also, in the learning stage, by cutting out a partial image of the product portion with respect to the product image, and learning by combining various backgrounds to this partial image, even when the recognition target product is imaged in various situations, Appropriate product recognition can be performed.

B. Others The present invention is not limited to the above-described embodiment, and can be implemented in various other forms without departing from the scope of the present invention. Therefore, the above embodiments are merely examples in all respects, and should not be construed as limited. For example, the processing steps described above can be arbitrarily changed in order or executed in parallel as long as the processing contents do not conflict.

  Further, although the system for recognizing a product has been described as an example in the above-described embodiment, the present invention can be similarly applied to a system for identifying an object from a captured image. For example, it can be applied to face authentication and biometric authentication such as fingerprint authentication.

  Further, in the above-described present embodiment, the recognition system 1 can be applied to a real-time cashier system. For example, the recognition device 106 processes the product images captured by the second imaging device 108 in parallel in each recognition unit to recognize the product. It is possible to specify the price of the product using the product ID of the recognized product. Moreover, only the necessary learning model generated by each learning unit can be transmitted to the recognition device 106. As a result, when a certain store has a small number of items, it is possible to prevent unnecessary models from being sent and reduce the processing load.

  Further, in the above-described embodiment, the present invention can be applied to the case where one store is provided with the recognition device 106 and a plurality of cash register devices including the second imaging device 108 are provided. The recognition device 106 acquires the product image from each cash register device. Then, the product ID or the price associated with the product ID is returned to the cash register device. Thereby, the introduction cost when the recognition system 1 is introduced into a store can be reduced.

  Further, in the above-described embodiment, the recognition device 106 is an in-store device, but the recognition device 106 is installed on the cloud server side, and the second imaging device 108 is installed on the client side as the cash register device. It is possible to execute the above-mentioned processing even when it is installed in.

1 ... Recognition system, 20 ... Information processing device, 102 ... First imaging device, 104 ... Learning device, 106 ... Recognition device, 108 ... Second imaging device, 202 ... Processor, 204 ... Memory, 206 ... Storage, 208 ... Input Output I / F, 210 ... Communication I / F, 302 ... Image correction unit, 304 ... Deep learning unit, 310 ... GPU control unit, 312 ... Learning processing unit, 502 ... First control unit, 504 ... First recognition processing unit , 506 ... Second recognition processing unit, 512 ... Image acquisition unit, 514 ... First acquisition unit, 516 ... Judgment unit, 518 ... Second acquisition unit, 520 ... Output unit, 532 ... Second control unit, 534 ... Second Recognition section.

Claims (7)

Product recognition based on a first model for product recognition generated by learning using a plurality of product images labeled with product identification information of each product and group identification information of a group including a plurality of similar products A first recognition processing unit having one or a plurality of first recognition units for performing processing;
A second recognition processing unit having one or a plurality of second recognition units that perform recognition processing of similar products based on a second model for similar product recognition generated by learning using product images of the plurality of similar products;
An image acquisition unit that acquires a first product image to be recognized,
If the recognized product is the similar product, the group identification information of the group including the similar product is acquired from the first recognition processing unit that has recognized the product of the first product image using the first model. If the recognized product is not the similar product, a first acquisition unit that acquires product identification information of the recognized product,
When the group identification information is acquired, a second acquisition unit that acquires product identification information of the recognized product from the second recognition processing unit that has recognized the product of the first product image using the second model. When,
An output unit that outputs the product identification information acquired by the first acquisition unit or the second acquisition unit;
An information processing apparatus including.   The information processing apparatus according to claim 1, wherein the first model is a model learned by a combination of an image of a product portion in the product image and an arbitrary background image of a plurality of background images.   The information processing apparatus according to claim 2, wherein the plurality of background images include at least one of a background image having a different color, an image of a hand, and images of other products.   The information processing apparatus according to claim 1, wherein the second model is a model learned by using a partial image similar to another product in the product image of the similar product.   The information processing apparatus according to any one of claims 1 to 4, wherein the first model includes a three-dimensional model capable of recognizing a product from an arbitrary angle. Product recognition based on a first model for product recognition generated by learning using a plurality of product images labeled with product identification information of each product, and group identification information of a group including a plurality of similar products A first recognition processing unit having one or a plurality of first recognition units for performing processing;
A second recognition processing unit having one or a plurality of second recognition units that perform recognition processing of similar products based on a second model for similar product recognition generated by learning using product images of the plurality of similar products;
An information processing method executed by an information processing device including a control unit,
The control unit is
Acquire the first product image to be recognized,
If the recognized product is the similar product, the group identification information of the group including the similar product is acquired from the first recognition processing unit that has recognized the product of the first product image using the first model. If the recognized product is not the similar product, obtain product identification information of the recognized product,
When the group identification information is acquired, the product identification information of the recognized product is acquired from the second recognition processing unit that has recognized the product of the first product image using the second model,
An information processing method for outputting the acquired product identification information. Product recognition based on a first model for product recognition generated by learning using a plurality of product images labeled with product identification information of each product and group identification information of a group including a plurality of similar products A first recognition processing unit having one or a plurality of first recognition units for performing processing;
A second recognition processing unit having one or a plurality of second recognition units that perform recognition processing of similar products based on a second model for similar product recognition generated by learning using product images of the plurality of similar products;
A program to be executed by an information processing device having a control unit,
In the control unit,
Acquire the first product image to be recognized,
If the recognized product is the similar product, the group identification information of the group including the similar product is acquired from the first recognition processing unit that has recognized the product of the first product image using the first model. If the recognized product is not the similar product, obtain product identification information of the recognized product,
When the group identification information is acquired, the product identification information of the recognized product is acquired from the second recognition processing unit that has recognized the product of the first product image using the second model,
A program for executing a process of outputting the acquired product identification information.

Images