JP2023026342A - Information processor, information processing method, and program - Google Patents

Abstract

To provide a classification system in which a time required for processing of identification information reading for an article group in which identification information positions or types are different is reduced, algorithm is simple, and addition of a new classification target is made to be easier.SOLUTION: A general type of an object to which identification information is assigned is determined from a first imaged picture obtained by imaging the object. On the basis of the general type of the object is set an imaging condition for imaging a picture for obtaining the identification information. The identification information is identified from a second imaged picture obtained by imaging the object on the set imaging condition and a detailed type of the object is determined on the basis of a result of the identification.SELECTED DRAWING: Figure 3

Description

The present invention relates to an information processing device, an information processing method, and a program.

2. Description of the Related Art A system has been devised in which an image of an article is captured by a camera and the article is classified by image processing when sorting articles such as recycled articles. Japanese Unexamined Patent Application Publication No. 2002-200002 discloses a system that captures images of a waste home appliance from the top and side directions and determines the type of the waste home appliance based on three-dimensional image information obtained from these imaging signals.

JP-A-11-83461

In the system described in Patent Document 1, although it is possible to determine general types such as televisions, washing machines, refrigerators, etc., it is not possible to perform more detailed classification.

On the other hand, it is conceivable to classify the article in detail by reading the information written on the label attached to the article. However, there is a problem that image processing tends to take a long time when trying to read descriptions on labels with a single image processing for a group of articles having different label positions, sizes, types of description formats, and the like. Moreover, in this case, the algorithm becomes complicated, and there is a high possibility that addition of new classification targets becomes difficult.

Embodiments of the present invention provide a sorting system that reduces the processing time of reading identification information for a group of items with different locations or types of identification information, simpler algorithms, and easier addition of new classification targets.

In order to achieve the object of the present invention, for example, an information processing apparatus according to one embodiment has the following configuration. That is, acquisition means for acquiring first information related to the identification information from a captured image obtained by capturing an object to which identification information is attached according to a first imaging condition; setting means for setting a second imaging condition for obtaining second information; obtaining the second information from a captured image obtained by imaging the object according to the second imaging condition; and a first classification means for identifying the identification information based on the identification information and classifying the object based on the identification result.

It is possible to provide a classification system that reduces processing time for reading identification information in a group of articles having different identification information positions or types, and that has a simple algorithm and makes it easier to add new classification targets.

FIG. 2 is a block diagram showing the configuration of the information processing device of the classification system according to the first embodiment; FIG. FIG. 4 is a diagram showing an example of an imaging mechanism in the classification system according to the first embodiment; FIG. 4 is a diagram showing an example of objects and labels in the classification system according to the first embodiment; 4 is a flowchart showing an example of information processing according to the first embodiment; 4A and 4B are diagrams showing examples of settings of imaging conditions by the information processing apparatus according to the first embodiment; FIG. 4A and 4B are diagrams showing examples of settings of imaging conditions by the information processing apparatus according to the first embodiment; FIG. 4A and 4B are diagrams for explaining pattern matching processing according to the first embodiment; FIG. FIG. 10 is a diagram showing an example of objects and labels in the classification system according to the second embodiment; FIG. 9 is a diagram showing an example of identification and classification processing according to the second embodiment; 10 is a flowchart showing an example of information processing according to the third embodiment; 10 is a flowchart showing an example of information processing according to the fourth embodiment; 14 is a flowchart showing an example of information processing according to the fifth embodiment; The figure which shows the concept of the determination result by a score. The figure which shows the characteristic of a classification means. The figure which shows the characteristic of a classification means. The figure which shows the characteristic of the classification means which changed distribution. 13 is a flowchart showing an example of information processing according to the sixth embodiment;

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. In addition, the following embodiments do not limit the invention according to the scope of claims. Although multiple features are described in the embodiments, not all of these multiple features are essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant description is omitted.

[Embodiment 1]
The information processing apparatus according to the present embodiment determines the general type of the object from the first captured image obtained by capturing the object to which the identification information is attached. Next, based on the determined general type, by setting imaging conditions for imaging an image for obtaining identification information, and identifying the identification information from a second captured image obtained by imaging the object under the set imaging conditions, Determine the detailed type of the object.

FIG. 1 is a block diagram showing an example configuration of a classification system including an information processing apparatus according to this embodiment. Here, the classification system according to the present embodiment has a first processing section (previous stage) and a second processing section (posterior stage), and these processing sections can basically perform the same processing. be.

The first processing unit includes an imaging unit 011 , an imaging control unit 021 , an image processing unit 031 and a light source sensor 091 . The image capturing unit 011 is a camera unit that captures an image of an article (object) and acquires image data. The imaging control unit 021 controls imaging by the imaging unit 011 . In the example of FIG. 1, the imaging control unit 021 causes the imaging unit 011 to perform imaging (for example, after a predetermined delay) when receiving a signal from the light source sensor 091 that detects articles to be classified. good too. A display unit 06 is a display (installed in a PC, a camera, or the like) and displays the result of processing. The operation unit 07 is a functional unit that receives user input, and may be, for example, a keyboard and mouse, a touch panel, or mechanical buttons of a camera. The PLC 08 is a control unit, and can control processing according to the present embodiment, such as processing by the second processing unit. The storage unit 04 is a memory that stores the results of processing by each functional unit, images captured by the imaging unit, or tables used in processing to be described later, and also functions as a work memory.

FIG. 2 is a diagram for explaining imaging processing by each imaging unit according to the present embodiment. In this example, the imaging unit 011 first captures an image of a recycled product, which is a classification target (article) flowing on the belt conveyor 10, according to a detection signal from the light source sensor 091. FIG. In this example, the image capturing unit 011 captures the image from above the recycled product (the upper surface side of the belt conveyor 10). The image processing unit 031 according to the present embodiment acquires information regarding the identification information of the article from the image of the article captured by the imaging unit 011 . Here, the identification information is characters or symbols for identifying an article, and is information that enables classification of the article by being recognized and read by image processing. In this embodiment, the identification information is described on a label attached to the article. The description form is not particularly limited as long as it can be read in association with.

For example, if a label is affixed to the side of an article, it is conceivable that it is easier to grasp the description on the label if the image is taken from the side rather than from the top side of the article. Further, for example, if the label has a darker color than the general one, it is conceivable to make it easier to grasp the description on the label by adjusting the imaging exposure conditions. From such a point of view, the information processing apparatus according to the present embodiment sets an imaging condition (label imaging condition) for making it possible to more accurately identify the description of the label, that is, the identification information, from the image captured by the imaging unit 011. set. The image processing unit 031 according to the present embodiment uses a learned model to set the label imaging conditions based on the features of the article (and the label, etc.) in the image. Any method may be used to set the label imaging conditions as long as the imaging conditions suitable for identifying the label can be set.

The image processing unit 031 sets label imaging conditions based on information acquired from the captured image. The image processing unit 031 according to the present embodiment can set the label imaging condition based on the feature amount such as the shape of the imaged article. For example, the image processing unit 031 can determine the general type of the article based on the characteristics of the article by inputting the captured image to the trained model, and set the label imaging condition based on the general type. In this way, by determining the label imaging conditions according to the shape of the article, it is possible to determine the position of the articles that have the same general type, that is, that have the same label position, size, and shape, and that are classified into detailed types based on the identification information. , size and shape of the label can be easily recognized.

Here, a table showing the correspondence relationship between the general types of goods and the label imaging conditions is prepared in advance, and the image processing unit 031 refers to the table based on the classification result of the general types to determine the label imaging conditions. shall be set. However, the learning model may output some or all of the label imaging conditions, and arbitrary processing may be performed as long as the label imaging conditions are set based on the output results of the learning model. In this embodiment, the label imaging condition is the imaging condition by the imaging unit. range of RoI to perform).

An example of determining the label imaging condition from the shape of the article as described above will be described below. FIG. 3 is a diagram for explaining recycled products (goods) and labels in this embodiment. FIG. 3(a) shows how labels 1111, 1121, 1131, and 1141 are attached to the recycled products 111 to 114, respectively. Recycled products have different sizes, and also different label positions and different label sizes.

Different kinds of labels 11111 to 11114 are shown in FIG. 3(b). The label according to the present embodiment describes information such as a character string or a symbol used for classifying the attached article. Here, for articles for which the same label imaging conditions are set, the character display position, the number of characters, the character font, etc. in the label are substantially the same (the width may be given), and the contents of the characters described respectively are different. Labels 11111 to 11114 are examples of variations of the label 1111 attached to the recyclable product 111, and labels for which the same label imaging conditions are set may have a plurality of types. Also, each label may additionally include a specific symbol in addition to letters, such as label 11114 .

In the recycled products 112 to 114 as well, there are a plurality of types of labels for which similar label imaging conditions are set, as in FIG. 3B. Labels 11211, 11311, and 11411 in FIGS. 3(c)-(e) show examples of labels attached to the recycled products 112-114, respectively. In this example, the label 11211 has identification information in white characters on a black background, the label 11311 has identification information in black characters and a barcode on a white background, and the label 11411 has identification information in black characters on a white background. each with a different label size and shape.

Next, in the present embodiment, the image capturing units 0121 to 0123 in the second processing unit capture images of the articles using the set label capturing conditions, and the image processing units 0321 to 0323 classify the items from the captured images. conduct. As described above, in the processing according to the present embodiment, the first processing unit roughly classifies the articles based on the image of the articles and performs the pre-processing of setting label imaging conditions. Then, the second processing unit performs a post-stage process of classifying the articles in detail based on the images obtained by imaging the articles under the label imaging conditions. In the following description, it is assumed that the first processing unit and the second processing unit are separate devices, but these imaging processes may be performed by one imaging unit. Also, although different imaging units are used for the pre-processing and post-processing, the processing of the image processing unit 031 and the processing of the image processing units 0321 to 0323 may be performed by a common processing unit. Furthermore, the number of imaging units is not limited to the example described, and the first processing unit may have a plurality of imaging units. can be one. For example, when a plurality of labels exist for one article, it is conceivable to image each label using a plurality of imaging units in the post-processing.

As described above, the image processing unit 031 sets the label imaging condition based on the information acquired from the first captured image (that is, adjusts the imaging condition so that the label can be easily read). Depending on the imaging environment, the color of the label and the article, etc., changing the exposure conditions may make it easier to read the label. Considering such a case, the image processing unit 031 may change, for example, the imaging exposure condition as the label imaging condition for imaging the second captured image. Here, the exposure condition refers to a gain value, exposure time, lens aperture, light amount of a light source, or the like at the time of imaging. For example, when label 1111 and label 1121 are compared, label 1121 has a darker shade. Therefore, when the image obtained by imaging the label 1111 is input and when the image obtained by imaging the label 1121 is input, the gain value is set higher (for example) when the label 1121 is used. Also, the angle of the light source may be set as the exposure condition.

Also, depending on the size of the label or the size of characters (symbols) written on the label, it may be easier to read the label if the resolution is changed. Therefore, the image processing unit 031 may set the resolution as a label imaging condition. In the following description, this "resolution" includes absolute resolution representing the total number of pixels and relative resolution representing the density of pixels. FIG. 5 is a schematic diagram showing the relationship between the imaging unit and the recycled product when capturing an image of the recycled product. Here, in order to set the label imaging condition based on the shape of the recycled product, the imaging unit 0111 captures the first captured image so that the entire recycled product 111 is included in the imaging range. On the other hand, when the identification information written on the label is read, it is sufficient that the entire label is within the angle of view. Therefore, here, the label imaging condition is set to an angle of view that allows the entire label 1111 to be included. . Similarly, in the case of the recycled product 112, the imaging range is set so that the entire label 1121 is within the angle of view. In this way, the image processing unit 031, based on the first captured image of the article, converts the second captured image so that the label can be contained within the angle of view and the identification information can be displayed more clearly. It is possible to set the imaging range.

Also, the image processing unit 031 may set the resolution as the label imaging condition by setting the working distance between the imaging unit and the article. FIG. 6 is a schematic diagram showing an example of changing the distance between the imaging unit and the recyclable product by setting label imaging conditions. When the label 1111 and the label 1141 are compared, the label size of the label 1141 is smaller and the printed characters are finer. Therefore, the image processing unit 031 sets the position of the imaging unit 0121 so that the distance between the imaging unit and the label is closer to the label 1141 than in the case of the label 1111 . According to such a process, it is possible to set the label imaging condition with an increased spatial resolution in order to make it easier to read the description of the label with finer characters. Note that the method of setting the resolution is not particularly limited as long as the reading accuracy of the description can be improved according to the label, and for example, the focal length of the lens may be set. In addition, since the image processing unit 031 only needs to read the identification information from the label portion, the image processing unit 031 may set the extraction position of the Region of Interest (RoI) for classifying the object as the label imaging condition. . That is, a range (partial region) in the captured image to be cut out for label recognition processing may be set as the label imaging condition.

In addition, depending on the position of the label, for example, when an article with a label attached to the side thereof is imaged from above, it may be easier to read the label by changing the imaging position. From such a point of view, the image processing unit 031 may set the imaging position as the imaging condition. In the example of FIGS. 1 and 2, the second processing unit has three imaging units 0121 to 0123, from which one imaging unit is selected according to label imaging conditions, and from the selected imaging unit , an image is acquired for detailed classification of the article. For example, for articles with a label attached to the upper surface, such as the recycle items 111 and 112, detailed classification of the articles is performed using an image captured by the imaging unit 0121 that captures images from above. Further, for articles with labels attached to the side surfaces, such as the recycle product 113, detailed classification of the articles is performed using an image captured by the imaging unit (here, the imaging unit 0122) corresponding to the side surface. will be Note that, as described above, it is not necessary to use a plurality of imaging units for recognizing identification information, and for example, the position and orientation of a single imaging unit may be set as label imaging conditions. Therefore, the label imaging condition may specify the number of imaging units used for recognition of identification information, or may specify the imaging units to be used.

The second processing section captures an image of the label of the article under the label imaging conditions set by the image processing section 031, reads the identification information of the label, and performs detailed classification of the article. In the example of FIGS. 1 and 2, the second processing unit has three imaging units 0121 to 0123 as described above, and each of them has an imaging control unit ( 0221 to 0223), an image processing unit (0321 to 0323), and a light source sensor 092. In this embodiment, as shown in FIGS. 1 and 2, label imaging conditions are first set from the image of the object captured by the imaging unit 0111, and then the article is captured by the imaging units 0121 to 0123 using the label imaging conditions. and classification are performed. The integrated control unit 051 transmits the processing result of the image processing unit 031 to the second processing unit. Hereinafter, the image processing units 0321 to 0323 are collectively referred to as the image processing unit 032, and the image capturing units 0121 to 0123 are collectively referred to as the image capturing unit 012, unless special distinction is required.

The image processing unit 032 images the label of the article under the set label imaging conditions. Here, in the second processing unit, an imaging unit corresponding to the set label imaging conditions may be selected, and each parameter of the imaging unit may be set according to the label imaging conditions. may be done. In addition, the image processing unit 032 uses a plurality of imaging units to image labels of articles under set label imaging conditions, performs detailed classification of articles from the respective captured images, and an integrated determination unit 052 performs a plurality of detailed classifications. The results may be combined to determine the final product subclass. In that case, the integrated determination unit 052 may determine the integrated result based on the score (likelihood, etc.) calculated at the time of each classification, and select the most frequently classified classification among the plurality of classification results as the final classification. may be determined as

The imaging control unit 022 controls imaging by each imaging unit in the same manner as the imaging control unit 021 . Note that the timing of image capturing (the delay time from the detection of the article by the light source sensor 092 to the image capturing) may be set as the label image capturing condition. By changing the imaging timing, it is conceivable that, for example, the degree of reflection of light from the light source on the label changes, making it easier to read the label.

The image processing unit 032 reads the identification information of the label from the image captured by the imaging unit 012 and classifies the articles. The image processing unit 032 can recognize characters and symbols written on the label as identification information, and can finally classify the articles. Here, the image processing unit 032 reads identification information using character reading (OCR) as image processing. The method is not particularly limited as long as it is possible. In addition, the image processing unit 032 reads, as identification information, a (one-dimensional) barcode as described in the label 11311 in FIG. Classification of articles may be performed. Here, the image processing unit 032 may switch the identification format of the identification information according to the label imaging conditions. That is, the image processing unit 032 reads characters or symbols from a label, reads a bar code, reads a QR code, or The identification format can be switched, such as whether two or more of Also, assuming that the presence or absence of a specific symbol described on the label 11114 is related to the detailed classification of the article, it is determined whether or not to confirm the presence or absence of the specific symbol within the label according to the label imaging conditions. You may As the label imaging condition, a parameter as a condition for identifying the identification information, such as a pattern matching search area, a search angle, or a search threshold when reading the identification information, may be set. In addition, the image processing unit 032 may change the color of the captured image according to the label imaging conditions (or the general type of the article) (for example, any known color processing such as grayscaling is possible). be). The image processing unit 032 holds, for example, a table indicating the correspondence between identification information and detailed types of articles, and can refer to this table to output the corresponding detailed types. These recognition processes are well-known techniques, and detailed description thereof will be omitted.

Here, if the presence or absence of a specific symbol such as that described on the label 11114 is related to the classification of the article, there is a possibility that the surrounding of the symbol will be dirty or that the symbol cannot be read depending on the degree of light reflection. That is, there is a possibility that an article with a symbol is misidentified as an article without a symbol. This is a circumstance peculiar to the case of discrimination based on the presence or absence of symbols. In this case, it is conceivable to set not one but two decision criteria for the score (likelihood, etc.) calculated at the time of classification, and classify into three types: with symbols, without symbols, and undecidable. FIGS. 13A and 13B are diagrams showing the concept of determination results based on scores when there is one determination criterion and when there are two determination criteria, respectively. Such three-way classification has the effect of reducing the probability of erroneously classifying items with symbols as those without symbols, and making it possible to take appropriate measures.

FIG. 7 is a diagram for explaining an example of processing for setting a template and a search range when each of the labels 11111 to 11114 is recognized by pattern matching. In the example of FIG. 7, a character string serving as identification information is set as a template (indicated by a dashed line here), and its search range (indicated by a dashed line here) is set. Here, it is possible to similarly set a template and a search range for a symbol (triangular mark) as indicated by the label 11114, for example. The image processing unit 032 can switch between these template and search range settings by setting label imaging conditions.

FIG. 4 is a flow chart showing an example of processing in the classification system by the information processing apparatus according to this embodiment. The process of FIG. 4 is started when the light source sensor detects an article to be sorted, for example, on the belt conveyor 10 shown in FIG.

In S401, the light source sensor 091 detects an article flowing on the belt conveyor and treats the article as a processing target. In S402, the imaging unit 0111 waits for a predetermined delay from detection by the light source sensor 091, and then images the article. Here, the predetermined delay may be set as any time.

In S403, the image processing unit 031 classifies the general type of the article based on the captured image captured in S402. Here, a captured image is used as an input, and a rough classification of the article type is output from the trained model. In S404, the image processing unit 031 sets label imaging conditions based on the type of article, and the integrated control unit 051 transmits the set label imaging conditions to the second processing unit.

In S405, the light source sensor 092 detects the processing target flowing on the belt conveyor. In S406, the imaging units 0121 to 0123 image the label to be processed according to the label imaging conditions set in S404. In this process, only the imaging unit selected according to the label imaging conditions may perform imaging. Alternatively, all the imaging units may perform imaging regardless of the label imaging conditions, and the label imaging conditions may be used to select an image to be used for classifying the detailed types of articles from among the captured images.

In S407, the image processing unit 032 performs image processing (adjustment of gain value, etc.) according to the label imaging conditions, reads the identification information of the label, and performs detailed classification of the article. In S408, the integrated determination unit 052 determines the detailed type as the final product classification result, transmits the classification result to the PLC 008, and ends the process.

According to such processing, it is possible to first set the imaging conditions for identifying the identification information of the label attached to the article with higher accuracy from the image of the article. Next, it becomes possible to classify the articles in detail based on the identification information of the labels on the articles imaged under the set imaging conditions. Therefore, it is sufficient to perform image processing for detailed classification only from images captured under conditions suitable for label recognition, and to provide a simple algorithm classification system that reduces processing time and enables the addition of new classification targets. becomes possible.

In FIG. 2 and the like, recycled products are used as examples of articles to be classified, but the classification system of the information processing apparatus according to the present embodiment can be used to classify arbitrary articles, such as article classification in distribution warehouses. Is possible.

[Embodiment 2]
For example, as shown in FIG. 8, although the articles have slightly different shapes but are similar (that is, in the processing according to the first embodiment, the same label imaging conditions can be set), the identification information on each article It is assumed that the positions marked with are significantly different. In such cases, it is necessary to perform image processing such that the article can be classified from both of these different labels. The information processing apparatus according to the present embodiment can perform processing similar to that of the first embodiment. In any case, the label imaging conditions are set for performing identification with higher accuracy.

As described above, FIG. 8 is a diagram showing recycled product 111 (label 1111) and recycled product 115 (label 1151) that are similar in shape but different in label position and description. FIG. 8(a) shows the appearance of each recycled product, and FIGS. 8(b) and 8(c) are examples of multiple types of labels included in the labels 1111 and 1151, respectively.

Here, the image processing unit 031 has been learned to output the same label imaging condition regardless of whether the image of the first article (recycled product 111) or the second article (recycled product 115) is input. A model can be used to set label imaging conditions. That is, the label imaging conditions can be set so that both the recognition of the label of the first article and the recognition of the label of the second article are possible. This processing will be described with reference to FIG. FIG. 9A is a diagram showing a case where different label imaging conditions are performed for the recycled product 111 and the recycled product 115 by the processing according to the first embodiment.

FIG. 9(b) is a diagram for explaining a model that has been learned to recognize the item type as "111+115" regardless of whether the recycle item 111 or the recycle item 115 appears in the input captured image. is. In this example, the image processing unit 031 sets label imaging conditions according to "111+115", which is commonly classified for both recycled products 111 and 115. Perform the expected classification process. This "classification process assuming both" is a process in which both can be classified simultaneously or in order, and in this example, it is pattern matching using both patterns.

The image processing unit 031 classifies the first item (recycled item 111) and the second item (recycled item 115) into different types according to the learning model. An assumed label imaging condition may be set. FIG. 9(c) is a diagram for explaining a case in which such an article classification process is performed. In this example, the recycle product 111 and the recycle product 115 are classified as different types by the learned model. Here, the image processing unit 031 performs classification processing assuming both of the classified types of the articles are the recycled products 111 and the recycled products 115 .

According to such a process, even if it is easy to make an error in the identification (classification of the general type of the article) by the model that has already learned from the shape of the article, the imaging condition for accurately performing the classification process of the detailed type of the article can be set.

[Embodiment 3]
The information processing apparatuses according to Embodiments 1 and 2 roughly classify articles by learning model, and set label imaging conditions based on the classification. On the other hand, the information processing apparatus according to the third embodiment identifies the position/type (label position/type) of the identification information from the image by performing object detection using the learned model, and identifies the identified label position/type. The type is used to set label imaging conditions. The information processing apparatus according to the present embodiment has processing units similar to those of the first embodiment, and is capable of performing similar processing, so redundant description will be omitted.

As described above, the learning model used by the information processing apparatus according to the present embodiment outputs the position, type, etc. of the label on the article from the captured image of the article. Therefore, the image processing unit 031 can set label imaging conditions according to the output of the learning model. For example, image resolution can be changed based on label position, or imaging parameters related to focus can be adjusted by image sharpness at label position.

FIG. 10 is a flow chart showing an example of processing in the classification system by the information processing apparatus according to this embodiment. The processing described in FIG. 10 is the same as that in FIG. 4 except that S1001 and S1002 are performed instead of S403 and S404, so redundant description will be omitted.

In S1001, the image processing unit 031 determines the general type of the article from the captured image captured in S402 by object detection using the learning model, and identifies the label position and type of the article. In S1002, the image processing unit 031 sets label imaging conditions using the label position/type identified in S1001, and advances the process to S405.

According to such processing, it is possible to set the label imaging conditions used for classifying the articles by using the position and type of the label specified from the image. Therefore, it is possible to set the label imaging condition by specifying the position and type of the label even for articles with similar shapes (errors are likely to occur in the classification of the type based on the shape). If a plurality of labels are affixed to one article and each label is present on a different surface of the article, each label is imaged by a plurality of imaging units 011, and subsequent processing is performed for each label. may be broken.

[Embodiment 4]
In the information processing apparatus according to Embodiments 1 to 3, the image processing unit 031 is assumed to perform processing using a single learning model, but it is possible to apply this to a plurality of learning models of two or more. is also possible. FIG. 11 is a flow chart showing an example of processing in the classification system by the information processing apparatus according to this embodiment. Note that the description will be made only for the parts unique to the present embodiment.

In this embodiment, a case in which two learning models, learning model A and learning model B, are provided will be described. In S1101, the image processing unit 031 uses the learning model A to classify the type of article from the captured image captured in S402. Also, in S1102, the image processing unit 031 uses the learning model B to classify the type of article from the captured image captured in S402. Next, in S1103, the image processing unit 031 determines a result with a higher score from the classification results of S1101 and S1102, that is, a result with higher classification accuracy, as the outline type. Subsequent processing is the same as in the first embodiment.

According to the processing described above, the type is determined based on the processing by a plurality of learning models, so an effect of improving the classification accuracy can be expected. In this embodiment, the case where two learning models are provided has been described, but three or more learning models may be provided.

[Embodiment 5]
The information processing apparatus according to the fourth embodiment determines a single outline type from a plurality of outline types that are processing results using a plurality of learning models in the image processing unit 031 . On the other hand, in the present embodiment, a plurality of images are captured by the imaging unit 012 using each of a plurality of general types, and the detailed type is determined from the detailed type determination results. FIG. 12 is a flow chart showing an example of processing in the classification system by the information processing apparatus according to this embodiment. Note that the description will be made only for the parts unique to the present embodiment.

In S1201, the image processing unit 031 sets label imaging conditions for each of a plurality of outline types, and the integrated control unit 05 transmits the set label imaging conditions to the second processing unit.

In S1202, the imaging unit 012 performs imaging according to the label imaging conditions based on the general type, which is the discrimination result of the learning model A set in S1201. In S1203, the image processing unit 032 performs image processing (adjustment of gain value, etc.) according to image processing conditions corresponding to the label imaging conditions based on the determination result of learning model A, reads the label identification information, and reads the article. Classify the detailed type of

In S1204, the imaging unit 012 performs imaging according to the label imaging condition based on the general type, which is the discrimination result of the learning model B set in S1201. In S1205, the image processing unit 032 performs image processing (adjustment of gain value, etc.) according to the image processing conditions corresponding to the label imaging conditions based on the determination result of the learning model B, reads the identification information of the label, and identifies the article. Perform detailed classification.

In S1206, the image processing unit 032 determines, as the detailed type, a classification result with a higher score, ie, a higher classification accuracy, from the detailed types that are the classification results of S1203 and S1205. According to the above-described processing, imaging is performed based on a plurality of imaging conditions according to processing results based on a plurality of learning models, and detailed types are determined, so further improvement in classification accuracy can be expected.

[Embodiment 6]
In the information processing apparatus according to the embodiments described above, as a characteristic of the classification processing of articles in the image processing unit 031 and the image processing unit 032, the number of objects to be classified (hereinafter referred to as the number of classification types) is There are a classification process A that has a characteristic that the required processing time is almost constant even if the number of classification types increases, and a classification process B that has a characteristic that the required processing time increases as the number of classification types increases. FIG. 14A shows the characteristic of the classification process A, and FIG. 14B shows the characteristic of the classification process B in a schematic diagram. For example, classification using a learning model is classification processing A, and classification using pattern matching or the like is classification processing B.

For example, it is assumed that the classification processing A is applied to the general type classification processing in the image processing unit 031 and the classification processing B is applied to the detailed type classification processing in the image processing unit 032 . In this case, the total processing time differs depending on whether the total number of target classification types is allocated to classification processing A or classification processing B. FIG. That is, the greater the number of items sorted into the classification process B, the longer the overall processing time. On the other hand, the total processing time is required to be less than the time required for the device. In addition, it is necessary to determine the allocation in consideration of not only the processing time of the classification processing A and the classification processing B cited in this embodiment, but also the ability to achieve the final classification accuracy requirement. 15A and 15B show an example of the case where the total processing time cannot meet the requirements. In FIG. 15A, 1501 indicates the number of classification types assigned to the classification characteristic A. In FIG. In FIG. 15B, 1502 indicates the number of classification types assigned to the classification characteristics B. In FIG. FIG. 15B shows that the requested processing time has been exceeded.

16A and 16B are examples in which the distribution is changed from that of FIGS. 15A and 15B. In FIG. 16A, 1601 indicates the number of classification types newly assigned to the classification characteristic A. In FIG. In FIG. 16B, 1602 indicates the number of classification types newly assigned to the classification characteristic B. In FIG. By reducing the number of classification types from 1502, it is shown that it is less than the required processing time.

In the operation of the apparatus, it is necessary to determine the allocation of all target classification types based on the above. FIG. 17 is a flowchart showing an example of processing in the classification system by the information processing apparatus according to this embodiment. In S1701, all target classification types are determined. In S1702, the integrated control unit 05 sorts out by which of the image processing unit 031 and the image processing unit 032 all classification types are to be classified. In S1703, the integrated control unit 05 calculates the processing time and classification accuracy based on the result of sorting. For the calculation, processing such as trial processing using an image of the article captured in advance can be considered. Also, if the learning model needs to be re-learned, the re-learning is performed, and the result is used for the calculation. Each image processing unit may calculate its own processing time and classification accuracy and notify the integrated control unit 05 of them.

In S1704, the integrated control unit 05 determines whether the calculated processing time and classification accuracy satisfy the requirements. If the request is not satisfied, the process advances to S1507 to sort the number of classification types again. If the request is satisfied, in S1705 the integrated control unit 05 stores the sorting information in the storage unit 04 in preparation for the actual operation. The above process is repeated until the requirements are met.

According to the processing as described above, it is possible to sort the number of classification types according to the characteristics of the classification processing and to achieve the request. Also, it is conceivable that the number of target classification types increases during the operation of the apparatus. Even in such a case, by executing the flowchart of FIG. 17, the operation can be continued after satisfying the increased number of classification types.

(Other examples)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the claims are appended to make public the scope of the invention.

011: imaging unit, 021: imaging control unit, 031: image processing unit, 051: integrated control unit

Claims (26)

general discrimination means for discriminating a general type of an object from a first captured image obtained by capturing an object to which identification information is attached;
setting means for setting imaging conditions for imaging an image for obtaining the identification information based on the general type of the object;
and detailed discrimination means for discriminating the identification information from a second captured image obtained by capturing the object under the imaging conditions, and discriminating a detailed type of the object based on the result of the discrimination. Device. 2. The information processing apparatus according to claim 1, wherein said setting means sets said imaging condition by referring to a table in which said rough type of said object and said imaging condition are associated with each other. The rough discrimination means acquires the features of the object from the first captured image,
A general classification of the object is determined from the characteristics of the object acquired from the first captured image using a learning model trained to classify the object into general classifications based on the characteristics of the object. Item 1. The information processing apparatus according to item 1. 3. The information processing apparatus according to claim 2, wherein the feature of said object is a feature representing the shape of said object. The rough discriminating means obtains a plurality of classification results for the object using a plurality of learning models trained to roughly classify the object in the input image, and obtains the plurality of classification results. 2. The information processing apparatus according to claim 1, wherein the final result of the general type is determined from the. The rough discriminating means derives a plurality of classification results and a plurality of imaging conditions for the rough classification of the object using a plurality of learning models trained to classify the object in the input image into the rough classification. 2. The information processing apparatus according to claim 1, wherein a plurality of detailed types are obtained, and a final result of the detailed types is determined from the plurality of detailed types. The setting means sets imaging conditions for identifying both the identification information attached to the first object and the identification information attached to the second object similar to the first object. 2. The information processing apparatus according to claim 1, wherein the setting is performed. The rough discrimination means discriminates the first object and the second object as common rough types,
8. The information processing apparatus according to claim 7, wherein said setting means sets said imaging conditions based on said common outline type. 2. The information processing apparatus according to claim 1, wherein said imaging condition includes an imaging position corresponding to a position of said identification information in said first captured image. 2. An information processing apparatus according to claim 1, wherein said imaging conditions include imaging exposure conditions. 11. The information processing apparatus according to claim 10, wherein said exposure condition includes at least one of gain, exposure time, lens aperture, amount of light of light source, and angle of light source. 2. The information processing apparatus according to claim 1, wherein said imaging conditions include imaging resolution. The setting means sets the resolution by setting the position or orientation of the imaging unit, setting the focal length of imaging, or setting the range for identifying the identification information. 13. The information processing device according to 12. 2. The information processing apparatus according to claim 1, wherein said imaging condition includes imaging timing. 2. The information processing apparatus according to claim 1, further comprising determining means for determining, based on said general classification, an identification format for identifying said identification information by said detailed determination means. The determination means determines the identification format as a format for reading characters or symbols, a format for recognizing the identification information by pattern matching, a format for reading barcodes, a format for reading QR codes, or a specific symbol 16. The information processing apparatus according to claim 15, wherein the determination is made as a format for confirming the presence or absence of. wherein said determining means classifies the identification results into three categories of presence/absence of a specific symbol and non-identifiable when the identification format is determined to confirm the presence/absence of the specific symbol. Item 17. The information processing device according to item 16. 16. The information processing apparatus according to claim 15, wherein said determining means further determines a parameter for identifying said identification information based on said general type. When the identification format is determined to be a format for recognizing the identification information by pattern matching, the determining means further determines, as the parameter, a search area, a search angle, or a search threshold by the pattern matching. 19. The information processing apparatus according to claim 18, characterized by: The detailed determination means identifies the identification information from an image obtained by changing the color of the captured image of the object captured according to the imaging condition,
16. The information processing apparatus according to claim 15, wherein said determination means determines said color change format in said identification format. A plurality of imaging devices capable of imaging the second captured image are provided, and the imaging condition is the number of imaging devices that capture the second captured image or the imaging devices that capture the second captured image. 2. The information processing apparatus according to claim 1, wherein the designation is . A plurality of imaging devices capable of imaging the second captured image are provided, and the detail determination means selects an imaging device that acquires the second captured image used for identification from the plurality of imaging devices. The information processing apparatus according to claim 1, characterized in that. 2. The information processing apparatus according to claim 1, wherein said identification information is written on a label attached to said object. The method according to claim 1, characterized in that the number of types to be classified by each of said rough discrimination means and said detailed discrimination means is distributed based on classification accuracy and processing time in said rough discrimination means and said detailed discrimination means. The information processing device described. a step of determining a rough type of the object from a first captured image obtained by capturing the object to which the identification information is attached;
setting imaging conditions for capturing an image for obtaining the identification information based on the general type of the object;
and identifying the identification information from a second captured image obtained by capturing the object under the imaging conditions, and determining a detailed type of the object based on the result of the identification. A program for causing a computer to function as each means of the information processing apparatus according to any one of claims 1 to 24.

Images