JP2023169922A - Information processing system and control method thereof, and program - Google Patents

Abstract

To provide a mechanism for further accurately recognizing an object to be recognized, possibly including an object different from the object to be recognized therein.SOLUTION: An information processing system comprises acquisition means for acquiring a container area including a container out of an image, processing means for applying specific processing of reducing a difference from other images to a partial area inside the container area acquired by the acquisition means, and control means for executing control to perform learning processing using the image processed by the processing means.SELECTED DRAWING: Figure 6

Description

The present invention relates to a technique for recognizing an object to be recognized included in an image using an image.

Conventionally, a trained model is generated by machine learning using images (training images, teacher data) that contain the object to be identified, and the object contained in the image is recognized by inputting the image to the generated trained model. There are known techniques to do this.

Prior Art Document 1 proposes to improve the detection accuracy of unknown objects (such as objects not in training images) by generating a mask image in which an image is separated into an object region and a background region. When generating a mask image, each pixel of the frame image is assigned "1" (a value corresponding to white) or "0" (a value corresponding to black) to the pixel, and the object area (white range) and It is disclosed that the image is classified into a background area (black area) and a background area (black area).

JP 2022-14263 Publication

A possible use case would be to recognize tableware from an image when checking out after a meal at a cafeteria, and then perform accounting processing according to the recognized tableware. In this way, when tableware is used as an object of image recognition, if there is leftover food in the tableware, recognition accuracy decreases. That is, if something different from the object to be recognized exists inside the object to be recognized, this may become a factor in reducing recognition accuracy. Prior Art Document 1 does not take into account the possibility that something different from the object to be recognized exists inside the object to be recognized.

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide a mechanism that can more accurately recognize an object to be recognized, which may have something different from the object to be recognized inside.

acquisition means for acquiring a container region including the container in the image;
processing means that performs specific processing on a partial area inside the container area acquired by the acquisition means to reduce differences from other images;
A control means for controlling the learning process to be performed using the image processed by the processing means.

According to the present invention, it is possible to more accurately recognize an object to be recognized in which there may be something different from the object to be recognized inside.

FIG. 1 is a diagram illustrating a system to which the information processing device according to the present embodiment can be applied. FIG. 2 is a diagram showing an example of the hardware configuration of various devices. It is a flowchart which shows an example at the time of AI learning. It is a flowchart which shows an example at the time of AI inference. It is a figure explaining an example of tableware detection. It is a figure explaining an example of the tableware blackening method.

Embodiments of the present invention will be described in detail below with reference to the drawings.

First, with reference to FIG. 1, an example of the configuration of an information processing system according to an embodiment of the present invention will be described.

In the information processing system of the present invention, a cafeteria payment lane 102 composed of a camera 103, a display 104, and a payment desk 105 is connected to a client terminal from a predetermined controller 106 (e.g., PoE hub) via a network 107 (e.g., Ethernet). 101 and configured to be communicably connected. Note that a plurality of cafeteria payment lanes 102 may be connected to the client terminal 101.

The camera 103 is installed at a position where the entire tray of the checkout table 105 can be photographed.

On the checkout table 105, a tray with after-meal dishes placed on it is placed for payment. Note that the tray on which the tableware is placed may be in a pre-meal state.

The client terminal 101 is, for example, a personal computer (hereinafter referred to as a PC), identifies tableware from an image captured by a camera 103, and performs processing such as payment. The client terminal 101 uses deep distance learning (Deep Metric Learning) technology to identify the type of tableware placed on the payment table 105.

Deep distance learning is a method that extracts only the features of an image, uses an algorithm to calculate the image feature vector from the extracted features, and measures the distance to determine which product is closest. Sample images are prepared in advance, and feature vectors are extracted from each image. For the input image, the distance between each sample image and the special energy vector is measured, and it is determined that the sample is of the same type as the closest sample. Although this embodiment will be explained using deep distance learning, other methods such as deep learning classification may also be used.

The display 104 displays information on the payment processed by the client terminal 101, and instructs the payer of the meal to settle the payment. Note that an image from the camera 103 may be displayed on the display 104.

Next, with reference to FIG. 2, an example of the configuration of the client terminal 101 as an example of a device to which the present invention can be applied is shown.

In FIG. 2, a CPU 201, memory 202, nonvolatile memory 203, image processing unit 204, display 205, operation unit 206, recording medium I/F 207, external I/F 209, and communication I/F 210 are connected to an internal bus 250. ing. Each unit connected to the internal bus 250 is configured to be able to exchange data with each other via the internal bus 250.

The memory 202 is composed of, for example, a RAM (volatile memory using a semiconductor element, etc.). The CPU 201 controls each part of the client terminal 101 according to a program stored in the nonvolatile memory 203, for example, using the memory 202 as a work memory. The nonvolatile memory 203 stores image data, audio data, other data, various programs for the CPU 201 to operate, and the like. The nonvolatile memory 203 is composed of, for example, a hard disk (HD) or a ROM.

Based on the control of the CPU 201, the image processing unit 204 processes image data stored in the nonvolatile memory 203 and the recording medium 208, video signals acquired via the external I/F 209, and image data acquired via the communication I/F 210. Performs various image processing on image data, captured images, etc. Image processing performed by the image processing unit 204 includes A/D conversion processing, D/A conversion processing, image data encoding processing, compression processing, decoding processing, enlargement/reduction processing (resizing), noise reduction processing, and color conversion. This includes processing, etc. The image processing unit 204 may be configured with a dedicated circuit block for performing specific image processing. Further, depending on the type of image processing, the CPU 201 may perform image processing according to a program without using the image processing unit 204. The CPU 201 performs the process of recognizing the object (tableware) to be recognized from the image in cooperation with the image processing unit 204 .

The display 205 displays images, a GUI screen forming a GUI (Graphical User Interface), etc. under the control of the CPU 201 . The CPU 201 generates a display control signal according to a program, and controls each part of the client terminal 101 to generate a video signal to be displayed on the display 205 and output it to the display 205. Display 205 displays video based on the output video signal. Note that the client terminal 101 itself includes an interface for outputting a video signal to be displayed on the display 205, and the display 205 may be an external monitor (such as a television).

The operation unit 206 is an input device for receiving user operations, including a character information input device such as a keyboard, a pointing device such as a mouse or a touch panel, a button, a dial, a joystick, a touch sensor, a touch pad, and the like. Note that the touch panel is an input device that is configured in a planar manner so as to be superimposed on the display 205, and outputs coordinate information according to a touched position.

A recording medium 208 such as a memory card, CD, or DVD can be attached to the recording medium I/F 207, and based on the control of the CPU 201, data can be read from the attached recording medium 208 and data can be written to the recording medium 208. I do. The external I/F 209 is an interface for connecting to external equipment via a wired cable or wirelessly, and for inputting and outputting video signals and audio signals. The communication I/F 210 is an interface for communicating with external devices, the Internet 211, etc., and transmitting and receiving various data such as files and commands.

The camera unit 212 is a camera unit that includes an image sensor (imaging sensor) that includes a CCD or CMOS device that converts an optical image into an electrical signal.

Next, with reference to FIG. 3, basic processing of learning processing related to tableware recognition (learning processing using AI: Artificial Intelligence) in the embodiment of the present invention will be described. Note that the processing of each step is executed by the CPU 201 of each device. The process shown in FIG. 3 is started when the client terminal 101 studies an image as a process before the restaurant customer uses the cafeteria payment lane 102.

In S<b>301 , the CPU 201 cuts out and acquires an image of each tableware using a circumscribing rectangle from the image including the tableware photographed by the camera 103 , and stores the image in the recording medium 208 . Specifically, the area of tableware is detected from the image taken by the camera 103. This detection process is different from the detection of the type of tableware, which will be described later, and is a detection process that allows it to be determined that there is a tableware (or that there is an object other than a tray), although the type is unknown. A rectangle (hereinafter referred to as a circumscribed rectangle) is set for the area of the tableware detected in this detection process so as to be in contact with the outer shape of the tableware. A partial image of the set circumscribed rectangular area (that is, an image including a single tableware) is cut out and acquired from the original image, and is stored in the recording medium 208.

In S302, the CPU 201 sets the ratio of blacking out a portion of the inside of the cutout image acquired and saved in S301. At this time, the percentage of blacking can be set arbitrarily, but if it is set to 100%, the entire area will be filled in, so it should be less than 100%. Furthermore, the color to be filled is not limited to black, and other colors may be used.

In S303, the CPU 201 blacks out a portion of the inside of each cutout image (area including tableware) acquired and saved in S301 based on the ratio set in S302, and saves it in the recording medium 208. This blacking process is similar to the blacking process in the inference phase described later with reference to FIG. However, in the process of S303, which is the training data creation process, an image of only tableware without cooking (food) or leftover food may be used. Even if it is an image of just a tableware, by learning images with the inside painted black, the inside part of the tableware, which may contain an image of leftover food rather than the tableware itself, is not considered as a feature during the inference phase. It is possible to have children learn a lot. That is, the process of S303 is a process of processing a portion of the inside of the container area containing the tableware to reduce the difference from other images.

In S304, the CPU 201 saves both the original image cut out by the circumscribed rectangle (the cutout image before blacking, the teacher image acquired in S301) and the blacking image (teacher image) saved in S303, and the tableware of each image. A trained model is created by performing a learning process using label information indicating the type of the model. The created trained model is recorded on the recording medium 208. Note that, without using the original image, only the black painted image may be used as the teacher image for learning. In that case, it is preferable that the images used in the inference phase are also only black images.

The above is the explanation of FIG. 3.

Next, with reference to FIG. 4, an example of tableware recognition processing in this embodiment will be described. This process is an inference phase process using the trained model generated in the learning process of FIG. 3, and is a process performed when a customer of the cafeteria uses the cafeteria payment lane 102. Note that the processing of each step is executed by the CPU 201. An example will be described in which the same client terminal 101 performs the processing in FIG. 3 and the processing in FIG. 4. However, if the trained model generated in the processing in FIG. 3 is used, the processing in FIG. The processing may be executed by a separate information processing device (for example, a PC) from the client terminal 101 that performs the processing.

In S<b>401 , the CPU 201 uses the camera 103 to photograph the area of the payment desk 105 . When photographing the checkout table with the camera 103, the camera 103 may continue photographing at all times, or may start photographing when some moving object is detected within the photographing range.

In S402, the CPU 201 executes tray placement determination processing to determine whether the tray is placed within a predetermined range from the photographed image. If it is determined in S403 that a tray is placed, tableware position detection processing is performed in S404, and if it is determined that a tray is not placed, tray placement determination in S402 is executed again.

In S404, the CPU 201 uses the camera 103 to take a picture, and from the captured image, similarly to S301, the CPU 201 cuts out and obtains an image for each tableware using a circumscribed rectangle. FIG. 5 shows an example of an image taken by the camera 103. The photographed image 501 shows a tray 502 placed on the checkout table 105 and tableware 503a to 503d. The positions of the dishes on the tray are detected, and circumscribed rectangles 504a to 504d are calculated for each dish. Note that FIG. 5 is an example of an image of tableware with no leftover food, but if there is leftover food, the leftover food will be shown inside each tableware.

In S405, the CPU 201 blacks out the inside of the partial image for each tableware acquired in S404 at a preset ratio.

FIG. 6 shows an example of a method for blackening tableware. Cutout images 601a to 601c cut out using the circumscribed rectangle (container area) include dishes 602a to 602c, leftover food 603a to 603c, and dirt, respectively. Since leftover food and dirt affect the accuracy of tableware identification, a process is performed to black out areas that are unnecessary for image recognition, such as leftover food and dirt. In this embodiment, a case will be described in which the blacking ratio is set to 50%. For the cutout images 601a to 601c cut out with the circumscribed rectangle, an ellipse (if the circumscribed rectangle is a square, it becomes a perfect circle) is painted black from the center of each at a ratio of 50% horizontally and 50% vertically. Regions 604a to 604c are generated. Regarding the horizontally elongated tableware 602b and the rectangular tableware 602c, if black is painted at a ratio of 50% horizontally and 50% vertically from the center (C2, C3) of the circumscribed rectangle (cutout images 601b, 601c), as shown in the figure. Horizontally elongated elliptical black areas 604b and 604c as shown above are generated. In this way, by blacking out unnecessary portions such as leftover food and dirt, it is possible to prevent leftover food from being erroneously detected and improve recognition accuracy. Furthermore, the area to be painted black changes (such as a perfect circle or a horizontally long ellipse) depending on the shape of the area where the presence of tableware is recognized (detected) (the shape of the cutout images 601a to 601c). In other words, the area where there is a high possibility that something different from the object to be detected, such as food, is placed is painted black according to the shape of the tableware, making it possible to detect the type of tableware that is the object of detection with higher accuracy. It becomes possible. Note that the unnecessary part is not limited to leftover food or dirt, as long as it has no characteristics (unnecessary part) when detecting a certain object.

In S406, the CPU 201 uses AI to determine the type of tableware. Specifically, the processed cutout image created in S406 is input to the trained model created in S304 (the trained model stored in the recording medium 208), and inference processing is performed. If a plurality of cutout images have been acquired in S404, inference processing is performed on all of them. As a result of the inference process, scores for each of a plurality of tableware types (probability for the corresponding tableware type) are output for each cutout image. The CPU 201 extracts those whose scores exceed a predetermined threshold and sets them as candidate types for the determination result. The number of types extracted as candidate types may be zero, one, or multiple.

In S407, it is determined whether a candidate type has been extracted as a result of the inference process in S406. If one or more candidate types have been extracted, the process advances to S408; if not, that is, if the candidate types are 0 (there is no type whose score exceeds the threshold), the process advances to S414.

The processes of S408 to S412 are performed for each candidate type. Hereinafter, an example will be described in which three candidate types, ie, a soup bowl, a rice bowl, and a grilled fish plate, are extracted in step S406 for one cutout image. In this case, the processes of S408 to S412 are performed for each of the soup bowl, tea bowl, and grilled fish plate.

In S408, the CPU 201 acquires a sample image corresponding to the candidate type extracted in S406 and to be processed in S408. The sample image is an image included in correct data (teacher data) of tableware that can be a possible detection result, and is an image that has been recorded in advance on the recording medium 208 in S301.

In S409, the CPU 201 executes a process of comparing the aspect ratio of the cutout image (circumscribed rectangle), which is the recognition target image from which the candidate type was obtained, with the aspect ratio of the sample image obtained in S408.

In S410, the CPU 201 determines whether the difference in aspect ratio is within an allowable range as a result of the comparison in S409. If it is within the allowable range, the process advances to S411, and if it is outside the allowable range, the process advances to S414. For example, in a sample image of a grilled fish plate, it is assumed that the aspect ratio of the circumscribed rectangle of the tableware is 2:3 of the horizontal length. On the other hand, if the aspect ratio of the cutout image (circumscribed rectangle) that is the recognition target image from which the candidate type of grilled fish plate was obtained is 1:1, then the aspect ratio of grilled fish plate is outside the allowable range. Therefore, the determination in this step is No, and the grilled fish dish is excluded from the candidate types.

In S411, the CPU 201 executes a process of comparing the size of the cutout image (circumscribed rectangle), which is the recognition target image from which the candidate type was obtained, with the size of the sample image obtained in S408. Specifically, the areas (number of pixels) are compared. Processing is performed to compare the area (number of pixels) of the circumscribed rectangle detected in the tableware position detection in S404 with the area (number of pixels) of the candidate of the sample image group narrowed down in S410.

In S412, the CPU 201 determines whether the size difference is within an allowable range as a result of the comparison in S411. If it is within the allowable range, the process advances to S413, and if it is outside the allowable range, the process advances to S414. For example, assume that the size of the sample image of a tea bowl is size 2, which is larger than size 1 of the sample image of a soup bowl. On the other hand, if the size of the cutout image (circumscribed rectangle) that is the recognition target image from which the candidate type of teacup is obtained is size 1, and the difference between size 1 and size 2 exceeds the allowable range. For example, the determination in this step is No, and the tea bowl is excluded from the candidate types. In this way, even tableware with similar shapes may have different sizes, so a process is performed to compare the sizes of the tableware and exclude tableware of different sizes from candidates. For example, there are various sizes of bowls, from large ones to small ones, and in order to identify them, candidates can be narrowed down by comparing the areas of images of tableware.

In S413, the CPU 201 determines whether all candidate types have been processed. If all have been processed, the process advances to S415; otherwise, the process advances to S408 to process the next candidate type.

In S414, the CPU 201 excludes the candidate type to be processed from the candidates. In other words, the type is not determined as a recognition result.

In S415, the CPU 201 determines whether there is any type among the candidate types extracted in S406 that was not excluded from the candidates in the processes from S408 to S414. If it exists, the process advances to S416; if it does not exist (all types have been excluded), the process advances to S417.

In S416, the CPU 201 identifies one tableware type with the highest score among the remaining types that were not excluded from the candidates in the processing from S408 to S412 among the candidate types extracted in S406, and selects it as the recognition result. Determine. That is, one type of tableware is specified for one container area.

On the other hand, in S417, the CPU 201 determines that the tableware to be detected is an unregistered tableware (unregistered article). In that case, unregistered goods will be processed so as not to be included in accounting. For example, if a towel or the like other than tableware is placed on the tray, it will be recognized as an unregistered item and will not be included in the bill. Furthermore, a notification such as "Unknown" may be given to the article so that it can be identified as an unregistered article.

When the type of tableware is thus specified in S416 and S417, the CPU 201 refers to that day's menu (menu) information and acquires the dish (menu) and price corresponding to the specified tableware. When the dishes and prices corresponding to all the tableware included in one tray image are acquired, the display 104 is controlled to display each dish name, price, and total amount as a detection result. Thereafter, in response to the user's payment operation, payment is made with the displayed total amount.

The above is the explanation of FIG. 4.

As explained above, according to the present embodiment, an image in which a partial area inside the tableware that is likely to contain something different from the tableware to be recognized (food) is painted black is trained. Also, use it for reasoning. By doing so, the blacked-out area has the same feature (the feature of being completely black) regardless of the type of tableware image. Therefore, in the learning phase, the blacked out area is not an area where data indicating features (differences) effective for determining the type of tableware can be obtained, so it is difficult to judge the type of tableware with respect to the blacked out area. A trained model with a low degree of basis will be generated. In the trained model generated in this way, even if there is any foreign object in the area of the tableware that corresponds to the area that was painted black at the time of learning, the effect on the judgment of the type of tableware is low. That is, the possibility of making a wrong judgment due to the influence of leftover food is reduced, and it becomes possible to discriminate the type of tableware with higher accuracy. The trained model generated in this way will not be blacked out during training even if an image with leftover food is input as it is without blacking out as the detection target image for the type of tableware used for inference. The area corresponding to the area where the tableware was used is of a low degree of use as a basis for determining the type of tableware. In other words, there is no or only limited reduction in recognition accuracy due to the influence of images of leftover food. Therefore, the effect of improving recognition accuracy can be obtained by simply performing learning using an image in which part of the inside of the tableware image is painted black as a teacher image during learning, as shown in Figure 3, without performing blacking during inference. be able to. Therefore, the blacking process of S405 in FIG. 4 does not need to be performed. Accordingly, the processing time and processing load required during inference can be reduced, and inference results can be notified with high responsiveness. Of course, higher accuracy can be expected by performing the process of S405.

Note that in the above-described embodiment, an example has been described in which the ratio of blacking out the inside of the tableware is set for an image cut out by the circumscribed rectangle of the area where the tableware is, but the present invention is not limited to this. AI may be used to extract only the uneaten portion of the detected tableware, and only that area may be painted black.

Alternatively, the following processing may be performed. After the processing in S404, S405 is omitted, and tableware type determination in S406 is performed on the image of the container region of the recognition target image that is not blacked out and detected in S404. Then, a plurality of sample images corresponding to the plurality of candidate types (pre-stored images of tableware without foreign objects such as leftover food) and a recognition target image without blacking detected in S404 are used. Extract each difference between the image of the container area and the image of the container area. Then, among the images of the container area of the recognition target image that is not blacked out and detected in S404, the difference area that is the difference (that is, the area that is not tableware and is estimated to be leftover food, etc.) is blacked out. Images are created for a plurality of tableware candidate types, and the tableware type determination in S406 is performed once again on these images. As a result, one type of tableware with the highest score may be determined as the type of the detected tableware. That is, it is also possible to specify the uneaten portion from the difference between the detection target image (recognition target image) and the candidate image, black out the leftover portion, and perform inference processing.

In addition, in the above-mentioned embodiment, when painting the tableware black as shown in FIG. 6, an example was explained in which the black painting is done in an oval shape, but the invention is not limited to this. The shape of the black coating may be changed to match the shape of the tableware. For example, if the shape of the tableware is a rectangle 606, the black shape may be made into a rectangle to match the shape.

Furthermore, we have explained an example in which a part of the inside of the tableware is painted black, where there is a high possibility that something different from the recognition target tableware (food) will be captured, but this reduces the possibility that it will become the basis for recognition. The processing is not limited to this, as long as it is possible to reduce characteristics (differences) between images. For example, an area described as being painted black may be filled with any single color, such as all white or all blue. Alternatively, it may be replaced with a simple pattern image, such as black and gray appearing alternately every few dots.

Further, the area described as being painted black may be a non-image area without an image. An image with a hole in the center excluding the area described as being painted black that is included in the circumscribed rectangle of the tableware may be used for learning and inference. Note that if the shape of the tableware in the image of tableware used during learning differs from the shape of the tableware in the image of tableware used during inference, there is a possibility that tableware that should be of the same type will be determined to be of a different type. For example, if the system learns images of dishes with a hole in the center, there is a higher possibility that images without a hole in the center will be determined to be different dishes during inference. From that point of view, if the process is filled with a single color or a pattern during learning, the shape itself will not change, increasing the degree of freedom of the image used during inference (it is also effective to use the original image without blacking). can make inferences). If you try to use a blank area instead of blacking out, the type of tableware included in the image to be detected is not accurately known before inference, so during inference, the same shape as during learning and no image will be used. It is difficult to isolate areas. In other words, it is assumed that replacing the area with a pattern without characteristics, such as black painting, is more effective than leaving the area as a no-image area.

In addition, although the above-mentioned embodiment explained the example of recognizing the type of tableware, the recognition is not limited to the recognition of the type of tableware, and it is possible that there is something different from the detection target inside the detection target. Applicable to For example, it can be applied to generate or infer a trained model that determines the type of pot itself from an image, regardless of the food or cooking inside the pot. It also recognizes (detects) containers used for experiments such as beakers and petri dishes, containers for medicines, cosmetics, food, drinks, etc. (bottles, cups, etc.), and cargo containers (wooden boxes, cardboard, resin containers, etc.). It is also applicable when Both contribute to highly accurate recognition (detection) of the container itself, regardless of the contents of the container.

The present invention can be implemented as, for example, a system, an apparatus, a method, a program, a recording medium, or the like. Specifically, the present invention may be applied to a system consisting of a plurality of devices, or may be applied to a device consisting of a single device.

Note that the various controls described above as being performed by the CPU 201 may be performed by a single piece of hardware, or multiple pieces of hardware (for example, multiple processors or circuits) may share the processing to control the entire device. You may do so.

Further, although the present invention has been described in detail based on its preferred embodiments, the present invention is not limited to these specific embodiments, and the present invention may be applied to various forms without departing from the gist of the present invention. included. Furthermore, each of the embodiments described above is merely one embodiment of the present invention, and it is also possible to combine the embodiments as appropriate.

Furthermore, in the above-described embodiments, the present invention has been described as an example in which the present invention is applied to a PC, but the present invention is not limited to this example, and can be applied to any device that can generate a blacked-out image. That is, the present invention is applicable to PDAs, mobile phone terminals (smartphones), tablet terminals, and the like.

(Other embodiments)
The present invention is also realized by performing the following processing. That is, the software (program) that realizes the functions of the embodiments described above is supplied to a system or device via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or device reads the program code. This is the process to be executed. In this case, the program and the storage medium storing the program constitute the present invention.

101 Client terminal 107 Network

Claims (17)

acquisition means for acquiring a container region including the container in the image;
processing means that performs specific processing on a partial area inside the container area acquired by the acquisition means to reduce differences from other images;
An information processing system comprising: a control means for performing a learning process using the image processed by the processing means. 2. The information processing system according to claim 1, wherein the control means performs control to perform learning processing using an image processed by the processing means and an image not processed by the processing means. Information processing according to claim 1, characterized in that the control means performs control so that the learning process is performed using images processed by the processing means without using images that have not been processed by the processing means. system. The partial area is an area corresponding to the object based on the difference between an image in which an object different from the container is inside the container area and an image in which an object different from the container is not inside the container area. The information processing system according to claim 1, characterized in that: The information processing system according to claim 1, wherein the learning process is a process of generating a trained model for identifying the type of container from an image. The control means inputs a recognition target image showing a container to the trained model generated in the learning process, and performs control to perform an inference process to determine the type of container included in the image. The information processing system according to claim 5. The processing means also performs the specific processing on a container region including the container in the recognition target image,
7. The information processing system according to claim 6, wherein the control means controls the image processed by the processing means to be input to the learned model and perform inference processing. acquisition means for acquiring a container region including the container in the image;
processing means that performs specific processing on a partial area inside the container area acquired by the acquisition means to reduce differences from other images;
An information processing system comprising: control means for controlling to perform inference processing using the image processed by the processing means. When the type of container specified by the inference process satisfies at least one of a condition related to aspect ratio and a condition related to size, the control means performs the inference process as the type of container included in the image to be recognized. 9. The information processing system according to claim 6, wherein the information processing system is controlled to output information indicating the type of container specified by. 2. The processing means performs the same processing as the specific processing on each partial region of the plurality of container regions acquired by the acquisition means from one or more images. 8. The information processing system according to any one of 8. 9. The information processing system according to claim 1, wherein the specific processing is a process of filling with a specific color or pattern. The information processing system according to claim 11, wherein the specific color is black. 9. The information processing system according to claim 1, wherein the processing means performs the specific processing on a part of a predetermined area from the center of the container area. 9. The information processing system according to claim 1, wherein the processing means performs the specific processing on a partial region having a different shape depending on the shape of the container region. an acquisition step of acquiring a container region including the container in the image;
a processing step of performing specific processing on a partial area inside the container area acquired in the acquisition step to reduce differences from other images;
A control method for an information processing system, comprising: a control step for performing a learning process using the image processed in the processing step. an acquisition step of acquiring a container region including the container in the image;
a processing step of performing specific processing on a partial area inside the container area acquired in the acquisition step to reduce differences from other images;
A control method for an information processing system, comprising: a control step for performing inference processing using the image processed in the processing step. A program for causing at least one computer to function as each means of the information processing system according to any one of claims 1 to 8.

Images