WO2024214680A1 - 情報処理システムおよびコンピュータプログラム - Google Patents

情報処理システムおよびコンピュータプログラム Download PDF

Info

Publication number
WO2024214680A1
WO2024214680A1 PCT/JP2024/014323 JP2024014323W WO2024214680A1 WO 2024214680 A1 WO2024214680 A1 WO 2024214680A1 JP 2024014323 W JP2024014323 W JP 2024014323W WO 2024214680 A1 WO2024214680 A1 WO 2024214680A1
Authority
WO
WIPO (PCT)
Prior art keywords
response
class
original image
image
classes
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2024/014323
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
一貴 木村
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Intellectual Property Management Co Ltd
Original Assignee
Panasonic Intellectual Property Management Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Panasonic Intellectual Property Management Co Ltd filed Critical Panasonic Intellectual Property Management Co Ltd
Priority to JP2025513952A priority Critical patent/JPWO2024214680A1/ja
Publication of WO2024214680A1 publication Critical patent/WO2024214680A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/55—Clustering; Classification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning

Definitions

  • This disclosure relates to data processing technology, and in particular to information processing systems and computer programs.
  • a video surveillance system has been proposed that has a display control means for displaying a category setting screen for setting categories for events contained in video data, accumulates category information set in response to an operator's operation as learning data, and performs learning processing using the learning data (see, for example, Patent Document 1).
  • a dataset may be used in which the data used for learning (training data) is assigned a class into which the data will be classified. If the accuracy of judgments made by an AI model is low, this may be due to a lack of training data in the dataset, or the class assigned to the training data may be incorrect. However, until now, there have been no adequate technologies proposed to assist in the correction of datasets used for machine learning.
  • the present disclosure has been made based on the inventor's above-mentioned recognition, and one objective is to provide a technology that assists in the correction of datasets for machine learning.
  • an information processing system includes a storage unit that stores a dataset for constructing a classification model, the dataset associating learning data extracted from an original image with a class into which the image is to be classified; a model generation unit that executes machine learning based on the dataset to generate a classification model; a response map generation unit that uses the classification model to generate a response map indicating the degree of response to the class with respect to the original image; a screen provision unit that provides a screen including both the original image and the response map; and a modification unit that modifies the dataset in response to a user operation.
  • any combination of the above components, or any conversion of the expressions of this disclosure between an apparatus, a method, a computer program, or a recording medium having a computer program recorded thereon, is also valid as an aspect of this disclosure.
  • the technology disclosed herein can assist in correcting datasets for machine learning.
  • FIG. 1 is a diagram illustrating a configuration of an information processing system according to an embodiment.
  • FIG. 2 is a block diagram showing functional blocks of the AI processing device of FIG. 1.
  • FIG. 11 is a diagram illustrating an example of model information.
  • 1 is a flowchart showing the operation of an AI processing device of an embodiment.
  • FIG. 5(A) is a diagram showing an example of an original image
  • FIGS. 5(B) to 5(F) are diagrams showing examples of response maps.
  • FIG. 13 is a diagram showing an example of a data set confirmation screen.
  • FIG. 13 is a diagram showing an example of a setting screen for setting the display mode of the entire image display area.
  • 8(A) to 8(F) are diagrams that show images displayed in the entire image display area of the data set confirmation screen.
  • the subject of the device or method disclosed herein is equipped with a computer.
  • the computer executes a program to realize the functions of the subject of the device or method disclosed herein.
  • the computer has a processor that operates according to a program as its main hardware configuration.
  • the type of processor is not important as long as it can realize the functions by executing the program.
  • the processor is composed of one or more electronic circuits including an integrated circuit (IC) or an LSI (Large Scale Integration).
  • IC integrated circuit
  • LSI Large Scale Integration
  • FPGAs Field Programmable Gate Arrays
  • FPGAs which are programmed after the LSI is manufactured, or reconfigurable logic devices that can reconfigure the connections within the LSI or set up circuit partitions within the LSI, can also be used for the same purpose.
  • Multiple electronic circuits may be integrated into one chip or may be provided on multiple chips. Multiple chips may be integrated into one device or may be provided on multiple devices.
  • the program may be recorded on a non-transitory recording medium such as a computer-readable Read Only Memory (ROM), optical disk, or hard disk drive, or may be recorded on a temporary storage medium such as a computer-readable Random Access Memory (RAM).
  • the program may be pre-stored in the recording medium or may be supplied to the recording medium or storage medium via a wide area communication network including the Internet.
  • the information processing system of the embodiment provides a user interface that supports the correction of the training dataset.
  • the AI model can be said to be a mathematical model created by machine learning, and can also be said to be a function approximator.
  • Figure 1 shows the configuration of an information processing system 10 according to the embodiment.
  • the information processing system 10 includes an AI processing device 12 and multiple user terminals 14.
  • Each device shown in Figure 1 is connected via a communication network 16, which may include a LAN, a WAN, the Internet, etc.
  • the AI processing device 12 is an information processing device that executes processes related to the generation of an AI model based on machine learning and also manages information related to the AI model.
  • the AI model of the embodiment analyzes an input image and classifies the content of the image (one or more areas within the image) into one of multiple classes (also called categories) specified in advance by the user, and is hereinafter also referred to as a "classification model.”
  • the AI processing device 12 of the embodiment is a cloud server that provides a data processing service as a cloud service.
  • the processing executed by the AI processing device 12 includes processing for cutting out images for training a classification model (hereinafter also referred to as "learning images”) from the original image (hereinafter also referred to as "original image”) as learning data in response to user operations.
  • a learning image can be said to be an image for training a classification model, or a "cut-out image” cut out from an original image.
  • processing on a learning image can also be applied to a verification image for verifying a classification model.
  • the multiple user terminals 14 are information processing devices operated by multiple users who use the services of the AI processing device 12, and are, for example, information processing devices operated by a developer of an inspection system that uses a classification model.
  • the multiple user terminals 14 include user terminal 14a, user terminal 14b, and user terminal 14c that are operated by different users.
  • the user terminal 14 may be a PC, a tablet terminal, or a smartphone.
  • the AI processing device 12 has a web server function.
  • the AI processing device 12 provides web content (HTML data, etc.) related to the development of the classification model to the user terminal 14.
  • the user terminal 14 accesses the AI processing device 12 via a web browser.
  • the web browser of the user terminal 14 displays the web content provided by the AI processing device 12 on a specified display.
  • FIG. 2 is a block diagram showing the functional blocks of the AI processing device 12 in FIG. 1.
  • Each block shown in the block diagram of this disclosure can be realized in hardware terms by elements and mechanical devices such as a computer's CPU and memory, and in software terms by a computer program, etc., but here we depict the functional blocks realized by the cooperation of these. Those skilled in the art will understand that these functional blocks can be realized in various ways by combining hardware and software.
  • the AI processing device 12 comprises a processing unit 20, a memory unit 22, and a communication unit 24.
  • the processing unit 20 executes various data processing related to the development of a classification model.
  • the memory unit 22 stores data referenced or updated by the processing unit 20.
  • the communication unit 24 communicates with an external device according to a predetermined communication protocol.
  • the processing unit 20 transmits and receives data to and from the user terminal 14 via the communication unit 24.
  • the storage unit 22 includes a model information storage unit 26 that stores model information.
  • the model information includes information related to the classification model.
  • FIG. 3 shows an example of model information, and shows multiple items that are associated and stored as model information.
  • the model information includes a name, class information, original image information, learning image information, a learning dataset, classification model data, and response map data.
  • “Class information” information on multiple classes predetermined by the user is set. Classes are candidates for classification by a classification model, and are also called categories.
  • “Original image information” information on the original image is set. The original image information may include a path name for accessing the original image.
  • Learning image information is information on a learning image cut out from the original image, and includes data on the learning image and information on the position of the learning image on the original image. The position of the learning image on the original image can also be referred to as the cut-out position from the original image, and may be the coordinate values of the top left and bottom right of the learning image in the original image.
  • the model information of the embodiment includes a training dataset as a dataset related to the construction of a classification model.
  • the "training dataset” is data in which a specific class from among multiple classes defined by the class information is assigned to a training image, in other words, a pair of a training image and a specific class.
  • the "classification model data” is data of the generated classification model. It may be data of a file in which the classification model is saved.
  • Response map data is response map data that indicates the degree of response to the class into which the learning data is classified with respect to the original image.
  • the response map is a heat map that uses colors and shades to represent the degree of response to the class into which the learning data is classified.
  • the model information storage unit 26 stores multiple response map data corresponding to multiple classes indicated by the class information.
  • the processing unit 20 includes an analysis support screen providing unit 30, an image cropping unit 32, a dataset generating unit 34, a model generating unit 36, a response map generating unit 38, and a dataset changing unit 40.
  • the functions of the multiple functional blocks of the processing unit 20 may be implemented in a computer program (herein referred to as an "AI analysis support program").
  • the AI analysis support program may be stored in a non-temporary recording medium and installed in the storage of the AI processing device 12 via the recording medium.
  • the AI analysis support program may also be downloaded via a network and installed in the storage of the AI processing device 12.
  • the processor (CPU, etc.) of the AI processing device 12 may perform the functions of the multiple functional blocks of the processing unit 20 by reading the AI analysis support program into main memory and executing it.
  • the analysis support screen providing unit 30 transmits data of the analysis support screen, which is a user interface for information processing by the AI processing device 12, to the user terminal 14.
  • the analysis support screen is a web page.
  • the web browser of the user terminal 14 displays the analysis support screen provided by the AI processing device 12 on a specified display device.
  • the analysis support screen includes a dataset confirmation screen.
  • the dataset confirmation screen includes both the original image from which the learning data was extracted and a response map related to that original image.
  • the image cropping unit 32 generates training images by cropping the training images from the original images uploaded in advance from the user terminal 14 in response to user operations input on the analysis support screen.
  • the dataset generation unit 34 generates a training dataset in which the training images generated by the image cropping unit 32 are associated with the classes specified on the analysis support screen, and stores the training dataset in the model information storage unit 26.
  • the model generation unit 36 executes machine learning based on the learning dataset stored in the model information storage unit 26, and generates a classification model as a trained model.
  • a known technique may be used to create the classification model.
  • the classification model may be a neural network, or may be a different type of mathematical model (such as a decision tree) from a neural network.
  • the model generation unit 36 stores data of the generated classification model in the model information storage unit 26.
  • the response map generating unit 38 uses the classification map generated by the model generating unit 36 to generate multiple response maps indicating the degree of response to multiple classes indicated by the class information for the original image.
  • the response map generating unit 38 stores data of the multiple generated response maps in the model information storage unit 26. Note that a publicly known method may be used to generate the response map using the classification map.
  • the dataset change unit 40 changes the class associated with the training image in the training dataset in response to user operations entered on the analysis support screen.
  • FIG. 4 is a flowchart showing the operation of the AI processing device 12 of the embodiment. The process in FIG. 4 is executed in parallel for each user terminal 14. The operation of the information processing system 10 will be described below with reference to FIG. 4.
  • the user terminal 14 requests the web page of the analysis support screen from the AI processing device 12 in response to user operation.
  • the analysis support screen is requested (Y in S10)
  • the analysis support screen providing unit 30 of the AI processing device 12 provides the web page of the analysis support screen to the user terminal 14 (S12).
  • the user terminal 14 displays the web page of the analysis support screen on the display.
  • the user terminal 14 transmits a registration request for an original image, with model information specified, to the AI processing device 12 in response to user operation on the analysis support screen.
  • the user terminal 14 also transmits class information, with model information specified, to the AI processing device 12 in response to user operation on the analysis support screen.
  • the AI processing device 12 stores the original image and class information transmitted from the user terminal 14 in the specified model information in the model information storage unit 26.
  • the processing from S14 onwards is carried out for each piece of model information specified by the user on the analysis support screen.
  • the processing from S14 onwards is related to the specific model information specified by the user (hereinafter also referred to as "target model information").
  • the user terminal 14 transmits to the AI processing device 12 information regarding the user's operation on the original image displayed on the analysis support screen, the information specifying the cut-out area of the learning image and the class to which the learning image is associated.
  • the image cut-out unit 32 of the AI processing device 12 When the specified information is accepted (Y in S14), the image cut-out unit 32 of the AI processing device 12 generates a learning image cut out from the original image.
  • the image cut-out unit 32 stores information regarding the generated learning image (information on the cut-out position, etc.) in the target model information of the model information storage unit 26.
  • the dataset generation unit 34 of the AI processing device 12 generates a learning dataset that associates the learning image generated by the image cut-out unit 32 with the specified class, and stores the learning dataset in the target model information of the model information storage unit 26 (S16). If the specified information is not accepted (N in S14), the process of S16 is skipped.
  • the user terminal 14 transmits an instruction to generate a classification model, including the specification of target model information, to the AI processing device 12 in response to a user's operation on the analysis support screen.
  • the model generation unit 36 of the AI processing device 12 executes machine learning based on the learning dataset of the specified model information to generate a classification model (S20).
  • the model generation unit 36 stores data of the generated classification model in the target model information of the model information storage unit 26.
  • the response map generation unit 38 of the AI processing device 12 uses the classification model generated in S20 to estimate the locations in the original image 110 that correspond to each class.
  • the response map generation unit 38 generates multiple response maps that indicate the degree of response to each class for the original image, and stores them in the target model information of the model information storage unit 26 (S22).
  • Figure 5 (A) shows an example of an original image 110.
  • Figures 5 (B) to 5 (F) show examples of response maps. In this example, five classes are defined: lead failure, dirt, case part, background part, and lead part.
  • Figure 5 (B) shows response map 58a for class "lead failure”.
  • Figure 5 (C) shows response map 58b for class "dirt”.
  • Figure 5 (D) shows response map 58c for class "case part”.
  • Figure 5 (E) shows response map 58d for class "background part”.
  • Figure 5 (F) shows response map 58e for class "lead part”.
  • Response maps 58a to 58e are collectively referred to as response maps 58.
  • Each area of the response map 58 is colored in 256 levels from 0 to 255 according to the probability value of falling into the class. 0 is the lowest probability value of falling into the class, and 255 is the highest probability of falling into the class.
  • the response map 58 contains values indicating the degree of response to each class for each area of the original image 110.
  • the response map for a particular class contains values (also called response values) indicating the degree of response to that class for each area of the original image 110.
  • the response value is 0 for the lowest degree of response and 255 for the highest degree of response.
  • a publicly known method may be used to generate a response map for each class.
  • the response map generator 38 may input the original image 110 to a classification model as a fully convolutional neural network.
  • the response map generator 38 may obtain a plurality of response maps 58 corresponding to a plurality of predefined classes as output data from the classification model.
  • the response map generator 38 may obtain a probability value corresponding to each class for each region of the original image 110 from the classification model.
  • the response map generator 38 may generate a plurality of response maps 58 for a plurality of classes by coloring each region in the response map 58 for a certain class according to the probability value for that region of the class.
  • the user terminal 14 transmits data requesting a dataset confirmation screen for the target model information to the AI processing device 12.
  • the analysis support screen providing unit 30 of the AI processing device 12 generates data for a dataset confirmation screen, which is one of the analysis support screens, based on the target model information and transmits it to the user terminal 14 (S26).
  • the user terminal 14 displays the dataset confirmation screen on the display.
  • FIG. 6 shows an example of a dataset confirmation screen 120.
  • the dataset confirmation screen 120 includes content that supports confirmation of the learning images of the learning dataset.
  • the dataset confirmation screen 120 includes a class selection area 122, a cut-out image area 124, and an entire image display area 125.
  • the class selection area 122 multiple tags are arranged that relate to multiple classes recorded in the target model information (in FIG. 6, "dirt,” “dent,” “background,” and “lead portion").
  • the cut-out image area 124 In the cut-out image area 124, one or more training images recorded in the target model information are placed.
  • the user terminal 14 transmits information about the selected tag (such as information about the class associated with the tag) to the AI processing device 12.
  • the analysis support screen providing unit 30 of the AI processing device 12 transmits one or more training images associated with the class of the selected tag to the AI processing device 12, and displays the one or more images in the cut-out image area 124. For example, when the user selects a tag of the class "dent", the cut-out image area 124 of the dataset confirmation screen 120 is updated to display one or more training images associated with the class "dent".
  • the objects placed in the file operation area 128 include a bulk selection button, a bulk deselection button, and a delete button.
  • any training image can be selected in the cut-out image area 124 and the selected training image can be deleted by pressing the delete button.
  • by pressing the bulk selection button multiple training images placed in the cut-out image area 124 can be selected at once.
  • the delete button is selected in this state, the multiple selected training images can be deleted all at once.
  • the bulk deselection button is selected, the selection state of multiple training images can be deselected all at once.
  • the whole image display area 125 of the dataset confirmation screen 120 is information that shows in a predetermined manner the area into which each learning image displayed in the cut-out image area 124 has been cut out.
  • a plurality of cut-out position guides 126 corresponding to the plurality of learning images cut out from the original image 110 are placed on the original image 110.
  • the analysis support screen providing unit 30 of the AI processing device 12 places the plurality of cut-out position guides 126 at positions on the original image 110 that correspond to the cut-out positions of the plurality of learning images, based on the learning image information (position information) recorded in the target model information.
  • the analysis support screen providing unit 30 records the correspondence between the multiple learning images and the multiple cut-out position guides 126 in the dataset confirmation screen 120.
  • the dataset confirmation screen 120 is configured to highlight the cut-out position guide 126 corresponding to the selected learning image using Javascript (registered trademark) or the like.
  • the highlighting may be, for example, in a manner that stands out more than the other cut-out position guides 126 (such as a pattern or color).
  • FIG. 7 shows an example of a setting screen 130 for the display mode of the whole image display area 125.
  • the display mode can be selected from among displaying the original image 110 alone, displaying the reaction map alone, and displaying both the original image 110 and the reaction map.
  • the composite result of the original image 110 and the reaction map is displayed.
  • model information selection area 134 model information to be used for creating the reaction map is set.
  • the class selection area 136 is an area for selecting a class to be displayed in the reaction map.
  • multiple tags are arranged to indicate multiple classes defined in the class information of the target model information.
  • four tags are arranged corresponding to the four classes (dirt, dent, background, lead) shown in FIG. 6.
  • the foreign matter class and dent class are associated with the higher-level NG (No Good) category.
  • the background class and lead class are associated with the higher-level OK category.
  • the user terminal 14 transmits data requesting the display of the reaction map, including the specification of the target model information, to the AI processing device 12.
  • the analysis support screen providing unit 30 of the AI processing device 12 transmits the reaction map data recorded in the target model information to the user terminal 14 (S30).
  • the user terminal 14 displays the reaction map provided by the AI processing device 12 in the entire image display area 125 of the dataset confirmation screen 120. If a request is not made to display a reaction map (N in S28), the processing of S30 is skipped.
  • FIGS. 8(A) to 8(F) show schematic images displayed in the entire image display area 125 of the dataset confirmation screen 120.
  • FIG. 8(A) shows the original image 110.
  • the analysis support screen providing unit 30 provides the original image 110 shown in FIG. 8(A) as the content of the entire image display area 125.
  • FIG. 8(B) shows a response map 140a corresponding to the foreign object class.
  • the analysis support screen providing unit 30 provides the response map 140a shown in FIG. 8(B) as the content of the entire image display area 125.
  • the response map 140a indicates that the classification model has determined that there is a high probability that the rectangular region of the original image 110 corresponds to the foreign object class.
  • FIG. 8(C) shows a response map 140b corresponding to the dent class.
  • the analysis support screen providing unit 30 provides response map 140b shown in FIG. 8(C) as the content of the entire image display area 125.
  • Response map 140b indicates that the classification model has determined that there is a high probability that the elliptical region of the original image 110 corresponds to the dent class.
  • FIG. 8(D) shows a composite image 144a that is the result of combining the original image 110 and the response map 140a.
  • the analysis support screen providing unit 30 provides the composite image 144a shown in FIG. 8(D) as the content of the whole image display area 125.
  • the composite image 144a may be the original image 110 with the response map 140a superimposed at a predetermined transparency.
  • FIG. 8(E) shows a composite image 144b that is the result of combining the original image 110 and the response map 140b.
  • the analysis support screen providing unit 30 provides the composite image 144b shown in FIG. 8(E) as the content of the entire image display area 125.
  • the composite image 144b may be the response map 140b superimposed on the original image 110 with a predetermined transparency.
  • the response map generation unit 38 of the AI processing device 12 generates multiple response maps (e.g., response map 140a, response map 140b) that indicate the degree of response to multiple classes for the original image 110.
  • the dataset confirmation screen 120 is configured to be able to switch between and display multiple response maps corresponding to multiple classes. With this configuration, it is possible to clearly show the user the agreement or inconsistency between the association made by the user and the classification result by the classification model for each class that the user has associated with the learning image.
  • FIG. 8(F) shows a composite image 144c that is the result of combining the original image 110 and the reaction map 140b.
  • the analysis support screen providing unit 30 provides the composite image 144c shown in FIG. 8(F) as the content of the whole image display area 125.
  • the analysis support screen providing unit 30 may identify multiple classes included in the NG category (foreign matter class and dent class in FIG. 7) and generate the composite image 144c by overlaying the reaction map 140a and the reaction map 140b on the original image 110 at a predetermined transparency.
  • the analysis support screen providing unit 30 of the AI processing device 12 provides a dataset confirmation screen 120 including the synthesis result of the response maps of one or more classes associated with the abnormal attribute.
  • a composite image is also provided when the user selects the OK category in the class selection area 136.
  • the analysis support screen providing unit 30 of the AI processing device 12 provides a dataset confirmation screen 120 including the composite result of the response maps of one or more classes associated with the normal attribute.
  • the analysis support screen providing unit 30 may identify multiple classes included in the OK category (background class and lead class in FIG. 7), and generate a composite image 144c by overlaying the response map of the background class and the response map of the lead class on the original image 110 at a predetermined transparency as the content of the whole image display area 125.
  • the analysis support screen providing unit 30 may also place multiple cut-out position guides 126 (see FIG. 6) corresponding to the cut-out positions of multiple learning images on the response map or on the composite image.
  • a screen element for changing the class assigned to the image selected in the cut-out image area 124 to a class different from the previous class.
  • the user compares the cut-out position guide 126 on the original image 110 displayed in the whole image display area 125 with the color shading of the response map.
  • the user wants to change a specific learning image from among the multiple learning images of dirt classes displayed in the cut-out image area 124 to a dent class.
  • the user selects the learning image to be changed in the cut-out image area 124, and then selects the "Change to dent" button in the change destination designation area 127.
  • the user terminal 14 transmits a class change instruction specifying the selected image and the changed class to the AI processing device 12 in response to the user's class change operation for the selected image.
  • the dataset change unit 40 of the AI processing device 12 receives the class change instruction transmitted from the user terminal 14 (Y in S32), it updates the class information of the target model information so as to change the class assigned to the selected image specified in the class change instruction to the changed class specified in the class change instruction (S34).
  • the dataset confirmation screen 120 also allows the deletion of training images. For example, if the cut-out position of a training image to which a dirt class has been assigned has a low response in the dirt class response map, the user selects the training image and presses the delete button. In response to the user's delete operation on the selected image, the user terminal 14 transmits an instruction to delete the selected image to the AI processing device 12.
  • the dataset change unit 40 of the AI processing device 12 deletes the selected image from the training dataset of the target model information and the training image information.
  • the dataset confirmation screen 120 it is also possible to add a training image; in other words, it is also possible to cut out a new training image from the original image 110. For example, if there is no training image cut out from a high-response area in the dirt class response map, the user specifies that area on the original image 110 as a new cut-out area.
  • the user terminal 14 transmits to the AI processing device 12 information regarding the user's operation on the original image 110 displayed on the dataset confirmation screen 120, which specifies the new cut-out area of the training image and the class (e.g., dirt class) to which the training image is to be associated.
  • the processing of the AI processing device 12 has already been described in relation to S14 and S16, so a description thereof will be omitted.
  • a dataset change instruction such as a class change instruction
  • the process of S34 is skipped. If the dataset confirmation screen 120 is not requested (N in S24), the process of S26 and subsequent steps is skipped. If the analysis support screen is not requested (N in S10), the process of S12 and subsequent steps is skipped.
  • the information processing system 10 of the embodiment provides the user with a dataset confirmation screen 120 that includes the original image 110 and the response map. This allows the user to determine the validity of the dataset used to create the classification model, and assists in adding new training images to the dataset, deleting existing training images, and correcting the association between training images and classes. This assists in improving or maintaining the accuracy of the classification model.
  • the composite image of the reaction map and the original image 110 is displayed in the dataset confirmation screen 120, but the composite image of the reaction map and the original image 110 may be displayed on a screen separate from the dataset confirmation screen 120.
  • the user may compare the dataset confirmation screen 120 with another screen that displays the composite image of the reaction map and the original image 110 to determine whether or not the dataset needs to be modified.
  • the AI processing device 12 is a cloud server, but in a modified example, the AI processing device 12 may be an on-premise server. Furthermore, the functions of the AI processing device 12 in the embodiment may be distributed and implemented in multiple information processing devices. In this case, the multiple information processing devices may communicate with each other and work together as a system to execute processing similar to that of the AI processing device 12 in the embodiment. Furthermore, at least a part of the functions of the AI processing device 12 in the embodiment may be implemented in an application running on the user terminal 14, and at least a part of the processing of the AI processing device 12 in the embodiment may be executed by the user terminal 14.
  • the AI processing device 12 includes a model information storage unit 26, but as a modified example, a device other than the AI processing device 12 may include the model information storage unit 26. In this case, the model information storage unit 26 may access the data in the model information storage unit 26 by communicating with the other device.
  • a storage unit that stores a data set for constructing a classification model, the data set associating learning data extracted from an original image with a class to which the learning data is classified; a model generation unit that performs machine learning based on the dataset and generates the classification model; a response map generator that uses the classification model to generate a response map indicating a degree of response to the class for the original image; a screen providing unit that provides a screen including both the original image and the response map; A change unit that changes the data set in response to a user operation;
  • An information processing system comprising: This information processing system can assist a user in associating input data of a dataset with an appropriate class by providing a screen including both the original image and the response map, and as a result, can assist in maintaining or improving the judgment accuracy of the classification model.
  • the screen providing unit provides a screen including a synthesis result of the original image and the response map.
  • the information processing system according to claim 1.
  • the information processing system can assist a user in associating input data of a dataset with an appropriate class.
  • a plurality of classes are defined as the classification destination classes,
  • the response map generator generates a plurality of response maps indicating a degree of response to the plurality of classes for the original image;
  • the screen is configured to be capable of switching and displaying a plurality of reaction maps corresponding to the plurality of classes. 3.
  • the screen providing unit provides a screen including a composite result of the response maps of the one or more classes associated with the normal attribute.
  • the screen providing unit provides a screen including a composite result of the response maps of the one or more classes associated with the abnormal attribute.
  • the information processing system according to any one of techniques 1 to 4. From the perspective of an abnormal attribute that aggregates one or more classes, the agreement or disagreement between the association made by the user and the classification result by the classification model can be shown to the user in an easy-to-understand manner.
  • a computer that can access a storage unit that stores a data set for constructing a classification model, the data set associating learning data extracted from an original image with a class to which the learning data is classified, performing machine learning based on the dataset to generate the classification model; generating a response map using the classification model that indicates a degree of response to the classes for the source image; providing a display including both the original image and the response map; Changing the data set in response to a user operation.
  • the technology disclosed herein can be applied to information processing systems and information processing devices.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Medical Informatics (AREA)
  • Mathematical Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Databases & Information Systems (AREA)
  • Image Analysis (AREA)
PCT/JP2024/014323 2023-04-13 2024-04-08 情報処理システムおよびコンピュータプログラム Ceased WO2024214680A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2025513952A JPWO2024214680A1 (https=) 2023-04-13 2024-04-08

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2023065627 2023-04-13
JP2023-065627 2023-04-13

Publications (1)

Publication Number Publication Date
WO2024214680A1 true WO2024214680A1 (ja) 2024-10-17

Family

ID=93059396

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2024/014323 Ceased WO2024214680A1 (ja) 2023-04-13 2024-04-08 情報処理システムおよびコンピュータプログラム

Country Status (2)

Country Link
JP (1) JPWO2024214680A1 (https=)
WO (1) WO2024214680A1 (https=)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017225122A (ja) * 2013-06-28 2017-12-21 日本電気株式会社 映像監視システム、映像処理装置、映像処理方法および映像処理プログラム
JP2021060692A (ja) * 2019-10-03 2021-04-15 株式会社東芝 推論結果評価システム、推論結果評価装置及びその方法

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017225122A (ja) * 2013-06-28 2017-12-21 日本電気株式会社 映像監視システム、映像処理装置、映像処理方法および映像処理プログラム
JP2021060692A (ja) * 2019-10-03 2021-04-15 株式会社東芝 推論結果評価システム、推論結果評価装置及びその方法

Also Published As

Publication number Publication date
JPWO2024214680A1 (https=) 2024-10-17

Similar Documents

Publication Publication Date Title
US20160261475A1 (en) System and method for accessibility assessment of a web application
CN112711526A (zh) Ui测试方法、装置、设备及存储介质
CN109671126A (zh) 使用神经网络预测分块位移图
US6993711B1 (en) Style-sheet output apparatus and method, and style-sheet output system
KR20230127351A (ko) 비전 소프트웨어 개발 시스템, 방법, 장치 및 컴퓨터저장 매체
CN113360737A (zh) 页面内容采集方法、装置、电子设备和可读介质
US11823086B2 (en) Membership analyzing method, apparatus, computer device and storage medium
KR102227127B1 (ko) 리소그래피 시뮬레이션을 이용한 디자인룰 생성 장치 및 방법
WO2024214680A1 (ja) 情報処理システムおよびコンピュータプログラム
JP2020201434A (ja) 生成装置、生成システムおよび生成方法
CN117744695B (zh) 一种融合二维码的自动生成方法及系统
WO2024214679A1 (ja) 情報処理システムおよびコンピュータプログラム
CN116331232B (zh) 车辆控制器的代码集成方法、装置、设备、介质和产品
KR20220007210A (ko) 딥러닝기법과 빅데이터를 활용한 사용자 맞춤형 감성 웹디자인 ui 설계 방법
US20220215034A1 (en) Electronic apparatus and controlling method thereof
WO2023239468A1 (en) Cross-application componentized document generation
JP7358979B2 (ja) 情報処理システム、評価方法
CN115145565A (zh) 一种基于规则引擎的数据可视化大屏布局方法与装置
WO2024210107A1 (ja) 画像処理システムおよびコンピュータプログラム
CN114721931A (zh) 一种数据处理方法、装置、设备及存储介质
JP7714769B1 (ja) 情報処理装置、情報処理システム、情報処理方法及びプログラム
CA3108148A1 (en) Systems and methods for generating data retrieval steps
JP7632190B2 (ja) 情報処理装置、情報処理方法、及び情報処理プログラム
CN116301754B (zh) 基于云服务器的嵌入式设备固件开发方法、装置及系统
KR20170107251A (ko) 분석 프리젠팅 프레임워크 시스템 및 분석 프리젠팅 프레임워크 시스템의 정보 처리 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24788704

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2025513952

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 2025513952

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 24788704

Country of ref document: EP

Kind code of ref document: A1