WO2023013788A1 - 学習済みモデル管理装置及び学習済みモデル管理方法 - Google Patents

学習済みモデル管理装置及び学習済みモデル管理方法 Download PDF

Info

Publication number
WO2023013788A1
WO2023013788A1 PCT/JP2022/030203 JP2022030203W WO2023013788A1 WO 2023013788 A1 WO2023013788 A1 WO 2023013788A1 JP 2022030203 W JP2022030203 W JP 2022030203W WO 2023013788 A1 WO2023013788 A1 WO 2023013788A1
Authority
WO
WIPO (PCT)
Prior art keywords
model
user
learning data
management device
storage unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/030203
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
匡芳 中村
匡史 堤
智之 和泉
康平 古川
慧 村岡
達将 樺澤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Rist Inc
Kyocera Corp
Original Assignee
Rist Inc
Kyocera Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Rist Inc, Kyocera Corp filed Critical Rist Inc
Priority to JP2023540442A priority Critical patent/JP7640710B2/ja
Priority to EP22853206.5A priority patent/EP4383150A4/en
Priority to CN202280054119.XA priority patent/CN117751371A/zh
Priority to US18/681,203 priority patent/US20240311702A1/en
Publication of WO2023013788A1 publication Critical patent/WO2023013788A1/ja
Anticipated expiration legal-status Critical
Priority to JP2025026215A priority patent/JP2025071258A/ja
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/40—Robotics, robotics mapping to robotics vision
    • G05B2219/40564—Recognize shape, contour of object, extract position and orientation
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/40—Robotics, robotics mapping to robotics vision
    • G05B2219/40607—Fixed camera to observe workspace, object, workpiece, global
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/45—Nc applications
    • G05B2219/45063—Pick and place manipulator

Definitions

  • the present disclosure relates to a trained model management device and a trained model management method.
  • Patent Literature 1 discloses a system that distributes a recognition model updated (re-learned) with a simulation video that reproduces a scene for which recognition has failed to a vehicle via a network.
  • a trained model management device a first storage unit that stores first learning data and a first model, which is a trained model that has undergone learning processing so as to be able to recognize an object included in input information based on the first learning data; a second storage unit that stores second learning data and a second model that is a trained model generated based on the second learning data and the first model; an update determination unit that determines whether or not to update the first model based on the second learning data when the second model is generated.
  • a trained model management method includes: a first storage unit that stores first learning data and a first model, which is a trained model that has undergone learning processing so as to be able to recognize an object included in input information based on the first learning data;
  • a trained model management device comprising: a second storage unit that stores second learning data; and a second model that is a trained model generated based on the second learning data and the first model.
  • a trained model management method executed by Determining whether to update the first model based on the second learning data when the second model is generated.
  • FIG. 1 is a diagram showing a configuration example of a trained model management system including a trained model management device according to one embodiment.
  • FIG. 2 is a schematic diagram showing a configuration example of a robot control system.
  • FIG. 3 is a diagram showing a configuration example of a group of models provided as a library.
  • FIG. 4 is a sequence diagram showing an example of the procedure of processing in the trained model management system.
  • FIG. 5 is a sequence diagram following FIG.
  • FIG. 1 shows a configuration example of a trained model management system 1 including a trained model management device 10 according to this embodiment.
  • the trained model management device 10 is a device that manages updates of trained models provided as a library.
  • the user uses the trained model for tasks such as identifying objects and displaying identification results.
  • the trained model management device 10 can execute a task using evaluation data and a specific model provided by a user, and output execution results.
  • the model group includes a plurality of machine learning models (learned models) generated so as to recognize an object by machine learning.
  • a user can utilize the trained model by downloading it via the network 40, for example.
  • the trained model management device 10 includes a communication unit 11, a first storage unit 12A, a second storage unit 12B, and a control unit 13.
  • the control unit 13 includes an update determination unit 131 , an update processing unit 132 and a detection unit 133 .
  • the trained model management device 10 may be, for example, a computer such as a server as a hardware configuration. Details of the components of the trained model management device 10 will be described later.
  • the trained model management device 10 may configure the trained model management system 1 together with the trained model generation device 20 and the communication terminal device 30 connected via the network 40 .
  • the network 40 is, for example, the Internet, but may be a LAN (Local Area Network).
  • the trained model management device 10 and the trained model generation device 20 may communicate via a LAN, and the trained model management device 10 and the communication terminal device 30 may communicate via the Internet.
  • the trained model generation device 20 can generate master models and custom models, which will be described later, by machine learning. That is, the trained model generation device 20 can generate a trained model by machine learning using learning data such as first learning data and second learning data described later. Also, the trained model generating device 20 generates a new trained model (for example, a custom model to be described later). Also, the trained model generation device 20 may add new learning data to update (re-learn) the specified model.
  • the learning data may be created by imaging with a camera or the like, or may be created using CAD (Computer Aided Design) data. Also, the learning data may be created by combining learning data obtained by imaging and learning data obtained by CAD.
  • the trained model generation device 20 can access the first storage unit 12A and the second storage unit 12B of the trained model management device 10. Further, in the present embodiment, the trained model generation device 20 is a computer separate from the trained model management device 10 and capable of communicating with the trained model management device 10, but the configuration is not limited to this.
  • the trained model generation device 20 may be implemented by a computer integrated with the trained model management device 10 . In this case, the trained model generation device 20 may have the configuration requirements of the trained model management device 10, or the trained model management device 10 may have the configuration requirements of the trained model generation device 20. You may have
  • the communication terminal device 30 is, for example, a general-purpose communicable terminal device such as a smartphone or a tablet terminal, but is not limited to these. As another example, the communication terminal device 30 may be configured as part of a robot control system 100 (see FIG. 2), which will be described later.
  • the communication terminal device 30 generates input information (for example, an image of an object to be identified) and outputs the input information to the trained model management device 10 .
  • the input information can be used as evaluation data to evaluate the updated model.
  • the communication terminal device 30 outputs learning data provided by the user to the trained model management device 10 . Further, the communication terminal device 30 may present the execution result of the task executed by the learned model management device 10 using the evaluation data and the specific model to the user.
  • the communication terminal device 30 includes a communication section 31 , a storage section 32 , a control section 33 , an input information generation section 34 and a user interface section 35 . Details of the components of the communication terminal device 30 will be described later.
  • the users who use the trained model management system 1 include the first user and the second user.
  • the first user can use the communication terminal device 30-1 to generate, for example, a custom model unique to the first user.
  • the second user can use the communication terminal device 30-2 to generate, for example, a custom model unique to the second user.
  • Communication terminal device 30-2 may have the same configuration as communication terminal device 30-1.
  • the communication terminal device 30-1 and the communication terminal device 30-2 have the same configuration, and will be described as the communication terminal device 30 unless otherwise distinguished.
  • FIG. 2 is a schematic diagram showing a configuration example of the robot control system 100.
  • FIG. A robot control system 100 includes a robot 2 and a robot controller 110 .
  • the robot 2 moves the work object 8 from the work start point 6 to the work target point 7 by the end effector 2B (gripping hand) at the tip of the arm 2A. That is, the robot control device 110 controls the robot 2 so that the work object 8 moves from the work start point 6 to the work target point 7 .
  • the robot control system 100 has a camera 4 .
  • a camera 4 photographs an article or the like within an influence range 5 that may affect the operation of the robot 2 .
  • the robot control device 110 Based on the image captured by the camera 4, the robot control device 110 recognizes the work object 8 existing in the space where the robot 2 works. Recognition of the work object 8 is performed using models included in the above model group (learned models capable of recognizing the work object 8). The robot controller 110 downloads and deploys the trained model via the network 40 before recognizing the work object 8 .
  • the models provided as a library can recognize various types of work objects 8 so as to meet the requirements of many users' robot control systems 100, for example. Therefore, it is preferable that the trained model management device 10 updates the trained model, which is the master model included in the model group, so that, for example, a new product can be recognized as an object.
  • the trained model management device 10 updates the trained model, which is the master model included in the model group, so that, for example, a new product can be recognized as an object.
  • the trained model management device 10 allows the continued use of the custom model in the user's usage environment without forcibly replacing the model with the master model.
  • the trained model management device 10 can update the master model so as to expand the recognition target using the learning data used in generating the custom model. Also, after confirming the user's instruction to change from the custom model to the master model, the learned model management device 10 sets the updated master model to be used in the user's usage environment. Details of the master model and the custom model will be described later with reference to FIG. Further, in the present embodiment, the communication terminal device 30 is described as a tablet terminal owned by the user. 30.
  • the communication unit 11 includes one or more communication modules connected to the network 40 .
  • the communication unit 11 may include a communication module compatible with mobile communication standards such as 4G (4th Generation) and 5G (5th Generation).
  • the communication unit 11 may include a communication module compatible with, for example, a wired LAN standard (eg, 1000BASE-T).
  • the communication unit 11 may include, for example, a communication module compatible with the wireless LAN standard (IEEE802.11 as an example).
  • the first storage unit 12A and the second storage unit 12B are one or more memories.
  • the memory is, for example, a semiconductor memory, a magnetic memory, an optical memory, or the like, but is not limited to these and can be any memory.
  • the first storage unit 12A and the second storage unit 12B are built in the trained model management device 10, for example, but it is also possible to access the trained model management device 10 from the outside via an arbitrary interface.
  • the first storage unit 12A and the second storage unit 12B may be configured with the same memory and may have separate storage areas. Also, the first storage unit 12A and the second storage unit 12B may be configured by physically different memories.
  • At least one of the first storage unit 12A and the second storage unit 12B stores various data used in various calculations executed by the control unit 13. Also, at least one of the first storage unit 12A and the second storage unit 12B may store results of various calculations executed by the control unit 13 and intermediate data.
  • the first storage unit 12A and the second storage unit 12B are accessed from the trained model generation device 20 as described above. That is, the first storage unit 12A and the second storage unit 12B are shared by the trained model management device 10 and the trained model generation device 20.
  • FIG. The first storage unit 12A and the second storage unit 12B also store model groups (a plurality of models) and the like.
  • the first storage unit 12A stores a master model.
  • the second storage unit 12B stores custom models.
  • the control unit 13 is one or more processors.
  • the processor is, for example, a general-purpose processor or a dedicated processor specialized for specific processing, but is not limited to these and can be any processor.
  • the control unit 13 controls the overall operation of the trained model management device 10 .
  • the trained model management device 10 may have the following software configuration.
  • One or more programs used for controlling the operation of the trained model management device 10 are stored in the first storage unit 12A or the second storage unit 12B.
  • the program when read by the processor of the control unit 13 , causes the control unit 13 to function as the update determination unit 131 , the update processing unit 132 and the detection unit 133 .
  • the update determination unit 131 determines whether or not to update the master model based on the learning data.
  • the update determination unit 131 for example, when determining that the target to be added has high versatility, that is, when the target to be identified can be expanded by updating and can be used for general purposes, the master model to be updated.
  • the update determination unit 131 determines to update the master model without determining versatility when an update instruction is received from a model group manager (for example, a model supplier). you can Further, the update determination unit 131 may determine to update the master model, for example, when the learning data used to generate the custom model can improve the recognition accuracy of the recognition target.
  • the update processing unit 132 executes processing related to updating the master model. More specifically, when the update determination unit 131 determines that the master model should be updated, the update processing unit 132 causes the trained model generation device 20 to perform machine learning (relearning), for example, to update the master model. to update. Machine learning (re-learning) uses the learning data (first learning data) used to generate the master model and the learning data (second learning data) used to generate the custom model. executed. In this embodiment, the learned model generation device 20 updates the master model according to the update instruction from the update processing unit 132, but as another example, the update processing unit 132 may update the master model.
  • machine learning uses the learning data (first learning data) used to generate the master model and the learning data (second learning data) used to generate the custom model.
  • the learned model generation device 20 updates the master model according to the update instruction from the update processing unit 132, but as another example, the update processing unit 132 may update the master model.
  • the update processing unit 132 may execute the task using the updated master model and present the task execution result to the user.
  • the task is object recognition. Based on the presented results, the user can evaluate, for example, precision and recall, to see if the updated master model is inferior to the custom model.
  • the update processing section 132 may update the data indicating the degree of contribution of the user stored in the second storage section 12B.
  • the user's contribution increases (for example, points are added). Contributions, depending on their size, can extend the user's access to master and custom models.
  • the detection unit 133 detects whether the user who generated the custom model uses the updated master model instead of the custom model.
  • the detection unit 133 may determine to use the updated master model instead of the custom model when the user who generated the custom model performs an operation to create a new usage environment.
  • An operation to create a new usage environment is, for example, downloading a learned model when adding a new work (recognition of another target object), and making new settings on the communication terminal device 30 for deployment. be.
  • the detection unit 133 may determine that the user who generated the custom model continues to use the custom model when the user continues to use the conventional usage environment.
  • the communication unit 31 includes one or more communication modules connected to the network 40 .
  • the communication unit 31 may include a communication module compatible with mobile communication standards such as 4G and 5G.
  • the communication unit 31 may include a communication module compatible with the LAN standard.
  • the communication unit 31 may be configured including the same communication module as the communication unit 11, or may be configured including a different communication module.
  • the storage unit 32 is one or more memories.
  • the memory is, for example, a semiconductor memory, a magnetic memory, an optical memory, or the like, but is not limited to these and can be any memory.
  • the storage unit 32 stores various data used in various calculations executed by the control unit 33 . Further, the storage unit 32 may store results of various calculations executed by the control unit 33 and intermediate data. For example, the storage unit 32 may temporarily store input information generated by the input information generation unit 34 and transmitted (uploaded) to the trained model management device 10 . Further, for example, the storage unit 32 may temporarily store the recognition result of the object acquired from the trained model management device 10 via the communication unit 31 .
  • the control unit 33 is one or more processors.
  • the processor is, for example, a general-purpose processor or a dedicated processor specialized for specific processing, but is not limited to these and can be any processor.
  • the control unit 33 controls the overall operation of the communication terminal device 30 .
  • the input information generation unit 34 generates input information.
  • the input information is an image containing the object to be identified.
  • the input information generation unit 34 may be configured including an imaging unit such as a camera, for example.
  • the input information generator 34 is a camera.
  • the user may capture an image including a new object with the camera of the communication terminal device 30 to generate input information.
  • This input information may be evaluation data used to confirm whether the custom model is capable of recognizing new objects.
  • the user interface unit 35 inputs information from the user and outputs information to the user.
  • the user interface unit 35 includes, for example, a touch sensor.
  • the touch sensor detects contact with a user's finger, stylus pen, or the like, and identifies the contact position.
  • the touch sensor may be integrated with the display to form a touch panel display.
  • the user interface unit 35 is a touch panel display.
  • the presentation contents from the trained model management device 10 are displayed on the touch panel display.
  • the user interface unit 35 may prompt the input by displaying a chat screen or the like when inputting the evaluation data and the second learning data.
  • Model group In this embodiment, the model groups stored in the first storage unit 12A and the second storage unit 12B are classified as shown in FIG. Models are broadly divided into master models and custom models.
  • a master model is, for example, a trained model available to all users.
  • a master model is, for example, a highly versatile trained model.
  • a master model is, for example, a model created by an administrator of a group of models.
  • a custom model is, for example, a trained model created by a particular user.
  • a custom model may be, for example, a trained model that is only available to certain users.
  • a custom model is, for example, better at recognizing a specific object or recognizing an object under a specific environment than a master model.
  • the user includes the first user and the second user.
  • the master model is a trained model that can be used by the first user and the second user.
  • the custom model generated by the second user may be a trained model available only to the second user.
  • the first user's custom model may be removed from the second user's library (available models).
  • the second user's library may consist of master models and custom models generated by the second user. Note that, for example, one user (eg, the second user) may generate multiple custom models.
  • the master model includes general-purpose models and specialized models.
  • a general-purpose model is a model for recognizing objects in a large classification (first classification).
  • a specialized model is a model for recognizing an object in a smaller classification (second classification). The second classification subdivides one item of the first classification.
  • the custom model is generated from the second learning data prepared by the user. Specifically, the custom model is generated based on the master model and the second learning data. That is, in this embodiment, a custom model is generated using an existing master model. A custom model may be generated, for example, by performing additional learning on an existing master model using the second learning data. As such, the custom model is related to the master model from which it originated. Such association between the master model and the custom model is stored as related information in the first storage unit 12A and the second storage unit 12B. In other words, the first storage unit 12A and the second storage unit 12B store related information that defines the relationship between a plurality of models.
  • the first category is industrial parts, stationery, etc.
  • the second category is bolts, nuts, pens, scissors, and the like.
  • the general-purpose model “Industrial Parts” is a trained model for recognizing bolts, nuts, springs, and the like.
  • the specialized model “bolt” is a learned model for recognizing bolt types such as bolt A, bolt B, and bolt C.
  • the specialized models “Bolt” and “Nut” are related to the general model "Industrial Part". Related information may include associations between such generic models and specialized models.
  • the specialized model "bolt” can be used to generate a custom model.
  • the second user's custom model "bolt” is a learned model for recognizing bolt types such as bolt A, bolt B, and bolt C'.
  • the first storage unit 12A stores the master model and the first learning data used to generate the master model.
  • the second storage unit 12B stores the custom model and the second learning data used to generate the custom model.
  • a storage area may be divided for each user.
  • the trained model management device 10 may have different storage units for different users.
  • the second storage unit 12B may store only the custom model generated by the second user and be available only to the second user.
  • the trained model management device 10 may include a separate third storage section that is different from the first storage section 12A and the second storage section 12B and that can be used only by the first user.
  • (Learning model management method) 4 and 5 are sequence diagrams showing an example of the procedure of processing in the trained model management system 1.
  • FIG. FIG. 5 shows the procedure of processing executed subsequent to FIG. Processing of the learned model management method executed by the trained model management device 10 according to the present embodiment will be described with reference to FIGS. 4 and 5.
  • FIG. As in the above example, as a specific example, the second user generates a custom model so that "Bolt C'" can be recognized from an image containing "Bolt C'" and other industrial parts. An example will be used.
  • the user uses the camera of the communication terminal device 30 to capture an image of "bolt C'” and other industrial parts, and generates a captured image that is input information (step S1).
  • This captured image becomes evaluation data used to evaluate the task execution results when executing a task of recognizing "bolt C'" using a custom model or the like.
  • the evaluation data is information about an object that becomes recognizable using the second learning data prepared by the second user.
  • the communication terminal device 30 transmits the input information (evaluation data) to the trained model management device 10 (step S2).
  • the second storage unit 12B of the trained model management device 10 stores evaluation data.
  • the user selects a master model and inputs it into the communication terminal device 30 (step S3).
  • the user may select the master model that has been used so far in the robot control system 100, for example.
  • the second user selects the specialized model "bolt" that has been used up to that point.
  • the communication terminal device 30 transmits the user-selected master model to the trained model management device 10 (step S4).
  • the communication terminal device 30 transmits to the trained model management device 10 information that the master model selected by the second user is the specialized model “Boruto”.
  • the relationship between the custom model generated by the second user and the specialized model "Bolt" is stored as related information.
  • the user generates second learning data for generating a custom model (step S5).
  • the second learning data may be created by imaging "bolt C'" and other industrial parts using the camera of the communication terminal device 30, like the evaluation data.
  • the second learning data may be created using CAD (Computer Aided Design) data of "Bolt C'". Further, the second learning data may be created by combining learning data obtained by imaging and learning data obtained by CAD.
  • CAD Computer Aided Design
  • the communication terminal device 30 transmits the second learning data to the trained model management device 10 (step S6).
  • the second learning data is stored in the second storage unit 12B of the trained model management device 10.
  • the communication terminal device 30 also instructs the trained model generation device 20 to generate a custom model (step S7).
  • the trained model generation device 20 generates a custom model based on the user-selected master model and the second learning data prepared by the user (step S8).
  • the trained model generation device 20 performs machine learning using the second learning data stored in the accessible second storage unit 12B using the specialized model “Bolt”. to generate a custom model "bolt" that is only available to the second user.
  • the trained model generation device 20 stores the generated custom model in the second storage unit 12B of the trained model management device 10 (step S9).
  • the update processing unit 132 of the trained model management device 10 uses the evaluation data and the generated custom model to execute the task (object recognition) (step S10).
  • the update processing unit 132 of the trained model management device 10 transmits the execution result of the task to the communication terminal device 30 (step S11).
  • a task execution result is displayed on the user interface unit 35 (touch panel display) of the communication terminal device 30 .
  • steps S10 and S11 may be omitted, for example, according to a user's instruction.
  • the update determination unit 131 of the trained model management device 10 determines whether or not to update the master model based on the second learning data.
  • the update determination unit 131 determines to update the master model because the object to be added (“bolt C′”, which is an improved product of “bolt C”) has high versatility (step S12).
  • the update determination unit 131 determines not to update the master model. you can Also, as noted above, the decision may be made by the manager of the model family.
  • the update determination unit 131 may identify some of the master models to be updated by learning using the second learning data based on the update information. Information on some of the identified master models may be output to the update processing unit 132 . As another example, the update determination unit 131 determines that objects to be made recognizable by learning (for example, “bolt A”, “bolt B”, “bolt C”, . . . ) are second learning data (for example, “bolt C′). ”), a trained model generated using the same or similar first training data may be specified as the master model to be updated. Alternatively, as another example, the update determination unit 131 may specify the master model used when generating the custom model as the master model to be updated. The update processing unit 132 updates the master model, which is the learned model identified by the update determination unit 131 .
  • the update processing unit 132 of the trained model management device 10 stores the second learning data in the first storage unit 12A (step S13). That is, since the second learning data is used for updating the master model, the update processing unit 132 copies the second learning data to the first storage unit 12A.
  • the first storage unit 12A stores the second learning data as well as the first learning data.
  • the update processing unit 132 of the learned model management device 10 updates the data indicating the degree of contribution of the second user stored in the second storage unit 12B (step S14). ).
  • the contribution increases the second user's access rights to the master and custom models in proportion to their size.
  • the second storage unit 12B stores the capacity of the second storage unit 12B that can be used by the user, the usage fee, the number of usage times, the usage period, and the like.
  • the update processing unit 132 may set the capacity of the second storage unit 12B, the usage fee, the number of times of usage, and the usage period to be advantageous according to the degree of contribution. This motivates the user to use the second learning data used in generating the custom model to update the master model.
  • the update processing unit 132 of the trained model management device 10 instructs the trained model generation device 20 to update the master model (step S15).
  • the trained model generation device 20 updates the master model by learning (relearning) using the first learning data and the second learning data stored in the first storage unit 12A (step S16). To explain with a specific example, the learned model generation device 20 re-learns the specialized model "bolt” so that "bolt C'" can also be identified. Since the specialized model "Bolt” is a master model, it is available not only to the second user, but also to other users, including the first user.
  • the trained model generation device 20 stores the updated master model in the first storage unit 12A of the trained model management device 10 (step S17).
  • the update processing unit 132 of the trained model management device 10 uses the evaluation data and the updated master model to execute the task (object recognition) (step S18).
  • the update processing unit 132 of the learned model management device 10 transmits the execution result of the task to the communication terminal device 30 (step S19).
  • a task execution result is displayed on the user interface unit 35 (touch panel display) of the communication terminal device 30 .
  • the user evaluates the task execution results, and if the evaluation does not meet expectations, the user may continue to use the custom model instead of using the updated master model. If, based on the evaluation, the user decides to utilize the updated master model, the user may perform an operation to create a new usage environment, for example, and download the updated master model.
  • the communication terminal device 30 transmits updated master model usage information to the trained model management device 10 (step S20). This information is detected by the detection unit 133 of the trained model management device 10 .
  • the user may store the input information input by the robot 2 using the master model or the custom model in the first storage unit 12A or the second storage unit 12B, and use it to evaluate the updated master model.
  • the user stores the data for confirming or evaluating the learning effect that was used when generating the past master model or custom model in the first storage unit 12A or the second storage unit 12B, and uses the updated master model. May be used for evaluation.
  • the evaluation of the updated master model is not limited to execution by the user, and may be automatically executed by the trained model management device 10, for example.
  • the master model before updating may be stored in the first storage unit 12A even when or after updating the master model.
  • the robot 2 can be made to perform the work or perform the updating process again using the pre-updated master model.
  • the update processing unit 132 of the trained model management device 10 deletes the second learning data stored in the second storage unit 12B when the user uses the updated master model (step S21). By deleting the relatively large size of the second learning data, the free space of the second storage unit 12B is increased, and the user can reduce the charge according to the used capacity, for example. Also, the trained model management device 10 can solve the problem of an increase in capacity due to the presence of a large amount of learning data for custom model generation.
  • the update processing unit 132 does not delete the evaluation data stored in the second storage unit 12B. This is because the evaluation data has a relatively small size and is more likely to be used again by the user than the second learning data.
  • the trained model management device 10 and the trained model management method according to the present embodiment can update the master model without deteriorating the recognition accuracy in the usage environment of the user.
  • the present invention is not limited to this. That is, for example, when a user uses a master model at a site such as the user's factory, the master model recognition accuracy may be reduced because the environment in which the master model is created differs from the environment in which the user uses it. At this time, in order to improve the accuracy of the master model, learning data acquired in the user's usage environment or learning data simulating the user's usage environment is used to obtain the master model (first model or first master model). , may generate an optimized master model (second model or second master model).
  • the update determination unit 131 determines whether or not to update the first master model based on the learning data used for learning the second master model. It may be determined whether The second master model may be generated by connecting to the first master model an additional trained model that has undergone learning processing related to the user's specification environment. Note that the additional trained model is also called an adapter module, for example.
  • the second master model is a model that does not have user usage restrictions, so it is stored in the first storage unit 12A in the same manner as the first master model. It should be noted that the second master model can also be called a custom model with no user restrictions. Therefore, in the present disclosure, the custom model of the above embodiment may be read as the second master model and applied as the present invention within a consistent range.
  • the embodiments of the trained model management device 10 and the trained model management method may also take the form of a system, a program, and a storage medium in which the program is recorded. It is possible. Examples of storage media include optical disks, magneto-optical disks, CD-ROMs, CD-Rs, CD-RWs, magnetic tapes, hard disks, memory cards, and the like.
  • the implementation form of the program is not limited to an application program such as an object code compiled by a compiler or a program code executed by an interpreter, and may be in the form of a program module incorporated in an operating system. . Further, the program may or may not be configured so that all processing is performed only in the CPU on the control board. The program may be configured to be partially or wholly executed by another processing unit mounted on an expansion board or expansion unit added to the board as required.
  • Embodiments according to the present disclosure are not limited to any specific configuration of the embodiments described above. Embodiments of the present disclosure extend to any novel feature or combination thereof described in the present disclosure or any novel method or process step or combination thereof described. be able to.
  • Descriptions such as “first” and “second” in this disclosure are identifiers for distinguishing the configurations. Configurations that are differentiated in descriptions such as “first” and “second” in this disclosure may interchange the numbers in that configuration. For example, a first taxonomy can exchange identifiers “first” and “second” with a second taxonomy. The exchange of identifiers is done simultaneously. The configurations are still distinct after the exchange of identifiers. Identifiers may be deleted. Configurations from which identifiers have been deleted are distinguished by codes. The description of identifiers such as “first” and “second” in this disclosure should not be used as a basis for interpreting the order of the configuration or the existence of lower numbered identifiers. For example, it is within the scope of the present invention that the identifiers of "first” and “second” are interchanged in the claims, such as the first model and the second model.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Image Analysis (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
PCT/JP2022/030203 2021-08-05 2022-08-05 学習済みモデル管理装置及び学習済みモデル管理方法 Ceased WO2023013788A1 (ja)

Priority Applications (5)

Application Number Priority Date Filing Date Title
JP2023540442A JP7640710B2 (ja) 2021-08-05 2022-08-05 学習済みモデル管理装置及び学習済みモデル管理方法
EP22853206.5A EP4383150A4 (en) 2021-08-05 2022-08-05 TRAINED MODEL MANAGEMENT DEVICE AND TRAINED MODEL MANAGEMENT METHOD
CN202280054119.XA CN117751371A (zh) 2021-08-05 2022-08-05 学习完毕模型管理装置以及学习完毕模型管理方法
US18/681,203 US20240311702A1 (en) 2021-08-05 2022-08-05 Trained model management device and trained model management method
JP2025026215A JP2025071258A (ja) 2021-08-05 2025-02-20 学習済みモデル管理装置及び学習済みモデル管理方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021129344 2021-08-05
JP2021-129344 2021-08-05

Publications (1)

Publication Number Publication Date
WO2023013788A1 true WO2023013788A1 (ja) 2023-02-09

Family

ID=85154589

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/030203 Ceased WO2023013788A1 (ja) 2021-08-05 2022-08-05 学習済みモデル管理装置及び学習済みモデル管理方法

Country Status (5)

Country Link
US (1) US20240311702A1 (https=)
EP (1) EP4383150A4 (https=)
JP (2) JP7640710B2 (https=)
CN (1) CN117751371A (https=)
WO (1) WO2023013788A1 (https=)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH1074188A (ja) * 1996-05-23 1998-03-17 Hitachi Ltd データ学習装置およびプラント制御装置
WO2019131527A1 (ja) * 2017-12-26 2019-07-04 株式会社エイシング 汎用学習済モデルの生成方法
JP2020144660A (ja) * 2019-03-07 2020-09-10 キヤノン株式会社 情報処理装置、情報処理方法及びプログラム
JP2021129344A (ja) 2020-02-12 2021-09-02 祐次 廣田 生活しやすいｅｖ

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH1074188A (ja) * 1996-05-23 1998-03-17 Hitachi Ltd データ学習装置およびプラント制御装置
WO2019131527A1 (ja) * 2017-12-26 2019-07-04 株式会社エイシング 汎用学習済モデルの生成方法
JP2020144660A (ja) * 2019-03-07 2020-09-10 キヤノン株式会社 情報処理装置、情報処理方法及びプログラム
JP2021129344A (ja) 2020-02-12 2021-09-02 祐次 廣田 生活しやすいｅｖ

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4383150A4

Also Published As

Publication number Publication date
EP4383150A4 (en) 2025-07-30
JP7640710B2 (ja) 2025-03-05
JP2025071258A (ja) 2025-05-02
CN117751371A (zh) 2024-03-22
EP4383150A1 (en) 2024-06-12
JPWO2023013788A1 (https=) 2023-02-09
US20240311702A1 (en) 2024-09-19

Similar Documents

Publication Publication Date Title
US11216656B1 (en) System and method for management and evaluation of one or more human activities
JP2018152063A (ja) 学習結果識別装置、学習結果識別方法、及びそのプログラム
JP2019171501A (ja) ロボットの干渉判定装置、ロボットの干渉判定方法、プログラム
CN103518183A (zh) 图形对象分类
CN114510142B (zh) 基于二维图像的手势识别方法及其系统和电子设备
US20040088144A1 (en) Interference verifying device and method for verifying interference between parts within a device
CN116958426A (zh) 虚拟调试配置方法、装置、计算机设备、存储介质
JP7640710B2 (ja) 学習済みモデル管理装置及び学習済みモデル管理方法
KR20190109652A (ko) 인공지능을 이용하여 생성되는 스타일 공간에 기반한 상품 추천 방법 및 시스템
Xiaoming et al. A model-based approach to assembly sequence planning
CN112669142A (zh) 高维行为数据的建模方法、装置、设备及可读存储介质
JP7568863B2 (ja) ライブラリ提示装置、ライブラリ提示方法及びロボット制御システム
Doshi et al. Problem space transformations for generalisation in behavioural cloning
CN110134236B (zh) 基于Unity3D和Kinect的低动作检测精度下的高交互反馈方法及系统
CN116127990A (zh) 设备测试方法、系统、计算机设备和存储介质
CN117632951A (zh) 算法流程编排方法、装置、计算机设备及存储介质
CN115114683A (zh) 用于将自主技能执行中的约束反馈到设计中的系统与方法
CN111185902B (zh) 基于视觉识别的机器人文字书写方法、装置和书写系统
CN114495257B (zh) 险态工况下融合行为机制的姿态预测方法和装置
Berger SenseDSL: Automating the integration of sensors for mcu-based robots and cyber-physical systems
CN117975555A (zh) 用户操作反馈方法、装置、计算机设备和存储介质
JP6916330B2 (ja) 画像解析プログラムの自動ビルドの方法およびシステム
CN111797879A (zh) 模型训练方法、装置、存储介质及电子设备
CN119597138A (zh) 交互方法及相关设备
Heinerud et al. Automatic testing of graphical user interfaces

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22853206

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023540442

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 202280054119.X

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2022853206

Country of ref document: EP

Effective date: 20240305