WO2023042895A1 - 学習済みモデル生成方法、推論装置、及び学習済みモデル生成装置 - Google Patents

学習済みモデル生成方法、推論装置、及び学習済みモデル生成装置 Download PDF

Info

Publication number
WO2023042895A1
WO2023042895A1 PCT/JP2022/034632 JP2022034632W WO2023042895A1 WO 2023042895 A1 WO2023042895 A1 WO 2023042895A1 JP 2022034632 W JP2022034632 W JP 2022034632W WO 2023042895 A1 WO2023042895 A1 WO 2023042895A1
Authority
WO
WIPO (PCT)
Prior art keywords
model
trained
learning
information
trained model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/034632
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
南己 淺谷
和久 荒川
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Kyocera Corp
Original Assignee
Kyocera Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kyocera Corp filed Critical Kyocera Corp
Priority to JP2023548510A priority Critical patent/JP7693010B2/ja
Priority to US18/692,761 priority patent/US20240289695A1/en
Priority to CN202280062804.7A priority patent/CN118119955A/zh
Publication of WO2023042895A1 publication Critical patent/WO2023042895A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Definitions

  • the present disclosure relates to a trained model generation method, an inference device, and a trained model generation device.
  • Patent Document 1 a system is known in which a robot uses a trained model to extract an object (see Patent Document 1, for example).
  • a trained model generation method is a learning model that outputs a recognition result of a recognition target included in input information based on a plurality of models each having at least one of a first part and a second part. Including generating a finished model.
  • a plurality of base models are obtained, each having a portion corresponding to the first part, trained based on at least one set of first information associated with the input information.
  • each part corresponding to the second part is trained based on at least one set of second information related to the input information while being connected to the plurality of base models.
  • a target model is obtained.
  • a trained model having a target model having at least a portion corresponding to the second part is generated.
  • a reasoning apparatus is generated based on a plurality of models each having at least one of a first part and a second part, and is included in input information. Equipped with ready-made models.
  • Said trained model comprises at least a target model having portions each corresponding to said second part.
  • a target model, each having a portion corresponding to the second part is connected to a plurality of base models, each having a portion corresponding to the first part, to at least one set of second information related to the input information.
  • It is a model obtained by learning based on A plurality of base models, each having a portion corresponding to the first part are models trained based on at least one set of first information associated with the input information.
  • a trained model generation device is a learning model that outputs a recognition result of a recognition target included in input information based on a plurality of models each having at least one of a first part and a second part.
  • a controller for generating a finished model is provided.
  • the control unit acquires a plurality of base models each having a part corresponding to the first part, which are learned based on at least one set of first information related to the input information in generating the trained model. do.
  • the control unit creates a target model having a portion corresponding to each of the second units, which is learned based on at least one set of second information related to the input information while being connected to the plurality of base models. get.
  • the control unit generates a trained model having a target model having at least a portion corresponding to the second part.
  • FIG. 4 is a schematic diagram showing an example of an operation of generating a plurality of preliminary models with different backbones through learning;
  • FIG. 4 is a schematic diagram showing an example of an operation of generating a head by learning using a backbone of a preliminary model;
  • FIG. 4 is a schematic diagram showing an example of an operation of generating a plurality of preliminary models with different heads through learning;
  • FIG. 4 is a schematic diagram showing an example of an operation of generating a backbone by learning using a preliminary model head;
  • FIG. 4 is a schematic diagram showing an example of an operation of generating a backbone by learning using a preliminary model head;
  • FIG. 4 is a schematic diagram showing an example of an operation of generating a backbone by learning using a preliminary model head;
  • FIG. 4 is a schematic diagram showing both the generation of a preliminary model and the generation of a head;
  • FIG. 4 is a schematic diagram showing both the generation of a preliminary model and the generation of a backbone;
  • 4 is a flow chart showing an example procedure of a learned model generation method;
  • 1 is a schematic diagram showing a configuration example of a robot control system;
  • FIG. 10 is a diagram showing a configuration example in which a trained model includes a branch model;
  • FIG. 4 is a diagram showing a configuration example of a model in which a first part is connected between two divided second parts;
  • a trained model generation system 1 (Configuration example of trained model generation system 1) generates a trained model 50 (see FIG. 2, etc.) that outputs a recognition result of a recognition target included in input information.
  • the trained model generation system 1 generates a plurality of preliminary models in preparation for generating the trained model 50, updates the model by learning a model in which a part of each preliminary model is connected, and generates a trained model. Generate 50.
  • the trained model generation system 1 can improve the inference accuracy of the trained model 50 by learning a model in which a plurality of preliminary models are connected. Also, the trained model generation system 1 can reduce the workload for generating the trained model 50 by generating the trained model 50 using part of the preliminary model.
  • the preliminary model is also called a base model.
  • a model generated by learning on a model connecting preliminary models is also called a target model.
  • a trained model generation system 1 includes a preliminary model generation device 10 and a trained model generation device 20.
  • the trained model generation system 1 generates a preliminary model by the preliminary model generation device 10 and generates a trained model 50 by the trained model generation device 20 .
  • the preliminary model generation device 10 and the trained model generation device 20 may be configured as separate devices, or may be configured as an integrated device.
  • the preliminary model generation device 10 includes a first control section 12 and a first interface 14 .
  • the trained model generation device 20 includes a second control section 22 and a second interface 24 .
  • the designations "first" and “second” are provided merely to distinguish between the features included in each of the different devices.
  • the first control unit 12 and the second control unit 22 are also simply referred to as control units.
  • the first interface 14 and the second interface 24 are also simply referred to as interfaces.
  • the controller may include at least one processor to provide control and processing power to perform various functions.
  • the processor may execute programs that implement various functions of the controller.
  • a processor may be implemented as a single integrated circuit.
  • An integrated circuit is also called an IC (Integrated Circuit).
  • a processor may be implemented as a plurality of communicatively coupled integrated and discrete circuits. Processors may be implemented based on various other known technologies.
  • the control unit may have a storage unit.
  • the storage unit may include an electromagnetic storage medium such as a magnetic disk, or may include a memory such as a semiconductor memory or a magnetic memory.
  • the storage unit stores various information.
  • the storage unit stores programs and the like executed by the control unit.
  • the storage unit may be configured as a non-transitory readable medium.
  • the storage unit may function as a work memory for the control unit. At least part of the storage section may be configured separately from the control section.
  • the first interface 14 of the preliminary model generation device 10 and the second interface 24 of the trained model generation device 20 input and output information or data to each other.
  • the first interface 14 outputs information or data acquired from the first control unit 12 to the learned model generation device 20 and outputs information or data acquired from the learned model generation device 20 to the first control unit 12 .
  • the second interface 24 outputs information or data acquired from the preliminary model generation device 10 to the second control section 22 .
  • the interface may comprise a communication device configured to communicate wiredly or wirelessly.
  • the interface is also called communication part.
  • a communication device may be configured to be able to communicate with communication schemes based on various communication standards.
  • the interface can be configured with known communication technology.
  • the trained model 50 is represented as a model in which a first trained model 51 and a second trained model 52 are connected.
  • the first trained model 51 is also called backbone.
  • the second trained model 52 is also called head.
  • the backbone is configured to output the result of extracting the feature quantity of the input information.
  • the feature amount represents, as a numerical value, an appearance feature such as an edge or pattern to be learned.
  • a backbone may include, for example, convolution and pooling.
  • the head is configured to make predetermined decisions about the input information based on the output of the backbone. Specifically, the head may output the recognition result of the recognition target included in the input information based on the extraction result of the feature amount of the input information output by the backbone.
  • the head may include a fully connected layer that processes the result of feature extraction by the backbone. Note that the head may also include convolution and pooling.
  • the head is configured to perform recognition of a recognition target as a predetermined decision.
  • the feature quantity can be a parameter representing the ratio of striped area on the body surface.
  • the predetermined determination may be to determine whether the recognition target is a horse or a zebra by comparing the area ratio of the striped pattern on the body surface with a threshold value.
  • the feature quantity may be a parameter representing the size or the number of holes in the shell.
  • the predetermined determination may be comparing the size or the number of holes in the shell with a threshold value to determine whether the recognition target is an abalone or a tokobushi.
  • the trained model 50 may be configured including a CNN (Convolution Neural Network) having multiple layers. Information input to the trained model 50 is subjected to convolution based on predetermined weighting factors in each layer of the CNN. In learning the trained model 50, the weighting factors are updated.
  • the trained model 50 may be configured including fully connected layers.
  • the trained model 50 may be configured by VGG16 or ResNet50.
  • Trained model 50 may be configured as a transformer. The trained model 50 is not limited to these examples, and may be configured as various other models.
  • the trained model generation device 20 In the trained model generation system 1, the trained model generation device 20 generates or obtains in advance a plurality of preliminary models including a backbone and a head. The trained model generation device 20 prepares one learning head in order to generate the head portion of the trained model 50 . The trained model generation device 20 sequentially connects the backbones of the plurality of preliminary models to one learning head. The trained model generation device 20 performs learning on a model in which the backbone of each preliminary model and the learning head are connected, and updates the learning head. The trained model generation device 20 sequentially connects the backbone of each preliminary model to the learning head, executes learning for each model, and updates the learning head.
  • the trained model generating device 20 applies the trained head obtained when learning of the model connecting the backbones of the preliminary models is completed as the head of the trained model 50 . Also, the trained model generation device 20 generates or acquires the backbone of the trained model 50 separately. The trained model generation device 20 generates a trained model 50 by connecting a trained head to a separately generated or acquired backbone.
  • the trained model generating device 20 may generate the backbone part of the trained model 50 by learning a model in which the head parts of each preliminary model are connected in sequence.
  • the trained model 50 is generated by sequentially connecting the preliminary models to the learning model for both the backbone portion and the head portion of the trained model 50 and performing learning. You may
  • the trained model generation device 20 generates the trained model 50 by updating the pre-learning model through learning.
  • the pre-learning model is a model obtained by connecting a first pre-learning model corresponding to the first trained preliminary model 41 and a second pre-learning model corresponding to the second trained preliminary model 42 .
  • the trained model generation device 20 updates the second pre-learned model by learning the first trained preliminary model 41 in the model connected to the second pre-learned model instead of the first pre-learned model.
  • a trained model 52 may be generated.
  • the trained model generation device 20 updates the first pre-learned model to generate the first pre-learned model by learning the second pre-learned preliminary model 42 instead of the second pre-learned model in the model connected to the first pre-learned model.
  • a trained model 51 may be generated.
  • the model is configured to have at least one of a first part and a second part. That is, the base model is configured to have at least one of a first part and a second part.
  • the first pre-learning model of the pre-learning models corresponds to the first part of the model.
  • a second pre-learning model of pre-learning models corresponds to the second part of the model.
  • a first trained model 51 of trained models 50 corresponds to the first part of the model.
  • a second trained model 52 of trained model 50 corresponds to the second part of the model.
  • the first control unit 12 of the preliminary model generation device 10 generates or acquires a plurality of pre-learning preliminary models 301 to 30N in advance, as shown in FIG.
  • Pre-learning preliminary models 301 to 30N are also collectively referred to as pre-learning preliminary model 30 .
  • the pre-learning preliminary model 301 includes a first pre-learning preliminary model 311 and a second pre-learning preliminary model 321 .
  • the pre-learning preliminary model 30N includes a first pre-learning preliminary model 31N and a second pre-learning preliminary model 32N.
  • the first pre-learning preliminary models 311-31N of the pre-learning preliminary model 30 correspond to the first part of the model.
  • the second pre-learning preliminary models 321-32N of the pre-learning preliminary model 30 correspond to the second part of the model.
  • FIG. 3 it is assumed that the CNN layer configuration or the filter size of each layer in the first pre-learning preliminary model 311 and the CNN layer configuration or the filter size of each layer in the first pre-learning preliminary model 31N are different from each other.
  • the configuration of CNN layers or the filter size of each layer in each of the first pre-learning preliminary models 311 to 31N is different from each other.
  • Filter size refers to the size of the filter used to perform convolution (downsampling) or transposed convolution (upsampling) in the CNN.
  • the model configurations of the first pre-learning preliminary models 311 to 31N may be different from each other or may be the same.
  • the fully connected layer configuration or the parameter size of each layer in the second pre-learning preliminary model 321 is the same as the fully connected layer configuration or the parameter size of each layer in the second pre-learning preliminary model 32N.
  • the fully-connected layer configuration or the parameter size of each layer is the same in each of the second pre-learning preliminary models 321 to 32N.
  • the fully-connected layer configuration or the parameter size of each layer in each of the second pre-learning preliminary models 321 to 32N may be different from each other.
  • the parameter size refers to the number of units constituting a fully connected layer, and the like.
  • the model configurations of the second pre-learning preliminary models 321 to 32N may be different from each other or may be the same.
  • the first control unit 12 learns each pre-learning preliminary model 30 using the first information that is the same as or related to the input information input to the trained model 50 as learning data.
  • the first information may be composed of a set of a plurality of learning images.
  • the first control unit 12 may learn the same set of first information as learning data, or may learn a different set of first information as learning data. That is, the first control unit 12 may perform learning of the pre-learning preliminary model 30 based on at least one set of first information.
  • the first control unit 12 updates each pre-learning preliminary model 30 by learning to generate a plurality of trained preliminary models 401 to 40N. Trained preliminary models 401-40N are also collectively referred to as trained preliminary model 40.
  • the learning data may include teacher data used in so-called supervised learning.
  • the learning data may include data generated by the device itself that performs learning, which is used in so-called unsupervised learning.
  • the trained preliminary model 401 includes a first trained preliminary model 411 and a second trained preliminary model 421 .
  • Trained preliminary models 40N include a first trained preliminary model 41N and a second trained preliminary model 42N.
  • the first trained preliminary models 411-41N of the trained preliminary model 40 correspond to the first part of the base model.
  • the second trained preliminary models 421-42N of the trained preliminary model 40 correspond to the second part of the base model.
  • the CNN layer configuration or the filter size of each layer in the first trained preliminary models 411 to 41N is the same as the CNN layer configuration or the filter size of each layer in the first pre-learning preliminary models 311 to 31N.
  • the fully-connected layer configuration or the parameter size of each layer in the second trained preliminary models 421 to 42N is the same as the fully-connected layer configuration or the parameter size of each layer in the second pre-learning preliminary models 321 to 32N.
  • the second control unit 22 of the trained model generation device 20 acquires the trained preliminary model 40 from the preliminary model generation device 10 as a preliminary model.
  • the preliminary model generation device 10 may output the trained preliminary model 40 to the trained model generation device 20 via the first interface 14 .
  • the trained model generation device 20 may acquire the trained preliminary model 40 from the preliminary model generation device 10 via the second interface 24 .
  • the second control unit 22 uses the second information that is the same as or related to the input information that is input to the trained model 50 as learning data, and executes learning for the model that connects the backbone of each preliminary model and the learning head. do.
  • the backbone of each preliminary model corresponds to the first part of the base model.
  • the training head corresponds to the second part of the target model.
  • the second information may be the same as or different from the first information.
  • the second information may be composed of a set of multiple learning images.
  • the second control unit 22 may learn the same set of second information as learning data for each model in which the backbone of each preliminary model and the learning head are connected, or may learn a different set of second information. may be learned as learning data.
  • the second control unit 22 further divides one set of second information into smaller groups, and learns different groups as learning data each time the backbone connected to the learning head is changed. you can The second control unit 22 may learn the same small group as learning data when the backbone connected to the learning head is changed. That is, the second control unit 22 may perform learning of a model in which the backbone of each preliminary model and the learning head are connected based on at least one set of second information.
  • the information amount of the second information used for learning the trained model 50 may be less than or equal to the information amount of the first information used for learning the preliminary model. Note that the amount of information refers to, for example, the number of learning images included in the second information.
  • the second control unit 22 transfers the backbone of each preliminary model (first trained preliminary models 411 to 41N) to a second pre-learning model 520 or a second learning model 520 Learning is sequentially executed for the models connected to the models 521 to 52(N-1).
  • the 2nd control part 22 may perform learning by the following procedures.
  • the second control unit 22 performs learning on a model connecting the first trained preliminary model 411 and the second pre-learning model 520 , and updates the second pre-learning model 520 to the second learning model 521 .
  • the second control unit 22 performs learning on a model connecting the first trained preliminary model 412 and the second learning model 521 updated in the previous step, and converts the second learning model 521 to the second learning model. 522.
  • the second control unit 22 performs learning on a model connecting the first trained preliminary model 41N and the second learning model 52 (N ⁇ 1) updated in the previous step, and performs learning on the second learning model 52 ( N-1) is updated to the second trained model 52N.
  • the second control unit 22 generates the second trained model 52N by executing the procedure described above.
  • the second control unit 22 applies the second trained model 52N as the second trained model 52 (the head of the trained model 50).
  • a second trained model 52N generated by learning by connecting to the first part of the base model corresponds to the second part of the target model.
  • the first control unit 12 of the preliminary model generation device 10 generates or acquires a plurality of pre-learning preliminary models 301 to 30N in advance, as shown in FIG.
  • FIG. 5 it is assumed that the fully-connected layer configuration or the parameter size of each layer in the second pre-learning preliminary model 321 and the fully-connected layer configuration or the parameter size of each layer in the second pre-learning preliminary model 32N are different from each other. .
  • the fully-connected layer configuration or the parameter size of each layer is different in each of the second pre-learning preliminary models 321 to 32N.
  • the first control unit 12 learns each pre-learning preliminary model 30 using the first information that is the same as or related to the input information input to the trained model 50 as learning data.
  • the first control unit 12 updates each pre-learning preliminary model 30 by learning to generate a plurality of trained preliminary models 401 to 40N.
  • the full connection or CNN layer configuration or the parameter size or filter size of each layer of each trained preliminary model 40 is the same as the full connection or CNN layer configuration or the parameter size or filter size of each layer of each pre-learning preliminary model 30 is.
  • the second control unit 22 of the trained model generation device 20 acquires the trained preliminary model 40 from the preliminary model generation device 10 as a preliminary model.
  • the second control unit 22 uses the third information, which is the same as or related to the input information input to the trained model 50, as learning data, and executes learning for the model in which the head of each preliminary model is connected to the learning backbone. do.
  • Each preliminary model head corresponds to the second part of the base model.
  • the training backbone corresponds to the first part of the target model.
  • the third information may be the same as or different from the first information or the second information.
  • the third information may be composed of a set of multiple learning images.
  • the second control unit 22 may learn the same set of third information as learning data for each model in which the head of each preliminary model and the learning backbone are connected, or may learn a different set of third information. may be learned as learning data. In addition, the second control unit 22 further divides one set of third information into smaller small groups, and learns different small groups as learning data each time the head connected to the learning backbone is changed. you can The second control unit 22 may learn the same small group as learning data when the head connected to the learning backbone is changed. That is, the second control unit 22 may perform learning of a model in which the head of each preliminary model is connected to the learning backbone based on at least one set of third information.
  • the information amount of the third information used for learning the trained model 50 may be less than or equal to the information amount of the first information used for learning the preliminary model. Note that the amount of information refers to, for example, the number of learning images included in the second information.
  • the second control unit 22 transfers the head of each preliminary model (second trained preliminary models 421 to 42N) to the first pre-learning model 510 or the first learning model 510. Learning is sequentially executed for the models connected to the models 511 to 51(N-1).
  • the 2nd control part 22 may perform learning by the following procedures.
  • the second control unit 22 performs learning on a model connecting the second trained preliminary model 421 and the first pre-learning model 510 , and updates the first pre-learning model 510 to the first learning model 511 .
  • the second control unit 22 performs learning on a model connecting the second trained preliminary model 422 and the first learning model 511 updated in the previous step, and converts the first learning model 511 to the first learning model. Update to 512.
  • the second control unit 22 performs learning on a model connecting the second trained preliminary model 42N and the first learning model 51 (N ⁇ 1) updated in the previous step, and performs learning on the first learning model 51 ( N ⁇ 1) is updated to the first trained model 51N.
  • the second control unit 22 generates the first trained model 51N by executing the procedure described above.
  • the second control unit 22 applies the first trained model 51N as the first trained model 51 (the backbone of the trained model 50).
  • the first trained model 51N generated by learning by connecting to the second part of the base model corresponds to the first part of the target model.
  • the second control unit 22 of the trained model generating device 20 connects the first trained model 51 and the second trained model 52 to generate the trained model 50 .
  • the second control unit 22 connects the first trained model 51 (backbone) to the generated second trained model 52 to complete the learning process.
  • Generate model 50 The second control unit 22 may generate the first trained model 51 by another means, or may acquire it from another device.
  • the second control unit 22 may acquire at least one of the plurality of first trained preliminary models 41 as the first trained model 51 .
  • the second control unit 22 connects the second trained model 52 (head) to the generated first trained model 51 to complete the training. Generate model 50 .
  • the second control unit 22 may generate the second trained model 52 by another means, or may acquire it from another device.
  • the second control unit 22 may acquire at least one of the plurality of second trained preliminary models 42 as the second trained model 52 .
  • the second control unit 22 may generate both the first trained model 51 (backbone) and the second trained model 52 (head) based on the preliminary model.
  • the second control unit 22 connects the first trained model 51 and the second trained model 52 generated based on the preliminary model to generate the trained model 50 .
  • the second control unit 22 may generate a trained model 50 that solely includes a second trained model 52 generated by learning by connecting to the first trained preliminary model 41 .
  • the second control unit 22 may generate a trained model 50 that solely includes the first trained model 51 generated by learning by connecting to the second trained preliminary model 42 .
  • the preliminary model generation device 10 generates a preliminary model
  • the trained model generation device 20 generates a trained model 50 based on the preliminary model.
  • the trained model generation system 1 learns the first information as learning data in the preliminary model generation device 10, thereby generating a first trained preliminary model 41 and a second trained preliminary model.
  • a trained preliminary model 40 is generated that includes a model 42 and a model 42 .
  • the trained model generation system 1 may transfer the first trained preliminary model 41 to the trained model generation device 20 as shown in FIG.
  • the trained model generation device 20 connects the first trained preliminary models 411 to 41N to the second pre-learning model 520 or the second learning models 521 to 52(N-1), respectively, and uses the second information for learning.
  • a second training model 521-52(N ⁇ 1) or a second trained model 52N may be generated by learning as data.
  • the trained model generation system 1 may transfer the second trained preliminary model 42 to the trained model generation device 20 as shown in FIG.
  • the trained model generation device 20 connects the second trained preliminary models 421 to 42N to the first pre-learning model 510 or the first learning models 511 to 51(N ⁇ 1), respectively, and uses the third information for learning.
  • the first training model 511-51(N ⁇ 1) or the first trained model 51N may be generated by learning as data.
  • the trained model generating device 20 may apply the generated second trained model 52N as the second trained model 52.
  • the trained model generation device 20 may apply any one of the second learning models 521 to 52 (N ⁇ 1) as the second trained model 52 .
  • the trained model generation device 20 may apply the generated first trained model 51 N as the first trained model 51 .
  • the trained model generation device 20 may apply any one of the first learning models 511 to 51 (N ⁇ 1) as the first trained model 51 .
  • the trained model generation device 20 generates the trained model 50 by connecting the first trained model 51 to the generated second trained model 52 .
  • the trained model generation device 20 may select the first trained model 51 to be connected to the generated second trained model 52 from among the plurality of first trained preliminary models 41 .
  • the trained model generation device 20 may acquire the first trained model 51 to be connected to the generated second trained model 52 from an external device.
  • the trained model generation device 20 generates the trained model 50 by connecting the second trained model 52 to the generated first trained model 51 .
  • the trained model generation device 20 may select the second trained model 52 to be connected to the generated first trained model 51 from among the plurality of second trained preliminary models 42 .
  • the trained model generating device 20 may acquire the second trained model 52 connected to the generated first trained model 51 from an external device.
  • the trained model generating device 20 may generate the trained model 50 by connecting the generated first trained model 51 and the generated second trained model 52 .
  • the trained model generation device 20 may execute a method of generating the trained model 50 including the procedure of the flowchart illustrated in FIG. 9 .
  • the method of generating the trained model 50 may be implemented as a training program for generating the trained model 50 that is executed by the processor constituting the second control unit 22 of the trained model generating device 20 .
  • a program for generating the trained model 50 may be stored in a non-transitory computer-readable medium.
  • the second control unit 22 acquires a plurality of trained preliminary models 40 from the preliminary model generation device 10 (step S1).
  • the second control unit 22 generates a model connecting the first trained preliminary model 41 of each trained preliminary model 40 and the second pre-learned model 520 (step S2).
  • the second control unit 22 updates the second pre-learning model 520 by learning the model generated in step S2, and generates a second learning model 521 (step S3).
  • the second control unit 22 determines whether all the first learned preliminary models 41 of the plurality of trained preliminary models 40 have been connected (step S4). If all of the first trained preliminary models 41 have not been connected (step S4: NO), the second control unit 22 returns to the procedure of step S2, and transfers the unconnected first trained preliminary models 41 to the second model. A model connected to the learning models 521 to 52 (N-1) is generated. In addition, the second control unit 22 updates the second learning models 521 to 52 (N-1) in the procedure of step S3, and the second learning models 522 to 52 (N-1) or the second trained models 52N.
  • step S4 When all the first trained preliminary models 41 are connected (step S4: YES), the second control unit 22 connects the second trained preliminary model 42 of each trained preliminary model 40 and the first pre-learned model 510. generated model (step S5). The second control unit 22 updates the first pre-learning model 510 by learning the model generated in step S5, and generates the first learning model 511 (step S6).
  • the second control unit 22 determines whether or not all the second learned preliminary models 42 of the plurality of trained preliminary models 40 have been connected (step S7). If all of the second trained preliminary models 42 have not been connected (step S7: NO), the second control unit 22 returns to the procedure of step S5, and connects the unconnected second trained preliminary models 42 to the first A model connected to the learning models 511 to 51(N-1) is generated. In addition, the second control unit 22 updates the first learning models 511 to 51 (N-1) in the procedure of step S6, and the first learning models 512 to 51 (N-1) or the first trained model 51N.
  • the second control unit 22 connects the first trained model 51 and the second trained model 52 to generate the trained model 50. (step S8). Specifically, the second control unit 22 applies the first trained model 51N generated by updating in the procedure of steps S2 and S3 as the first trained model 51. FIG. The second control unit 22 applies the second learned model 52N generated by updating in the procedure of steps S5 and S6 as the second learned model 52. FIG. After executing the procedure of step S8, the second control unit 22 ends the execution of the procedure of the flowchart of FIG. After executing the procedure of step S4, the second control unit 22 may proceed to the procedure of step S8 without executing the procedure of steps S5 to S7. After executing the procedure of step S1, the second control unit 22 may proceed to the procedure of step S5 without executing the procedure of steps S2 to S4.
  • the trained model generation system 1 and the trained model generation device 20 generate a plurality of preliminary models, and generate the trained model 50 using each preliminary model.
  • the trained model generation device 20 connects a part of a plurality of preliminary models to a learning model corresponding to the first trained model 51 or the second trained model 52, which is part of the trained model 50. to generate
  • the trained model generation device 20 learns the generated model and generates the first trained model 51 or the second trained model 52 by updating the learning model.
  • the trained model generating device 20 generates a trained model 50 using the generated first trained model 51 or second trained model 52 .
  • the trained model generation system 1 uses information that is the same as or related to the input information that is input to the trained model 50 as learning data.
  • the information that is the same as or related to the input information may be information of a task that is the same as or related to the task executed by the trained model 50 that receives the input information.
  • an example of input information is an image depicting organisms including mammals.
  • the information about the learning object generated as the information of the same task as the input information is an image of a mammal.
  • the information about the learning target generated as the information of the task related to the input information is, for example, an image of a reptile.
  • a task may include, for example, a classification task that classifies recognition targets included in input information into at least two types.
  • the classification task can be subdivided into, for example, a task of distinguishing whether the recognition target is a dog or a cat, or a task of distinguishing whether the recognition target is a cow or a horse.
  • Tasks are not limited to classification tasks, and may include tasks that implement various other operations.
  • a task may include a segmentation determining from pixels belonging to a particular object.
  • a task may include object detection to detect an enclosing rectangular region.
  • the task may include object pose estimation.
  • a task may include keypoint detection to find certain feature points.
  • both the input information and the information about the learning target are classification task information
  • the relationship between the input information and the information about the learning target is assumed to be related task information.
  • both the input information and the information about the learning target are task information for distinguishing whether the recognition target is a dog or a cat
  • the relationship between the input information and the information about the learning target is the same. task information.
  • the relationship between the input information and the learning target information is not limited to these examples, and can be determined under various conditions.
  • the second control unit 22 of the trained model generation device 20 combines the second information or the third information used as learning data for learning with the first information used as learning data for learning for generating the preliminary model.
  • the second control unit 22 specializes only screws among industrial parts.
  • Information for recognizing detailed types may be used as the second information or the third information.
  • the second control unit 22 specializes only in dogs among animals, and selects detailed types of dogs. may be used as the second information or the third information.
  • the trained model 50 that recognizes broad categories such as industrial parts or animals is also called a general-purpose model.
  • a trained model 50 that recognizes a narrow classification such as the type of screw or the type of dog is also called a dedicated model.
  • the second control unit 22 may make the granularity of the second information or the third information smaller than the granularity of the first information.
  • Granularity of information means fineness of classification of recognition targets. For example, assume that the trained model 50 recognizes an industrial part as a recognition target. The granularity of information for classifying industrial parts into screws, nuts, washers, brackets, etc. is greater than the granularity of information classifying screws by length, diameter, or the like. In other words, the granularity of information differs depending on the large classification, medium classification, or small classification of the recognition target. The smaller the granularity of the information used as the learning data, the finer the difference in the recognition target can be recognized by the trained model 50 .
  • the trained model 50 may not be able to recognize large differences in recognition targets.
  • a trained model 50 that can recognize the difference in screw length or diameter may not be able to recognize the difference between a screw and a nut.
  • a trained model 50 generated by learning using large granularity information as learning data corresponds to a general-purpose model.
  • a trained model 50 generated by learning using small granularity information as learning data corresponds to a dedicated model.
  • the second control unit 22 of the trained model generating device 20 may evaluate the recognition accuracy of the recognition target using the generated trained model 50 .
  • the second control unit 22 may regenerate the trained model 50 based on the recognition accuracy evaluation result.
  • the second control unit 22 acquires the recognition result output from the trained model 50 when input information is input to the generated trained model 50 .
  • the second control unit 22 may input information for which the correct recognition result is known to the trained model 50 as input information, and evaluate the ratio (accuracy rate) at which the obtained recognition result matches the correct recognition result.
  • the second control unit 22 may calculate the accuracy rate as the evaluation value. In this case, the higher the evaluation value, the higher the recognition accuracy of the trained model 50 .
  • the second control unit 22 may determine that the recognition accuracy of the generated trained model 50 is sufficient when the evaluation value is equal to or greater than a predetermined threshold.
  • the second control unit 22 may determine that the recognition accuracy of the generated trained model 50 is insufficient when the evaluation value is less than a predetermined threshold.
  • the trained model generation system 1 may regenerate the trained model 50.
  • the second control unit 22 may change the second information or the third information used as learning data in the learning executed before regenerating the trained model 50. .
  • the second control unit 22 does not change the second information or the third information used as learning data in the learning executed before regenerating the trained model 50, and learns the same information as learning data.
  • the second control unit 22 uses the combination of the preliminary model connected to the model for learning and the set of the second information or the third information for the learning executed before regenerating the trained model 50. You may change from the combination in the data for learning. Further, when dividing the second information or the third information into small groups, the second control unit 22 selects a combination of the preliminary model connected to the model for learning and the small group of the second information or the third information for learning. The combination in the learning data used for the learning executed before regenerating the finished model 50 may be changed.
  • the second control unit 22 may change the order of information used as learning data with respect to the order of changing the combination of the learning model and the preliminary model. That is, the second control unit 22 may shuffle the order of information used as learning data. The second control unit 22 changes the information used as the learning data or sets the learning data for the combination of the learning model and the preliminary model until the recognition accuracy of the trained model 50 reaches a predetermined accuracy or higher. The trained model 50 may be regenerated by changing the order of the information to be used.
  • the second control unit 22 may change the configuration of the small group of the second information or the third information. That is, the second control unit 22 may regenerate the trained model 50 by changing the content of the small group while using the same set of learning data.
  • the method of evaluating and regenerating the trained model 50 may also be applied to learning using the first information.
  • the second control unit 22 may generate the target model by re-learning. In this case, the second control unit 22 may regenerate the trained model 50 as a new target model by learning based on new learning data without using a preliminary model (base model). Note that the new learning data is also called fourth information.
  • the fourth information may be information that is the same as or related to the input information.
  • a robot control system 100 includes a robot 2 and a robot control device 110 .
  • the robot 2 moves the work object 8 from the work start point 6 to the work target point 7 . That is, the robot control device 110 controls the robot 2 so that the work object 8 moves from the work start point 6 to the work target point 7 .
  • the work object 8 is also referred to as work object.
  • the robot control device 110 controls the robot 2 based on information regarding the space in which the robot 2 works. Information about space is also referred to as spatial information.
  • the robot 2 has an arm 2A and an end effector 2B.
  • the arm 2A may be configured as, for example, a 6-axis or 7-axis vertical articulated robot.
  • the arm 2A may be configured as a 3-axis or 4-axis horizontal articulated robot or SCARA robot.
  • the arm 2A may be configured as a 2-axis or 3-axis Cartesian robot.
  • Arm 2A may be configured as a parallel link robot or the like.
  • the number of shafts forming the arm 2A is not limited to the illustrated one.
  • the robot 2 has an arm 2A connected by a plurality of joints and operates by driving the joints.
  • the end effector 2B may include, for example, a gripping hand configured to grip the work object 8.
  • the grasping hand may have multiple fingers. The number of fingers of the grasping hand may be two or more. The fingers of the grasping hand may have one or more joints.
  • the end effector 2B may include a suction hand configured to be able to suction the work object 8 .
  • the end effector 2B may include a scooping hand configured to scoop the work object 8 .
  • the end effector 2 ⁇ /b>B includes a tool such as a drill, and may be configured to be able to perform various machining operations such as drilling a hole in the work object 8 .
  • the end effector 2B is not limited to these examples, and may be configured to perform various other operations. In the configuration illustrated in FIG. 10, the end effector 2B is assumed to include a grasping hand.
  • the robot 2 can control the position of the end effector 2B by operating the arm 2A.
  • the end effector 2 ⁇ /b>B may have an axis that serves as a reference for the direction in which it acts on the work object 8 . If the end effector 2B has an axis, the robot 2 can control the direction of the axis of the end effector 2B by operating the arm 2A.
  • the robot 2 controls the start and end of the action of the end effector 2B acting on the work object 8 .
  • the robot 2 can move or process the workpiece 8 by controlling the position of the end effector 2B or the direction of the axis of the end effector 2B and controlling the operation of the end effector 2B. In the configuration illustrated in FIG.
  • the robot 2 causes the end effector 2B to grip the work object 8 at the work start point 6 and moves the end effector 2B to the work target point 7 .
  • the robot 2 causes the end effector 2B to release the work object 8 at the work target point 7 . By doing so, the robot 2 can move the work object 8 from the work start point 6 to the work target point 7 .
  • the robot control system 100 further comprises a sensor 3, as shown in FIG. A sensor 3 detects physical information of the robot 2 .
  • the physical information of the robot 2 may include information on the actual position or orientation of each constituent part of the robot 2 or the velocity or acceleration of each constituent part of the robot 2 .
  • the physical information of the robot 2 may include information about forces acting on each component of the robot 2 .
  • the physical information of the robot 2 may include information about the current flowing through the motors that drive each component of the robot 2 or the torque of the motors.
  • the physical information of the robot 2 represents the result of the actual motion of the robot 2 . In other words, the robot control system 100 can grasp the result of the actual motion of the robot 2 by acquiring the physical information of the robot 2 .
  • the sensor 3 may include a force sensor or a tactile sensor that detects force acting on the robot 2, distributed pressure, slip, or the like as physical information of the robot 2.
  • the sensor 3 may include a motion sensor that detects the position or posture, or the speed or acceleration of the robot 2 as the physical information of the robot 2 .
  • the sensor 3 may include a current sensor that detects the current flowing through the motor that drives the robot 2 as the physical information of the robot 2 .
  • the sensor 3 may include a torque sensor that detects the torque of the motor that drives the robot 2 as the physical information of the robot 2 .
  • the sensor 3 may be installed in a joint of the robot 2 or in a joint driving section that drives the joint.
  • the sensor 3 may be installed on the arm 2A of the robot 2 or the end effector 2B.
  • the sensor 3 outputs the detected physical information of the robot 2 to the robot control device 110 .
  • the sensor 3 detects and outputs physical information of the robot 2 at a predetermined timing.
  • the sensor 3 outputs physical information of the robot 2 as time-series data.
  • the robot control system 100 is assumed to have two cameras 4 .
  • the camera 4 captures an image of an object, a person, or the like located within the influence range 5 that may affect the motion of the robot 2 .
  • An image captured by the camera 4 may include monochrome luminance information, or may include luminance information of each color represented by RGB (Red, Green and Blue) or the like.
  • the range of influence 5 includes the motion range of the robot 2 . It is assumed that the influence range 5 is a range obtained by expanding the motion range of the robot 2 further outward.
  • the range of influence 5 may be set so that the robot 2 can be stopped before a person or the like moving from the outside to the inside of the motion range of the robot 2 enters the inside of the motion range of the robot 2 .
  • the range of influence 5 may be set, for example, as a range that extends a predetermined distance from the boundary of the motion range of the robot 2 to the outside.
  • the camera 4 may be installed so as to capture a bird's-eye view of the influence range 5 or the motion range of the robot 2 or a peripheral area thereof.
  • the number of cameras 4 is not limited to two, and may be one or three or more.
  • the robot control device 110 acquires the learned model 50 generated by the trained model generation device 20 . Based on the image captured by the camera 4 and the learned model 50, the robot control device 110 identifies the work object 8, the work start point 6, the work target point 7, etc., which exist in the space where the robot 2 works. to recognize In other words, the robot control device 110 acquires the learned model 50 generated for recognizing the work object 8 and the like based on the image captured by the camera 4 .
  • the robot controller 110 is also called a reasoning device.
  • the robot controller 110 may be configured with at least one processor to provide control and processing power to perform various functions.
  • Each component of the robot control device 110 may be configured including at least one processor.
  • a plurality of components among the components of the robot control device 110 may be realized by one processor.
  • the entire robot controller 110 may be implemented with one processor.
  • the processor may execute programs that implement various functions of the robot controller 110 .
  • a processor may be implemented as a single integrated circuit.
  • An integrated circuit is also called an IC (Integrated Circuit).
  • a processor may be implemented as a plurality of communicatively coupled integrated and discrete circuits. Processors may be implemented based on various other known technologies.
  • the robot control device 110 may include a storage unit.
  • the storage unit may include an electromagnetic storage medium such as a magnetic disk, or may include a memory such as a semiconductor memory or a magnetic memory.
  • the storage unit stores various information, programs executed by the robot control device 110, and the like.
  • the storage unit may be configured as a non-transitory readable medium.
  • the storage unit may function as a work memory for the robot control device 110 . At least part of the storage unit may be configured separately from the robot controller 110 .
  • the robot control device 110 acquires the learned model 50 in advance.
  • the robot control device 110 may store the trained model 50 in the storage unit.
  • the robot control device 110 obtains an image of the work object 8 from the camera 4 .
  • the robot control device 110 inputs the captured image of the work target 8 to the learned model 50 as input information.
  • the robot control device 110 acquires output information output from the trained model 50 in accordance with the input of input information.
  • the robot control device 110 recognizes the work object 8 based on the output information, and performs work such as gripping and moving the work object 8 .
  • the robot control system 100 can acquire the trained model 50 from the trained model generation system 1 and recognize the work object 8 using the trained model 50 .
  • the actor can be the administrator of the trained model generation device 20, the user who introduces the robot 2, or the robot control device 110.
  • the system utilized by the actor can be a trained model generation system 1 or a robot control system 100 that performs pick and place tasks. Use cases for each actor are illustrated below.
  • An administrator of the trained model generation device 20 generates a general-purpose model.
  • a user who introduces the robot 2 creates a dedicated model and registers parts to be recognized. Also, the user who introduces the robot 2 causes the robot 2 to perform a pick-and-place task.
  • the robot controller 110 acquires the trained model 50 .
  • ⁇ Usage pattern A> As a usage pattern A, the user who introduces the robot 2 does not include the user's own parts in the recognition target, or even if the user's own parts are included in the recognition target, a large classification such as a screw or a nut. It is assumed that the robot control system 100 is requested to recognize a part by .
  • the administrator of the trained model generating device 20 generates the trained model 50 as a general-purpose model.
  • the trained model generating device 20 generates the trained model 50 using the same information as the first information used to generate the preliminary model as the second information.
  • the second information or the third information is required as learning data for recognizing the user's unique parts.
  • the administrator of the trained model generating device 20 generates a trained model 50 as a dedicated model by learning the second information or the third information for recognizing the user's unique parts as learning data.
  • the trained model generating device 20 may generate the backbone as a general-purpose model or as a dedicated model.
  • the trained model 50 is generated by dividing it into two, the first trained model 51 and the second trained model 52 .
  • the trained model 50 is not limited to two and may be divided into three or more models. For example, if the trained model 50 has multiple layers, the trained model 50 may be divided into models for each layer.
  • the trained model generation system 1 may generate a trained model 50 corresponding to each divided model by performing learning on a model in which each divided model is connected to a plurality of preliminary models.
  • the trained model generating device 20 treats the middle model as a learning model and corresponds to the head and tail models among the preliminary models. training may be performed by connecting the part to the intermediate model.
  • the second pre-learned model (head) may be divided into two or more.
  • the second control unit 22 of the trained model generating device 20 may fix at least one of the two or more divided parts of the second pre-learned model (head).
  • the second control unit 22 connects the head connecting the fixed part of the second pre-learning model and the part corresponding to the other part of the second pre-learning model in the preliminary model, and the backbone of the preliminary model. Training may be performed on the model.
  • the first pre-learned model (backbone) may be divided into two or more.
  • the second control unit 22 may fix at least one of the two or more parts obtained by dividing the first pre-learning model (backbone).
  • the second control unit 22 connects the head of the preliminary model to the backbone that connects the fixed portion of the first pre-learning model and the portion of the preliminary model that corresponds to the other portion of the first pre-learning model. Training may be performed on the model.
  • the portion of the head may support lower-dimensional processing than other portions of the head that are not fixed.
  • the portion of the backbone may support lower dimensional processing than other portions of the backbone that are not fixed.
  • the portion of the head or backbone that corresponds to low-dimensional processing may be fixed.
  • the portion corresponding to the upstream of the CNN layer may be fixed.
  • the trained model 50 may include a branching model as illustrated in FIG.
  • a branching model means a model in which an output from a layer branches into two or more and is input to the next layer.
  • the first trained model 51 and the second trained model 52 may be connected in various combinations.
  • the model corresponding to the first part and the model corresponding to the second part may be connected in various combinations.
  • the branching model may be, for example, RPN (Region Proposal Network).
  • the second control unit 22 of the trained model generation device 20 generates a base model corresponding to the first part of the model and a pre-learning model (the second pre-learning model 520) corresponding to the second part of the model. are connected, learning is performed based on the second information.
  • the second control unit 22 not only changes the pre-learning model corresponding to the second part of the model, but also changes the base model corresponding to the first part of the model according to the change in the pre-learning model. you can In other words, the second control unit 22 may change various parameters in the base model set by preliminary learning when performing learning by connecting the base model and the target model.
  • the domain gap is a phenomenon that occurs because the learning environment and the reasoning environment are different. That is, even for the same subject, a domain gap may occur due to differences in the acquisition environment of the image used as learning data and the acquisition environment of the image used as inference data. As such, fine-tuning may need to be performed based on images acquired in the inference environment (the environment in which the robot is used) in order to reduce the effects of domain gaps. In other words, image-based retraining of the target model in the robotic environment may be required to reduce the effects of domain gaps. In other words, if the target model contains part of the base model, image-based learning in the robot environment reduces the effect of domain gaps when changing the parameters of the base model contained in the target model. can do.
  • Models in which Part 1 or Part 2 is divided into multiple parts At least one of the first part or the second part of the model may be divided into multiple parts.
  • trained preliminary models 401-40N are first trained preliminary models 411-41N, second trained preliminary models 421-42N, and third trained preliminary models 431-43N, respectively. 43N.
  • the second trained preliminary models 421-42N correspond to the first part of the model.
  • the first trained preliminary models 411-41N and the third trained preliminary models 431-43N correspond to the second part of the models.
  • the second control unit 22 of the trained model generation device 20 sequentially transfers each of the second trained preliminary models 421 to 42N corresponding to the first part of the model to generate the first model corresponding to the second part of the model.
  • Models connected between pre-learning model 510 and third pre-learning model 530 may be learned.
  • the part connected to the input side (left side) is also called an encoder.
  • the part connected to the output side (right side) is also called a decoder.
  • the second part be divided into multiple parts as illustrated in FIG. 12, but also the first part can be divided into multiple parts.
  • the first part is divided into multiple parts, the second part connected between the multiple first parts may be learned.
  • the head of the trained model 50 can be generated by learning a model in which the backbone of the preliminary model is connected to the learning head.
  • the backbone of the trained model 50 may be generated by learning a model in which the head of the preliminary model is connected to the learning backbone. Note that the trained model 50 may be used only for the head or backbone.
  • the backbone may be generated as follows, for example. That is, the first control unit 12 of the preliminary model generation device 10 generates or acquires a plurality of pre-learning preliminary models 301 to 30N in advance, as shown in FIG. In FIG. 5, it is assumed that the fully-connected layer configuration or the parameter size of each layer in the second pre-learning preliminary model 321 and the fully-connected layer configuration or the parameter size of each layer in the second pre-learning preliminary model 32N are different from each other. . In other words, it is assumed that the fully-connected layer configuration or the parameter size of each layer is different in each of the second pre-learning preliminary models 321 to 32N.
  • the first control unit 12 learns each pre-learning preliminary model 30 using the first information that is the same as or related to the input information input to the trained model 50 as learning data.
  • the first control unit 12 updates each pre-learning preliminary model 30 by learning to generate a plurality of trained preliminary models 401 to 40N.
  • the full connection or CNN layer configuration or the parameter size or filter size of each layer of each trained preliminary model 40 is the same as the full connection or CNN layer configuration or the parameter size or filter size of each layer of each pre-learning preliminary model 30 is.
  • the second control unit 22 of the trained model generation device 20 acquires the trained preliminary model 40 from the preliminary model generation device 10 as a preliminary model.
  • the second control unit 22 uses the third information, which is the same as or related to the input information input to the trained model 50, as learning data, and executes learning for the model in which the head of each preliminary model is connected to the learning backbone. do.
  • Each preliminary model head corresponds to the first part of the base model.
  • the training backbone corresponds to the second part of the target model.
  • the third information may be the same as or different from the first information or the second information.
  • the third information may be composed of a set of multiple learning images.
  • the second control unit 22 may learn the same set of third information as learning data for each model in which the head of each preliminary model and the learning backbone are connected, or may learn a different set of third information. may be learned as learning data. In addition, the second control unit 22 further divides one set of third information into smaller small groups, and learns different small groups as learning data each time the head connected to the learning backbone is changed. you can The second control unit 22 may learn the same small group as learning data when the head connected to the learning backbone is changed. That is, the second control unit 22 may perform learning of a model in which the head of each preliminary model is connected to the learning backbone based on at least one set of third information.
  • the information amount of the third information used for learning the trained model 50 may be less than or equal to the information amount of the first information used for learning the preliminary model. Note that the amount of information refers to, for example, the number of learning images included in the second information.
  • the second control unit 22 transfers the head of each preliminary model (second trained preliminary models 421 to 42N) to the first pre-learning model 510 or the first learning model 510. Learning is sequentially executed for the models connected to the models 511 to 51(N-1).
  • the 2nd control part 22 may perform learning by the following procedures.
  • the second control unit 22 performs learning on a model connecting the second trained preliminary model 421 and the first pre-learning model 510 , and updates the first pre-learning model 510 to the first learning model 511 .
  • the second control unit 22 performs learning on a model connecting the second trained preliminary model 422 and the first learning model 511 updated in the previous step, and converts the first learning model 511 to the first learning model. Update to 512.
  • the second control unit 22 performs learning on a model connecting the second trained preliminary model 42N and the first learning model 51 (N ⁇ 1) updated in the previous step, and performs learning on the first learning model 51 ( N ⁇ 1) is updated to the first trained model 51N.
  • the second control unit 22 generates the first trained model 51N by executing the procedure described above.
  • the second control unit 22 applies the first trained model 51N as the first trained model 51 (the backbone of the trained model 50).
  • the first trained model 51N generated by learning by connecting to the first part of the base model corresponds to the second part of the target model.
  • both or one of the backbone and the head can be generated based on learning in the model to which the corresponding head or backbone is connected.
  • the relationship between the first part and the second part of the model having the backbone generated based on the learning with the multiple heads connected is generated based on the learning with the multiple backbones connected in the embodiment described in the first half of this disclosure. It is inversely related to parts 1 and 2 of the model with a head that has a flat head. Specifically, in generating a backbone based on learning by connecting multiple heads, it can be read that learning is performed by connecting the first part of each of the multiple preliminary models to the second part of the target model. can be done.
  • the second part of each of the plurality of preliminary models is connected to the first part of the target model. It can also be read as learning is executed as That is, the first part and the second part of the model may be interchanged as appropriate.
  • both or one of the first part and the second part of the model can be generated based on learning in the model connecting the corresponding second part or the first part.
  • the trained model generation system 1 may set the loss function so that the output when input information is input to the generated trained model 50 approaches the output when learning data is input.
  • cross-entropy can be used as the loss function.
  • Cross-entropy is calculated as a value representing the relationship between two probability distributions. Specifically, in this embodiment, the cross-entropy is calculated as a value representing the relationship between the input information and the backbone or head.
  • the trained model generation system 1 learns so that the value of the loss function becomes small.
  • the output corresponding to the input of the input information can approach the output corresponding to the input of the learning data.
  • Discrimination Loss is a loss function used to learn the authenticity of a generated image by labeling it with a numerical value between 1, which represents complete truth, and 0, which represents complete falsehood. .
  • the embodiments of the trained model generation system 1 and the robot control system 100 have been described above. It can also be embodied as a medium (for example, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a CD-RW, a magnetic tape, a hard disk, or a memory card).
  • a medium for example, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a CD-RW, a magnetic tape, a hard disk, or a memory card.
  • the implementation form of the program is not limited to an application program such as an object code compiled by a compiler or a program code executed by an interpreter. good.
  • the program may or may not be configured so that all processing is performed only in the CPU on the control board.
  • the program may be configured to be partially or wholly executed by another processing unit mounted on an expansion board or expansion unit added to the board as required.
  • Embodiments according to the present disclosure are not limited to any specific configuration of the embodiments described above. Embodiments of the present disclosure extend to any novel feature or combination thereof described in the present disclosure or any novel method or process step or combination thereof described. be able to.
  • Descriptions such as “first” and “second” in this disclosure are identifiers for distinguishing the configurations. Configurations that are differentiated in descriptions such as “first” and “second” in this disclosure may interchange the numbers in that configuration. For example, the first information can replace the identifiers “first” and “second” with the second information. The exchange of identifiers is done simultaneously. The configurations are still distinct after the exchange of identifiers. Identifiers may be deleted. Configurations from which identifiers have been deleted are distinguished by codes. The description of identifiers such as “first” and “second” in this disclosure should not be used as a basis for interpreting the order of the configuration or the existence of lower numbered identifiers.
  • Preliminary model generation device (12: control unit) 20 Trained model generation device (22: control unit) 30 (301-30N) pre-learning preliminary model (31 (311-31N): first pre-learning preliminary model, 32 (321-32N): second pre-learning preliminary model) 40 (401-40N) trained preliminary models (41 (411-41N): first trained preliminary model, 42 (421-42N): second trained preliminary model, 431-43N: third trained preliminary model) 50 trained model (51: first trained model, 52: second trained model, 510: first pre-learned model, 520: second pre-learned model) 100 robot control system (2: robot, 2A: arm, 2B: end effector, 3: sensor, 4: camera, 5: range of influence of robot, 6: work start table, 7: work target table, 8: work object , 110: robot controller (reasoning device)

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Image Analysis (AREA)
PCT/JP2022/034632 2021-09-17 2022-09-15 学習済みモデル生成方法、推論装置、及び学習済みモデル生成装置 Ceased WO2023042895A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2023548510A JP7693010B2 (ja) 2021-09-17 2022-09-15 学習済みモデル生成方法、及び学習済みモデル生成装置
US18/692,761 US20240289695A1 (en) 2021-09-17 2022-09-15 Trained model generation method, inference apparatus, and trained model generation apparatus
CN202280062804.7A CN118119955A (zh) 2021-09-17 2022-09-15 学习完毕模型生成方法、推论装置以及学习完毕模型生成装置

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021152586 2021-09-17
JP2021-152586 2021-09-17

Publications (1)

Publication Number Publication Date
WO2023042895A1 true WO2023042895A1 (ja) 2023-03-23

Family

ID=85602963

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/034632 Ceased WO2023042895A1 (ja) 2021-09-17 2022-09-15 学習済みモデル生成方法、推論装置、及び学習済みモデル生成装置

Country Status (4)

Country Link
US (1) US20240289695A1 (https=)
JP (1) JP7693010B2 (https=)
CN (1) CN118119955A (https=)
WO (1) WO2023042895A1 (https=)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7317246B1 (ja) * 2022-08-02 2023-07-28 三菱電機株式会社 推論装置、推論方法及び推論プログラム

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11183246A (ja) * 1997-12-25 1999-07-09 Toshiba Corp 周期的振動現象監視診断方法
JP2020522764A (ja) * 2018-05-10 2020-07-30 ベイジン センスタイム テクノロジー デベロップメント カンパニー， リミテッド 生体検知方法および装置、システム、電子機器、記憶媒体
US20210097428A1 (en) * 2019-09-30 2021-04-01 International Business Machines Corporation Scalable and dynamic transfer learning mechanism
CN112613375A (zh) * 2020-12-16 2021-04-06 中国人寿财产保险股份有限公司 一种轮胎受损检测识别方法和设备
CN113159049A (zh) * 2021-04-23 2021-07-23 上海芯翌智能科技有限公司 弱监督语义分割模型的训练方法及装置、存储介质、终端

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11183246A (ja) * 1997-12-25 1999-07-09 Toshiba Corp 周期的振動現象監視診断方法
JP2020522764A (ja) * 2018-05-10 2020-07-30 ベイジン センスタイム テクノロジー デベロップメント カンパニー， リミテッド 生体検知方法および装置、システム、電子機器、記憶媒体
US20210097428A1 (en) * 2019-09-30 2021-04-01 International Business Machines Corporation Scalable and dynamic transfer learning mechanism
CN112613375A (zh) * 2020-12-16 2021-04-06 中国人寿财产保险股份有限公司 一种轮胎受损检测识别方法和设备
CN113159049A (zh) * 2021-04-23 2021-07-23 上海芯翌智能科技有限公司 弱监督语义分割模型的训练方法及装置、存储介质、终端

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
TAMMINA SRIKANTH: "Transfer learning using VGG-16 with Deep Convolutional Neural Network for Classifying Images", INTERNATIONAL JOURNAL OF SCIENTIFIC AND RESEARCH PUBLICATIONS (IJSRP), vol. 9, no. 10, 1 October 2019 (2019-10-01), pages p9420 - 3153, XP093049085, DOI: 10.29322/IJSRP.9.10.2019.p9420 *

Also Published As

Publication number Publication date
US20240289695A1 (en) 2024-08-29
CN118119955A (zh) 2024-05-31
JP7693010B2 (ja) 2025-06-16
JPWO2023042895A1 (https=) 2023-03-23

Similar Documents

Publication Publication Date Title
JP7693648B2 (ja) 視角エンベディングに基づいた自律タスク実行
EP3486041B1 (en) Gripping system, learning device, and gripping method
CN108247637B (zh) 一种工业机器人手臂视觉防撞操控方法
CN111881772A (zh) 基于深度强化学习的多机械臂协同装配方法和系统
CN111055279A (zh) 基于触觉与视觉结合的多模态物体抓取方法与系统
WO2020057440A1 (zh) 一种装配方法、装配装置及装配设备
CN109079777B (zh) 一种机械臂手眼协调作业系统
US20230234232A1 (en) Autonomous control system, autonomous control method, and storage medium
CN114131603B (zh) 基于感知增强和场景迁移的深度强化学习机器人抓取方法
CN112257293A (zh) 一种基于ros的非标准物体抓取方法及装置
CN114723831B (zh) 基于启发式的机器人柔性织物展平方法及系统
CN116079723A (zh) 基于视觉的机器人抓取和装配技能深度强化学习方法
JP7693010B2 (ja) 学習済みモデル生成方法、及び学習済みモデル生成装置
CN113762159B (zh) 一种基于有向箭头模型的目标抓取检测方法及系统
Ichiwara et al. Modality attention for prediction-based robot motion generation: Improving interpretability and robustness of using multi-modality
CN111496794B (zh) 一种基于仿真工业机器人的运动学自抓取学习方法和系统
Hossain et al. Object recognition and robot grasping: A deep learning based approach
Tokuda et al. CNN-based visual servoing for pose control of soft fabric parts
JP2023099084A (ja) 学習済みモデル生成装置、学習済みモデル生成方法、及び認識装置
CN114789454A (zh) 基于lstm和逆运动学的机器人数字孪生轨迹补全方法
CN114029941A (zh) 一种机器人抓取方法、装置、电子设备及计算机介质
CN114399789B (zh) 一种基于静态手势识别的机械臂远程控制方法
JP7271810B2 (ja) 学習済みモデル生成装置、学習済みモデル生成方法、及び認識装置
JP7470062B2 (ja) 情報処理装置、および、学習認識システム
Andersen et al. Using a flexible skill-based approach to recognize objects in industrial scenarios

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22870042

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2023548510

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 202280062804.7

Country of ref document: CN

Ref document number: 18692761

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22870042

Country of ref document: EP

Kind code of ref document: A1