WO2025041240A1 - Dispositif de commande, procédé de commande et programme - Google Patents

Dispositif de commande, procédé de commande et programme Download PDF

Info

Publication number
WO2025041240A1
WO2025041240A1 PCT/JP2023/030055 JP2023030055W WO2025041240A1 WO 2025041240 A1 WO2025041240 A1 WO 2025041240A1 JP 2023030055 W JP2023030055 W JP 2023030055W WO 2025041240 A1 WO2025041240 A1 WO 2025041240A1
Authority
WO
WIPO (PCT)
Prior art keywords
learning
transfer
model
models
source
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/JP2023/030055
Other languages
English (en)
Japanese (ja)
Inventor
崇洋 秦
翠 児玉
正人 神谷
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NTT Inc
Original Assignee
Nippon Telegraph and Telephone Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nippon Telegraph and Telephone Corp filed Critical Nippon Telegraph and Telephone Corp
Priority to PCT/JP2023/030055 priority Critical patent/WO2025041240A1/fr
Priority to JP2025541196A priority patent/JPWO2025041240A1/ja
Publication of WO2025041240A1 publication Critical patent/WO2025041240A1/fr
Anticipated expiration legal-status Critical
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/092—Reinforcement learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/096—Transfer learning

Definitions

  • the disclosed technology relates to a control device, a control method, and a program.
  • Reinforcement learning is a technology that makes it possible to select optimal control (policy).
  • Patent Document 1 there is a technology that optimizes behavior for controlling air conditioning in a target space (see Patent Document 1). With this technology, the environment of the space to be controlled is reproduced, and reinforcement learning is used in behavior exploration to optimize the behavior. As noted in Patent Document 1, reinforcement learning is known to be effective in optimizing the control of air conditioning equipment and the like.
  • the disclosed technology has been developed in consideration of the above points, and aims to provide a control device, control method, and program that can reduce the computational costs associated with learning using transfer learning.
  • a first aspect of the present disclosure is a control device, comprising: a transfer learning unit that performs transfer learning in a predetermined manner to obtain multiple models using information on one or more source models that are models used for predetermined control and that have been previously trained at the source, in transfer learning using reinforcement learning, and creates multiple models; and a transfer model selection unit that selects a transfer model to be output to the destination from among the multiple models created, based on at least one index of the reward, loss function, and learning step.
  • the second aspect of the present disclosure is a control method, in which a computer executes a process of performing transfer learning using reinforcement learning, using information on one or more source models that are used for a predetermined control and that have been previously trained at the source, performing transfer learning in a predetermined manner to obtain multiple models, creating multiple models, and selecting a transfer model to be output to the destination from among the multiple models created based on at least one index of the reward, loss function, and learning step.
  • the disclosed technology uses transfer learning to reduce the computational costs associated with learning.
  • FIG. 1 is a schematic diagram showing an example of a system for calculating a scenario by optimal control using reinforcement learning.
  • FIG. 2 is a block diagram showing the configuration of the control system.
  • FIG. 3 is a block diagram showing the hardware configuration of the control device.
  • FIG. 4 is a diagram for explaining the relationship between the control device and the system to be controlled.
  • FIG. 5 is a diagram showing an aspect of a system to be controlled.
  • FIG. 6 is a flowchart showing the flow of the control process.
  • FIG. 7 is a schematic diagram showing an image of the method of this embodiment.
  • FIG. 1 is a schematic diagram showing an example of a system that performs scenario calculation by optimal control using reinforcement learning.
  • processing is performed in an environment reproduction unit and a scenario calculation unit, and a model used in the scenario calculation unit is learned using reinforcement learning.
  • environment reproduction unit people flow data is input and an environment is reproduced using an environment reproduction simulator.
  • the people flow data may be, for example, a people flow vector predicted using a prediction model from past people flow data.
  • the scenario calculation unit Since energy saving information and comfort information (temperature, humidity, and the amount of movement of people flow, etc.) can be obtained from the reproduced environment, these various information obtained from the reproduced environment are output to the scenario calculation unit and used as reward information.
  • a model using DNN Deep Neural Network
  • a scenario that optimizes behavior control is calculated using the model.
  • the temperature for each time period for each of multiple cold air outlets can be calculated as an air conditioning system for a building. Examples include “cold air outlet 1, time zone 9:00, temperature 22.5°C/time zone 9:30, temperature 22.0°C --, "cold air outlet 2, time zone 9:00, temperature 24.1°C/time zone 9:30, temperature 24.5°C!, etc.
  • the above example can refer to the method of action calculation by an optimal control device in Patent Document 1.
  • the method in Patent Document 1 it is possible to acquire environmental data of the real space, reproduce the environment of the target space, obtain a model that calculates optimal control measures based on the reproduced environment through machine learning (reinforcement learning), and select optimal actions.
  • machine learning reinforcementment learning
  • optimal actions when applied to an actual building, learning and inference must be performed with a calculation cost lower than the energy that can be reduced by the above method, and depending on the measured data and environment, this condition may not be met.
  • a method of transfer learning in which a base model is selected from a model learned in a finite time. For example, a model is learned for each of multiple floors in the same building, and a model learned on one floor is transferred to multiple floors.
  • reinforcement learning of an air conditioning control agent is used as an example of reinforcement learning, but this embodiment can apply the transfer model (control model) method using transfer learning using reinforcement learning to the control of various facilities and equipment in environments that can be reproduced by an environmental reproduction model, not limited to air conditioning equipment.
  • Transfer learning using reinforcement learning refers to bringing a model into an environment other than the original environment (an environmental simulator in this embodiment) to speed up learning and improve accuracy.
  • a method of deep reinforcement learning is used among the reinforcement learning methods, and hereinafter, even when simply referring to reinforcement learning, deep reinforcement learning is used.
  • the trained model at the transfer source is selected as follows: (1) for a reward function in which the reward is regarded as a function of the learning step, the center of the maximum peak in multiple ranges in which the moving average curve of the reward function is upwardly convex (projecting), (2) the center of the minimum peak in multiple ranges in which the moving average curve of the loss function is downwardly convex, and (3) the final model in a limited learning period.
  • models (1), (2), and (3) are given as initial values, and (a) all-layer learning or (b) learning of only the output layer is performed as the learning means.
  • the trained model and learning means at the transfer source are selected based on the criteria that the moving average curve of the reward function is larger and the moving average of the loss value is smaller.
  • an environmental simulation is performed as necessary, and if none of the models meets the building criteria, additional learning is performed to advance the agent's learning.
  • FIG. 2 is a block diagram showing the configuration of the control system.
  • the control system 100 is connected to a control device 110 and a plurality of systems 102 to be controlled.
  • the systems 102 are the systems to be controlled, and become a source system (102A) and a destination system (102B) depending on the processing mode of the control device 110.
  • the relationship between the systems 102 and the control device 110 will be described later.
  • the systems 102 and the control device 110 will be described as separate systems, but the control device 110 may be a device that includes the systems 102.
  • FIG. 3 is a block diagram showing the hardware configuration of the control device 110.
  • the control device 110 has a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage 14, an input unit 15, a display unit 16, and a communication interface (I/F) 17.
  • CPU Central Processing Unit
  • ROM Read Only Memory
  • RAM Random Access Memory
  • storage 14 an input unit
  • I/F communication interface
  • the CPU 11 is a central processing unit that executes various programs and controls each part. That is, the CPU 11 reads the programs from the ROM 12 or storage 14, and executes the programs using the RAM 13 as a working area. The CPU 11 controls each of the above components and performs various calculation processes according to the programs stored in the ROM 12 or storage 14. In this embodiment, the control programs are stored in the ROM 12 or storage 14.
  • ROM 12 stores various programs and data.
  • RAM 13 temporarily stores programs or data as a working area.
  • Storage 14 is composed of a storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive), and stores various programs including an operating system, and various data.
  • HDD Hard Disk Drive
  • SSD Solid State Drive
  • the input unit 15 includes a pointing device such as a mouse and a keyboard, and is used to perform various inputs.
  • the display unit 16 is, for example, a liquid crystal display, and displays various information.
  • the display unit 16 may also function as the input unit 15 by adopting a touch panel system.
  • the communication interface 17 is an interface for communicating with other devices such as terminals.
  • a wired communication standard such as Ethernet (registered trademark) or FDDI
  • a wireless communication standard such as 4G, 5G, or Wi-Fi (registered trademark) is used.
  • Each functional configuration of the control device 110 is realized by the CPU 11 reading out a control program stored in the ROM 12 or storage 14, expanding it in the RAM 13, and executing it.
  • control device 110 includes a memory unit 112, a learning model selection unit 120, a transfer learning unit 122, and a transfer model selection unit 124. Each unit of the control device 110 will be described below with reference to its relationship with the system 102.
  • FIG. 4 is a diagram for explaining the relationship between the control device 110 and the system 102 to be controlled.
  • the system from which the transfer model is learned is the transfer source system 102A
  • the system that performs control using the transfer model that has been transferred and learned is the transfer destination system 102B.
  • the transfer source system 102A is a system that corresponds to the representative floor of the facility.
  • the transfer destination system 102B is a system that corresponds to each floor in relation to the representative floor.
  • the system 102 in FIG. 4 is a basic configuration for explaining the parts necessary for model transfer by the processing of the control device 110.
  • the system 102 is explained as an aspect including a control learning unit 1020 and a scenario calculation unit 1022.
  • the control learning unit 1020 of the transfer source system 102A performs reinforcement learning of the air conditioning control agent for a set time.
  • the environmental data and BEMS data in the environmental DB are used for the reinforcement learning.
  • the environmental prediction result is output to the control learning unit 1020.
  • the control model is stored in the control model DB by learning of the control learning unit 1020.
  • the scenario calculation unit 1022 calculates a scenario using the control model in the control model DB.
  • the control model DB is updated with the transfer model by the control device 110.
  • the control model is a model used to control the equipment in the environment, and is a model used to calculate the control scenario in the scenario calculation unit. If no transfer occurs, the control model is learned in the transfer destination system 102B, but if a transfer occurs, the control model is learned instead using a transfer model obtained by transfer learning.
  • the source system 102A is assumed to be performing normal deep reinforcement learning. Therefore, as a prerequisite, deep reinforcement learning is performed in the control learning section of the source system 102A, and the learned control model is stored in the control model DB.
  • the control models stored in the control model DB include (1) a unique control model when learning has converged, and (1) multiple model candidates when learning has not converged. Therefore, (1) if learning has converged, the learned control model is input to the control device 110. (2) If learning has not converged, all models in the middle of learning (model candidates) are input to the control device 110.
  • the learning model selection unit 120 acquires the source control model (including models in the middle of learning), the score of the sum of the rewards and penalties of each of the control models (hereinafter also simply referred to as the score), and the loss value of each of the control models (values output by a loss function) from the control model of the source system 102A as information on the source model (input of a1 in Figure 4). For example, the learning model selection unit 120 selects a model to be used as a learning target for transfer from among the transfer models input from the source system 102A using a predetermined index. Note that the loss values arranged in the order of the learning steps are used as an index for the loss function.
  • the learning model selection unit 120 (1) if the learning has converged, only one control model that has completed learning is input, and the input control model is output as is. Therefore, in this case, the processing of the transfer model selection unit 124 may be omitted. (2) If the learning has not converged, all models in the middle of learning are input together with the reward and loss value, and an arbitrary model is selected and output to the transfer learning unit 122.
  • the indicator of the model selected by the learning model selection unit 120 can be, for example, "the model with the maximum reward (the model with the maximum score),” “the model with the minimum loss value (the value output by the loss function),” and "the model with the most learning steps in the aborted range.” Any number of models may be selected, which is one or more. As described above, when the learning of the transfer source model is aborted midway, the learning model selection unit 120 can select the transfer source model from among the multiple transfer source models in the middle of learning, based on at least one indicator of the reward, loss function, and learning steps.
  • the transfer learning unit 122 takes over the learning of the control model performed by the control learning unit of the transfer destination system 102B.
  • the transfer learning unit 122 acquires in advance environmental data (including BEMS data) from the environmental data DB and an environmental simulator from the transfer destination system 102B and stores them in the storage unit 112.
  • the transfer learning unit 122 uses these to carry out additional learning using the trained model of the transfer source, or a candidate model in the middle of learning, as a base model.
  • the transfer learning unit 122 performs multiple types of reinforcement learning with the weights of the selected source model transferred (copied to the destination model) to create multiple models. Examples of multiple transfer methods are given below. For example, there is a method (first method) in which "the weights of the source model are set as initial values, all weights are updated, and reinforcement learning is performed.” In addition, for example, reinforcement learning is performed using a method (second method) in which "the weights of the source model are frozen, and the transferred weights are not updated.” All of these methods are applied to each of the one or more selected source models.
  • first method the weights of the source model are set as initial values, all weights are updated, and reinforcement learning is performed.
  • reinforcement learning is performed using a method (second method) in which "the weights of the source model are frozen, and the transferred weights are not updated.” All of these methods are applied to each of the one or more selected source models.
  • the number of models (transfer model candidates) created is "the number of models selected (by the learning model selection unit 120) x the number of methods applied (by the transfer learning unit 122)."
  • the transfer learning unit 122 applies the first method and the second method to the source model to create multiple models.
  • the transfer model selection unit 124 selects a transfer model based on the reward and loss value of the transfer model created by the transfer learning unit 122. For example, it selects one "model with the maximum reward in transfer learning”. It may also select one "model with the minimum loss value in transfer learning”. It may also select one "model with the most learning steps in transfer learning”. In other words, the transfer model selection unit 124 selects a transfer model based on any one of the indicators of reward, loss function, and learning steps, and outputs it to the transfer destination system 102B. In the transfer destination system 102B, the output transfer model is updated as the control model in the control model DB.
  • FIG. 6 is a flowchart showing the flow of the control process.
  • the CPU 11 reads out a control program from the ROM 12 or storage 14, expands it in the RAM 13, and executes it, thereby performing the control process.
  • the process is performed via the system 102 (102A, 102B).
  • step S100 the source system 102A (control learning unit 1020) performs reinforcement learning of the air-conditioning control agent for a set period of time, for example, and stores the learned control model in the control DB.
  • the control learning unit 1020 performs reinforcement learning of the air-conditioning control agent using the source environmental simulator for a limited period of time, such as four hours.
  • the score of the sum of the rewards and penalties acquired is stored together with the learning step.
  • step S102 the source system 102A outputs one or more source models stored in the control DB and information about the source models to the control device 110.
  • step S104 the CPU 11, as the learning model selection unit 120, selects a model to be used as the transfer learning target from among the input transfer models using a predetermined index.
  • a model to be used as the transfer learning target is given below.
  • the scores of each learning step of the reinforcement learning performed in the control learning unit 1020 are tallied, and the learning steps are regarded as the time axis to calculate a moving average of the scores, for example, every 1000 learning steps.
  • the model closest to the maximum step around the maximum value of the calculated moving average of the scores is selected as a candidate for the air conditioning control agent.
  • the model of the last learning step within the limited time is also selected as a candidate for the air conditioning control agent. Note that a loss value may be used instead of a reward for the score.
  • step S106 the CPU 11, as the transfer learning unit 122, performs multiple types of reinforcement learning with the weights of the source model transferred, to create multiple models.
  • the reinforcement learning in this processing step creates, for example, two new air conditioning control agents with the selected source model as the initial value, and performs multiple types of reinforcement learning.
  • the first method freezes everything except the output layer so that no updates are made through learning.
  • the second method does not freeze any layers, and all layers are updated through learning. In this processing, reinforcement learning is performed for each method.
  • step S108 the CPU 11, as the transfer model selection unit 124, selects a transfer model based on the reward and loss values of the multiple models created.
  • the scores of each learning step in which multiple models (transfer model candidates) are learned are tallied, and the learning steps are treated as the time axis to calculate a moving average of the scores, for example, every 1000 learning steps. Then, the model closest to the maximum step around the maximum value of the moving average of the scores is selected as the transfer model.
  • step S110 the CPU 11, functioning as the transfer model selection unit 124, outputs the selected transfer model to the transfer destination model.
  • step S112 the transfer destination system 102B (scenario calculation unit 1022) uses the input transfer model (control model) to calculate a control scenario for the floor corresponding to the system.
  • control system 100 of this embodiment can reduce the computational costs associated with learning by using transfer learning.
  • FIG. 7 is a schematic diagram showing an image of the method of this embodiment.
  • the best models are selected from among models that have stopped learning in a limited time in any one or a small number of spaces. For example, it is expected that there will be multiple spaces (floors) with similar environments in an office building. Taking into account divisions, there can be 30 or more source systems, such as 30 floors in a 30-story office building. Therefore, when a model is transferred, the learning time can be reduced on the destination floor.
  • the reduction rate be ⁇ , and assume that learning can be done in ⁇ T ⁇ T for the learning time T for one floor. In this case, whereas 30T was required to train 30 floors, it can be done in T + 29 ⁇ T ⁇ 30T. The larger ⁇ is, the greater the reduction, and the lower the average learning cost per floor can be.
  • the index in the learning model selection unit 120 is exemplified as the score of the sum of rewards and penalties, but is not limited to this.
  • the following conditions may be used instead of the score as an index, and the maximum or minimum value of the corresponding condition may be used as the selection criterion in the learning model selection unit 120.
  • the following combinations may also be used as indexes.
  • the transfer model selection unit 124 may also use the following conditions and the following combinations as indexes.
  • the control process executed by the CPU after reading the software may be executed by various processors other than the CPU.
  • processors in this case include PLDs (Programmable Logic Devices) whose circuit configuration can be changed after manufacture, such as FPGAs (Field-Programmable Gate Arrays), GPUs (Graphics Processing Units), and dedicated electrical circuits that are processors having a circuit configuration designed specifically to execute specific processes, such as ASICs (Application Specific Integrated Circuits).
  • the control process may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (for example, multiple FPGAs, a combination of a CPU and an FPGA, etc.). More specifically, the hardware structure of these various processors is an electrical circuit that combines circuit elements such as semiconductor devices.
  • the program is pre-stored (installed) in the storage 14, but this is not limiting.
  • the program may be provided in a form stored in a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory.
  • the program may also be downloaded from an external device via a network.
  • Memory at least one processor coupled to the memory; Including, The processor, In transfer learning using reinforcement learning, a plurality of models are created by performing transfer learning using a predetermined method for obtaining a plurality of models using information on one or a plurality of transfer source models that are used for a predetermined control and have been previously learned at the transfer source; Selecting a transfer model to be output to a transfer destination from among the multiple models created based on at least one index of a reward, a loss function, and a learning step;
  • the control device is configured as follows.
  • a non-transitory storage medium storing a program executable by a computer to execute a control process
  • a plurality of models are created by performing transfer learning in a predetermined manner for obtaining a plurality of models using information on one or a plurality of transfer source models which are models used for a predetermined control and have been previously learned at a transfer source; Selecting a transfer model to be output to a transfer destination from among the multiple models created based on at least one index of a reward, a loss function, and a learning step;
  • Non-transitory storage media
  • Control system 102 System 102A Transfer source system 102B Transfer destination system 110 Control device 112 Storage unit 120 Learning model selection unit 122 Transfer learning unit 124 Transfer model selection unit 1020 Control learning unit 1022 Scenario calculation unit

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Feedback Control In General (AREA)

Abstract

Un dispositif de commande comprend : une unité d'apprentissage par transfert qui, dans un apprentissage par transfert employant un apprentissage par renforcement, utilise des informations concernant un ou plusieurs modèles de source de transfert qui sont utilisés pour une commande prescrite et qui ont été pré-entraînés à une source de transfert pour créer une pluralité de modèles en effectuant un apprentissage par transfert à l'aide d'un procédé prescrit pour obtenir une pluralité de modèles; et une unité de sélection de modèle de transfert qui sélectionne un modèle de transfert à produire en sortie à une destination de transfert, parmi la pluralité créée de modèles sur la base d'un indicateur d'une récompense, d'une fonction de perte et/ou d'une étape d'apprentissage.
PCT/JP2023/030055 2023-08-21 2023-08-21 Dispositif de commande, procédé de commande et programme Pending WO2025041240A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/JP2023/030055 WO2025041240A1 (fr) 2023-08-21 2023-08-21 Dispositif de commande, procédé de commande et programme
JP2025541196A JPWO2025041240A1 (fr) 2023-08-21 2023-08-21

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2023/030055 WO2025041240A1 (fr) 2023-08-21 2023-08-21 Dispositif de commande, procédé de commande et programme

Publications (1)

Publication Number Publication Date
WO2025041240A1 true WO2025041240A1 (fr) 2025-02-27

Family

ID=94731862

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/030055 Pending WO2025041240A1 (fr) 2023-08-21 2023-08-21 Dispositif de commande, procédé de commande et programme

Country Status (2)

Country Link
JP (1) JPWO2025041240A1 (fr)
WO (1) WO2025041240A1 (fr)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021089446A (ja) * 2019-03-13 2021-06-10 ダイキン工業株式会社 モデルの選定方法および深層強化学習方法
JP2021117625A (ja) * 2020-01-24 2021-08-10 株式会社奥村組 土の粒度分布の推定方法
WO2021157067A1 (fr) * 2020-02-07 2021-08-12 株式会社日立ハイテク Dispositif et procédé de traitement d'apprentissage
JP2021182329A (ja) * 2020-05-20 2021-11-25 株式会社日立製作所 学習モデル選択方法
JP2022153142A (ja) * 2021-03-29 2022-10-12 ソニーグループ株式会社 情報処理システム、生体試料処理装置及びプログラム
JP2022174740A (ja) * 2021-05-11 2022-11-24 株式会社島津製作所 学習支援システムおよび学習支援方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021089446A (ja) * 2019-03-13 2021-06-10 ダイキン工業株式会社 モデルの選定方法および深層強化学習方法
JP2021117625A (ja) * 2020-01-24 2021-08-10 株式会社奥村組 土の粒度分布の推定方法
WO2021157067A1 (fr) * 2020-02-07 2021-08-12 株式会社日立ハイテク Dispositif et procédé de traitement d'apprentissage
JP2021182329A (ja) * 2020-05-20 2021-11-25 株式会社日立製作所 学習モデル選択方法
JP2022153142A (ja) * 2021-03-29 2022-10-12 ソニーグループ株式会社 情報処理システム、生体試料処理装置及びプログラム
JP2022174740A (ja) * 2021-05-11 2022-11-24 株式会社島津製作所 学習支援システムおよび学習支援方法

Also Published As

Publication number Publication date
JPWO2025041240A1 (fr) 2025-02-27

Similar Documents

Publication Publication Date Title
CN114580688A (zh) 水冷系统的控制模型优化方法、电子设备和存储介质
JP4392604B2 (ja) 空調負荷予測方法
KR20230052450A (ko) 딥 러닝 기반의 반도체 소자의 특성 예측 방법 및 이를 수행하는 컴퓨팅 장치
CN113821903A (zh) 温度控制方法和设备、模块化数据中心及存储介质
JPWO2020065808A1 (ja) 情報処理装置及びシステム、並びに、モデル適応方法及びプログラム
CN111090899A (zh) 一种用于城市建筑空间布局设计方法
KR20190043419A (ko) 조기 중단에 기반한 심층 신경망의 연산 제어 방법 및 시스템
WO2018143019A1 (fr) Dispositif de traitement d'informations, procédé de traitement d'informations et support d'enregistrement de programme
CN118313484B (zh) 一种自动驾驶中基于模型的离线到在线强化学习方法
CN118378094B (zh) 芯片布局模型训练及运用方法、装置、电子设备及存储介质
CN118940624B (zh) 模型训练方法、装置及可读存储介质
JP2021089446A (ja) モデルの選定方法および深層強化学習方法
KR20230027588A (ko) 딥 러닝 모델의 생성 방법 및 이를 수행하는 컴퓨팅 장치 장치
JP2025009891A (ja) デジタルツインを使用してシステムを制御するためのデバイスおよび方法
CN114692348A (zh) 基于多保真深度学习代理模型的组件布局温度场预测方法
CN114676640A (zh) 基于遗传算法和maddpg算法的楼栋排布方法
CN115222310A (zh) 路径规划方法及装置、存储介质、终端
JP7841613B2 (ja) 推定装置、学習装置、推定方法、学習方法、及びプログラム
CN111680838A (zh) 一种空调负荷聚合功率预测方法及系统
CN117444978A (zh) 一种气动软体机器人的位置控制方法及其系统、设备
JP7533595B2 (ja) 予測方法、予測装置、及び予測プログラム
CN116451007A (zh) 用于运行技术系统的设备、系统和计算机实现的方法
KR20250009080A (ko) 메타버스 서비스를 위한 하이브리드 모델을 포함하는 컴퓨팅 부하 예측 시스템 및 그 방법
JP7687520B2 (ja) 強化学習装置、強化学習方法、及び強化学習プログラム
KR20230012869A (ko) 공조 시스템의 하이브리드 모델, 하이브리드 모델 학습 시스템 및 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23949701

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2025541196

Country of ref document: JP

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 2025541196

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE