JP7036545B2 - 能動的探索なしの強化学習に基づくオンライン学習法及び車両制御方法 - Google Patents

能動的探索なしの強化学習に基づくオンライン学習法及び車両制御方法 Download PDF

Info

Publication number
JP7036545B2
JP7036545B2 JP2017131700A JP2017131700A JP7036545B2 JP 7036545 B2 JP7036545 B2 JP 7036545B2 JP 2017131700 A JP2017131700 A JP 2017131700A JP 2017131700 A JP2017131700 A JP 2017131700A JP 7036545 B2 JP7036545 B2 JP 7036545B2
Authority
JP
Japan
Prior art keywords
vehicle
cost
control
arrival
state
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2017131700A
Other languages
English (en)
Japanese (ja)
Other versions
JP2018037064A (ja
JP2018037064A5 (enExample
Inventor
智樹 西
Original Assignee
トヨタ モーター エンジニアリング アンド マニュファクチャリング ノース アメリカ，インコーポレイティド
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by トヨタ モーター エンジニアリング アンド マニュファクチャリング ノース アメリカ，インコーポレイティド filed Critical トヨタ モーター エンジニアリング アンド マニュファクチャリング ノース アメリカ，インコーポレイティド
Publication of JP2018037064A publication Critical patent/JP2018037064A/ja
Publication of JP2018037064A5 publication Critical patent/JP2018037064A5/ja
Application granted granted Critical
Publication of JP7036545B2 publication Critical patent/JP7036545B2/ja
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/06—Improving the dynamic response of the control system, e.g. improving the speed of regulation or avoiding hunting or overshoot
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/0098—Details of control systems ensuring comfort, safety or stability not otherwise provided for
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W60/00—Drive control systems specially adapted for autonomous road vehicles
    • B60W60/001—Planning or execution of driving tasks
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B13/00—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
    • G05B13/02—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
    • G05B13/0265—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric the criterion being a learning criterion
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B13/00—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
    • G05B13/02—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
    • G05B13/04—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators
    • G05B13/041—Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators in which a variable is automatically adjusted to optimise the performance
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G05D1/0221—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving a learning process
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/092—Reinforcement learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N7/00—Computing arrangements based on specific mathematical models
    • G06N7/01—Probabilistic graphical models, e.g. probabilistic networks
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W2050/0001—Details of the control system
    • B60W2050/0002—Automatic control, details of type of controller or control system architecture
    • B60W2050/0013—Optimal controllers
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W2050/0001—Details of the control system
    • B60W2050/0002—Automatic control, details of type of controller or control system architecture
    • B60W2050/0014—Adaptive controllers
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W2050/0062—Adapting control system settings
    • B60W2050/0075—Automatic parameter input, automatic initialising or calibrating means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W2050/0062—Adapting control system settings
    • B60W2050/0075—Automatic parameter input, automatic initialising or calibrating means
    • B60W2050/0083—Setting, resetting, calibration
    • B60W2050/0088—Adaptive recalibration
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
    • B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
    • B60W2420/403—Image sensing, e.g. optical camera
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
    • B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
    • B60W2420/408—Radar; Laser, e.g. lidar
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2520/00—Input parameters relating to overall vehicle dynamics
    • B60W2520/10—Longitudinal speed
    • B60W2520/105—Longitudinal acceleration
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2520/00—Input parameters relating to overall vehicle dynamics
    • B60W2520/12—Lateral speed
    • B60W2520/125—Lateral acceleration
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2520/00—Input parameters relating to overall vehicle dynamics
    • B60W2520/14—Yaw
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2520/00—Input parameters relating to overall vehicle dynamics
    • B60W2520/16—Pitch
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2520/00—Input parameters relating to overall vehicle dynamics
    • B60W2520/18—Roll
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2520/00—Input parameters relating to overall vehicle dynamics
    • B60W2520/28—Wheel speed
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2556/00—Input parameters relating to data
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2556/00—Input parameters relating to data
    • B60W2556/10—Historical data
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/80—Technologies aiming to reduce greenhouse gasses emissions common to all road transportation technologies
    • Y02T10/84—Data processing systems or methods, management, administration

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Theoretical Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Mathematical Physics (AREA)
  • Computing Systems (AREA)
  • Medical Informatics (AREA)
  • Mechanical Engineering (AREA)
  • Human Computer Interaction (AREA)
  • Transportation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Pure & Applied Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Mathematical Analysis (AREA)
  • Computational Mathematics (AREA)
  • Algebra (AREA)
  • Probability & Statistics with Applications (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Feedback Control In General (AREA)
  • Business, Economics & Management (AREA)
  • Game Theory and Decision Science (AREA)
JP2017131700A 2016-07-08 2017-07-05 能動的探索なしの強化学習に基づくオンライン学習法及び車両制御方法 Active JP7036545B2 (ja)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US15/205,558 2016-07-08
US15/205,558 US10065654B2 (en) 2016-07-08 2016-07-08 Online learning and vehicle control method based on reinforcement learning without active exploration

Publications (3)

Publication Number Publication Date
JP2018037064A JP2018037064A (ja) 2018-03-08
JP2018037064A5 JP2018037064A5 (enExample) 2020-08-20
JP7036545B2 true JP7036545B2 (ja) 2022-03-15

Family

ID=60892997

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2017131700A Active JP7036545B2 (ja) 2016-07-08 2017-07-05 能動的探索なしの強化学習に基づくオンライン学習法及び車両制御方法

Country Status (2)

Country Link
US (1) US10065654B2 (enExample)
JP (1) JP7036545B2 (enExample)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023199610A1 (en) * 2022-04-14 2023-10-19 Mitsubishi Electric Corporation System and method for motion and path planning for trailer-based vehicle

Families Citing this family (41)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10296004B2 (en) * 2017-06-21 2019-05-21 Toyota Motor Engineering & Manufacturing North America, Inc. Autonomous operation for an autonomous vehicle objective in a multi-vehicle environment
EP3721340A4 (en) * 2017-12-04 2021-09-08 Optimum Semiconductor Technologies Inc. NEURONAL NETWORK ACCELERATOR SYSTEM AND ARCHITECTURE
JP6884685B2 (ja) * 2017-12-08 2021-06-09 三菱重工業株式会社 制御装置、無人システム、制御方法及びプログラム
US11130497B2 (en) 2017-12-18 2021-09-28 Plusai Limited Method and system for ensemble vehicle control prediction in autonomous driving vehicles
US20190185012A1 (en) 2017-12-18 2019-06-20 PlusAI Corp Method and system for personalized motion planning in autonomous driving vehicles
US11273836B2 (en) * 2017-12-18 2022-03-15 Plusai, Inc. Method and system for human-like driving lane planning in autonomous driving vehicles
US20210116930A1 (en) * 2018-02-28 2021-04-22 Sony Corporation Information processing apparatus, information processing method, program, and mobile object
JP7035734B2 (ja) * 2018-03-30 2022-03-15 富士通株式会社 強化学習プログラム、強化学習方法、および強化学習装置
US10990096B2 (en) * 2018-04-27 2021-04-27 Honda Motor Co., Ltd. Reinforcement learning on autonomous vehicles
CN108959467B (zh) * 2018-06-20 2021-10-15 华东师范大学 一种基于强化学习的问句和答案句相关度的计算方法
US11823039B2 (en) 2018-08-24 2023-11-21 International Business Machines Corporation Safe and fast exploration for reinforcement learning using constrained action manifolds
EP3629105A1 (en) * 2018-09-27 2020-04-01 Bayerische Motoren Werke Aktiengesellschaft High-level decision making for safe and reasonable autonomous lane changing using reinforcement learning
US10940863B2 (en) * 2018-11-01 2021-03-09 GM Global Technology Operations LLC Spatial and temporal attention-based deep reinforcement learning of hierarchical lane-change policies for controlling an autonomous vehicle
JP7206874B2 (ja) 2018-12-10 2023-01-18 富士電機株式会社 制御装置、制御方法及びプログラム
EP3877117B1 (en) 2018-12-13 2024-10-16 Siemens Aktiengesellschaft Automated system including reachability analysis
CN109901572B (zh) * 2018-12-13 2022-06-28 华为技术有限公司 自动驾驶方法、训练方法及相关装置
EP3693243B1 (en) * 2019-02-06 2024-11-06 Zenuity AB Method and system for controlling an automated driving system of a vehicle
DE102019206908B4 (de) * 2019-05-13 2022-02-17 Psa Automobiles Sa Verfahren zum Trainieren wenigstens eines Algorithmus für ein Steuergerät eines Kraftfahrzeugs, Computerprogrammprodukt, Kraftfahrzeug sowie System
JP2021017168A (ja) * 2019-07-22 2021-02-15 本田技研工業株式会社 ダンパ制御システム、車両、情報処理装置およびそれらの制御方法、ならびにプログラム
EP3783446B1 (de) * 2019-08-21 2021-08-11 dSPACE digital signal processing and control engineering GmbH Computerimplementiertes verfahren und testeinheit zum approximieren einer teilmenge von testergebnissen
CN110843746B (zh) * 2019-11-28 2022-06-14 的卢技术有限公司 一种基于强化学习的防抱死刹车控制方法及系统
CN113552869B (zh) * 2020-04-23 2023-07-07 华为技术有限公司 优化决策规控的方法、控制车辆行驶的方法和相关装置
US12246699B2 (en) * 2020-06-26 2025-03-11 Mitsubishi Electric Research Laboratories, Inc. System and method for data-driven reference generation
CN111796522B (zh) * 2020-07-16 2022-06-03 上海智驾汽车科技有限公司 一种车辆状态估计方法
CN112289044B (zh) * 2020-11-02 2021-09-07 南京信息工程大学 基于深度强化学习的高速公路道路协同控制系统及方法
US12384410B2 (en) 2021-03-05 2025-08-12 The Research Foundation For The State University Of New York Task-motion planning for safe and efficient urban driving
US11669593B2 (en) 2021-03-17 2023-06-06 Geotab Inc. Systems and methods for training image processing models for vehicle data collection
US12322192B2 (en) 2021-03-17 2025-06-03 Geotab Inc. Systems and methods for vehicle data collection by image analysis
US11682218B2 (en) 2021-03-17 2023-06-20 Geotab Inc. Methods for vehicle data collection by image analysis
US11886196B2 (en) * 2021-04-05 2024-01-30 Mitsubishi Electric Research Laboratories, Inc. Controlling machine operating in uncertain environment discoverable by sensing
CN113253612B (zh) * 2021-06-01 2021-09-17 苏州浪潮智能科技有限公司 一种自动驾驶控制方法、装置、设备及可读存储介质
CN113359476B (zh) * 2021-07-09 2022-09-16 广东华中科技大学工业技术研究院 离散时间下多智能体系统的一致性控制算法设计方法
US12272184B2 (en) 2021-11-05 2025-04-08 Geotab Inc. AI-based input output expansion adapter for a telematics device
US11693920B2 (en) * 2021-11-05 2023-07-04 Geotab Inc. AI-based input output expansion adapter for a telematics device and methods for updating an AI model thereon
US20230376832A1 (en) * 2022-05-18 2023-11-23 GM Global Technology Operations LLC Calibrating parameters within a virtual environment using reinforcement learning
CN117437771A (zh) * 2022-07-15 2024-01-23 北京图森智途科技有限公司 一种目标状态估计方法、装置、电子设备和介质
CN115230720A (zh) * 2022-07-22 2022-10-25 江苏大学 基于深度拉格朗日神经网络的自动驾驶车辆控制系统及方法
US20240157978A1 (en) * 2022-11-11 2024-05-16 Waabi Innovation Inc. Mixed reality simulation for autonomous systems
US20240326815A1 (en) * 2023-03-31 2024-10-03 Ford Global Technologies, Llc Vehicle maneuvering
CN116149262B (zh) * 2023-04-23 2023-07-04 山东科技大学 一种伺服系统的跟踪控制方法及系统
CN119739039B (zh) * 2024-12-20 2025-11-25 北京航空航天大学 拦截制导策略获取方法、装置、设备及存储介质

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006313512A (ja) 2005-04-04 2006-11-16 Sony Corp 学習制御装置、学習制御方法、およびプログラム
US20130262353A1 (en) 2012-03-30 2013-10-03 Board Of Regents, The University Of Texas System Optimal online adaptive controller
US20160092764A1 (en) 2011-12-07 2016-03-31 Paul Burchard Sparse Neural Control

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU3477397A (en) * 1996-06-04 1998-01-05 Paul J. Werbos 3-brain architecture for an intelligent decision and control system
JPH10254505A (ja) * 1997-03-14 1998-09-25 Toyota Motor Corp 自動制御装置
US6532454B1 (en) * 1998-09-24 2003-03-11 Paul J. Werbos Stable adaptive control using critic designs
US6882992B1 (en) * 1999-09-02 2005-04-19 Paul J. Werbos Neural networks for intelligent control
US6411871B1 (en) 2000-08-05 2002-06-25 American Gnc Corporation Autonomous navigation, guidance and control using LDRI
US8854199B2 (en) 2009-01-26 2014-10-07 Lytx, Inc. Driver risk assessment system and method employing automated driver log
US8812226B2 (en) * 2009-01-26 2014-08-19 GM Global Technology Operations LLC Multiobject fusion module for collision preparation system
US8380367B2 (en) 2009-03-26 2013-02-19 The University Of North Dakota Adaptive surveillance and guidance system for vehicle collision avoidance and interception
JP2012533056A (ja) * 2009-07-09 2012-12-20 トムトム インターナショナル ベスローテン フエンノートシャップ 経路探索加速データとともに地図データを用いるナビゲーション装置
WO2012117044A2 (de) 2011-03-01 2012-09-07 Continental Teves Ag & Co. Ohg Verfahren und vorrichtung zur prädiktion und adaption von bewegungstrajektorien von kraftfahrzeugen
US8965834B2 (en) * 2011-12-07 2015-02-24 Extendabrain Corporation Particle methods for nonlinear control
CN104247467A (zh) 2012-02-17 2014-12-24 英特托拉斯技术公司 用于交通工具策略施行的方法和系统
US20140142948A1 (en) 2012-11-21 2014-05-22 Somya Rathi Systems and methods for in-vehicle context formation
US10133250B2 (en) * 2014-06-20 2018-11-20 Veritone Alpha, Inc. Managing construction of decision modules to control target systems
US20160202670A1 (en) * 2015-01-08 2016-07-14 Northwestern University System and method for sequential action control for nonlinear systems
US9511767B1 (en) * 2015-07-01 2016-12-06 Toyota Motor Engineering & Manufacturing North America, Inc. Autonomous vehicle action planning using behavior prediction
US9916703B2 (en) * 2015-11-04 2018-03-13 Zoox, Inc. Calibration for autonomous vehicle operation
US9568915B1 (en) * 2016-02-11 2017-02-14 Mitsubishi Electric Research Laboratories, Inc. System and method for controlling autonomous or semi-autonomous vehicle
US10061316B2 (en) * 2016-07-08 2018-08-28 Toyota Motor Engineering & Manufacturing North America, Inc. Control policy learning and vehicle control method based on reinforcement learning without active exploration

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006313512A (ja) 2005-04-04 2006-11-16 Sony Corp 学習制御装置、学習制御方法、およびプログラム
US20160092764A1 (en) 2011-12-07 2016-03-31 Paul Burchard Sparse Neural Control
US20130262353A1 (en) 2012-03-30 2013-10-03 Board Of Regents, The University Of Texas System Optimal online adaptive controller

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023199610A1 (en) * 2022-04-14 2023-10-19 Mitsubishi Electric Corporation System and method for motion and path planning for trailer-based vehicle

Also Published As

Publication number Publication date
US20180009445A1 (en) 2018-01-11
US10065654B2 (en) 2018-09-04
JP2018037064A (ja) 2018-03-08

Similar Documents

Publication Publication Date Title
JP7036545B2 (ja) 能動的探索なしの強化学習に基づくオンライン学習法及び車両制御方法
JP6856575B2 (ja) 能動的探索なしの強化学習に基づく制御ポリシー学習及び車両制御方法
US10061316B2 (en) Control policy learning and vehicle control method based on reinforcement learning without active exploration
US11269329B2 (en) Dynamic model with learning based localization correction system
US11467591B2 (en) Online agent using reinforcement learning to plan an open space trajectory for autonomous vehicles
US11493926B2 (en) Offline agent using reinforcement learning to speedup trajectory planning for autonomous vehicles
US20240409126A1 (en) Systems and Methods for Generating Behavioral Predictions in Reaction to Autonomous Vehicle Movement
JP7361775B2 (ja) 自律運転のための個人運転スタイル学習
CN112977472B (zh) 预测车辆的移动轨迹的方法及系统
CN110244701B (zh) 用于基于自动生成的课程序列的自主车辆的强化学习的方法和装置
CN112829769B (zh) 自动驾驶车辆的混合规划系统
CN113525406B (zh) 用于车辆运动控制器的基于贝叶斯全局优化的参数调谐
US10800427B2 (en) Systems and methods for a vehicle controller robust to time delays
US11409284B2 (en) Relaxation optimization model to plan an open space trajectory for autonomous vehicles
US11815891B2 (en) End dynamics and constraints relaxation algorithm on optimizing an open space trajectory
US11055540B2 (en) Method for determining anchor boxes for training neural network object detection models for autonomous driving
CN112977473B (zh) 用于预测移动障碍物驶出十字路口的方法及系统
CN111208814B (zh) 用于自动车辆的、利用动态模型的、基于记忆的最优运动规划
WO2023147769A1 (en) Systems, methods, and computer-readable media for spatio-temporal motion planning
US11656627B2 (en) Open space path planning using inverse reinforcement learning
EP3857327B1 (en) Implementation of dynamic cost function of self-driving vehicles
US11467584B2 (en) Multi-layer grid based open space planner
CN112947405B (zh) 使用用于自主驾驶车辆的怠速的速度规划准则的速度规划
US20240025442A1 (en) Trajectory planning in autonomous driving vehicles for unforeseen scenarios
KR20230075897A (ko) 주행 경로에 따른 이동체의 적응적 제어 방법

Legal Events

Date Code Title Description
A521 Request for written amendment filed

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20200703

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20200703

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20210528

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20210615

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20220201

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20220303

R150 Certificate of patent or registration of utility model

Ref document number: 7036545

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R150

S111 Request for change of ownership or part of ownership

Free format text: JAPANESE INTERMEDIATE CODE: R313113

R350 Written notification of registration of transfer

Free format text: JAPANESE INTERMEDIATE CODE: R350

R250 Receipt of annual fees

Free format text: JAPANESE INTERMEDIATE CODE: R250