CN113759715A - Intelligent vibration control system with reinforcement learning function for ship equipment - Google Patents

Intelligent vibration control system with reinforcement learning function for ship equipment Download PDF

Info

Publication number
CN113759715A
CN113759715A CN202110916702.4A CN202110916702A CN113759715A CN 113759715 A CN113759715 A CN 113759715A CN 202110916702 A CN202110916702 A CN 202110916702A CN 113759715 A CN113759715 A CN 113759715A
Authority
CN
China
Prior art keywords
neural network
equipment
vibration
controlled
signal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110916702.4A
Other languages
Chinese (zh)
Inventor
丰星星
陈虹
喻卫宁
张昊
钱家昌
赵志高
蔡玄
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
719th Research Institute of CSIC
Original Assignee
719th Research Institute of CSIC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 719th Research Institute of CSIC filed Critical 719th Research Institute of CSIC
Priority to CN202110916702.4A priority Critical patent/CN113759715A/en
Publication of CN113759715A publication Critical patent/CN113759715A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B13/00Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion
    • G05B13/02Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
    • G05B13/04Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators
    • G05B13/042Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators in which a parameter or coefficient is automatically adjusted to optimise the performance

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Feedback Control In General (AREA)
  • Vibration Prevention Devices (AREA)

Abstract

The invention provides an intelligent vibration control system of ship equipment with a reinforcement learning function, wherein in the actual vibration control process of the ship equipment, a neural network control module is divided into a neural network A and a neural network B, the neural network A directly maps a control current signal required by a current controller according to an acceleration signal, a speed signal and a displacement signal output by a signal acquisition module and transmits the control current signal to the current controller, the current controller generates a corresponding current and transmits the current to a motor, and the motor generates a corresponding control force to inhibit the vibration of the equipment to be controlled or a ship body; the neural network B updates the parameters thereof only by reinforcement learning after the vibration working condition changes, so that the neural network A maintains a better vibration control effect; therefore, the intelligent vibration control system does not need to establish a mathematical model of the controlled object and the parameters of the controlled equipment, has strong self-adaptive capacity and can adapt to the complex vibration working condition change.

Description

Intelligent vibration control system with reinforcement learning function for ship equipment
Technical Field
The invention belongs to the technical field of vibration control of ship equipment, and particularly relates to an intelligent vibration control system of ship equipment with a reinforcement learning function.
Background
The motors, turbines, pumps and other equipment on various ships are generally connected with the ship body through rubber vibration isolators. The rubber vibration isolator has the connecting function and the vibration isolating function. However, the rubber vibration isolator generally has a good vibration isolation effect on high-frequency spectral lines, and has a poor vibration isolation effect on low-frequency spectral lines. Therefore, active vibration control techniques have been proposed to solve the problem of damping low frequency spectral lines. The traditional active vibration control technology depends on an accurate mathematical model of a controlled object, and needs to know information such as the accurate mass center position, mass and inertia of equipment, the rigidity and damping of a rubber vibration isolator, a vibration source and the like, and many information of the information can not be accurately acquired, so that the active vibration control effect is poor. Secondly, the traditional active vibration control technology has weak self-adaption capability, and when the vibration working condition is greatly changed, the active vibration reduction effect is deteriorated.
Disclosure of Invention
In order to solve the problems, the invention provides the intelligent vibration control system of the ship equipment with the reinforcement learning function, which can directly map the control force according to the equipment response without establishing a mathematical model of a controlled object, has strong self-adaptive capacity and can adapt to the complex vibration working condition change.
An intelligent vibration control system of ship equipment with a reinforcement learning function comprises an acceleration sensor 1, a signal acquisition module 2, a neural network control module 3, a current controller 4 and a motor 5, wherein the neural network control module 3 comprises a parameter extraction unit, and a neural network A and a neural network B which are identical in structure;
the acceleration sensor 1 is used for acquiring an acceleration signal of a ship body 8 or a device 6 to be controlled which is arranged on the ship body 8; the signal acquisition module 2 is used for sequentially carrying out filtering conditioning and integration on the acceleration signal to obtain a speed signal and a displacement signal;
the neural network A directly outputs a control current signal required for inhibiting the current vibration of the ship body 8 or the equipment to be controlled 6 according to the currently input equipment response, wherein the equipment response is any combination of an acceleration signal, a speed signal and a displacement signal; the current controller is used for driving the motor 5 to generate control force acting on the ship body 8 or the equipment to be controlled 6 according to the control current signal so as to realize the inhibition of the vibration of the ship body 8 or the equipment to be controlled 6;
the neural network B is used for updating the network parameters of the neural network B based on the current equipment response and the control current signal in a reinforcement learning mode when the vibration working condition of the ship body 8 or the equipment to be controlled 6 is changed;
the parameter extraction unit is used for extracting updated network parameters of the neural network B and assigning the updated network parameters to the neural network A when the vibration working condition of the ship body 8 or the equipment to be controlled 6 changes, so that the suppression effect of the control current signals output by the updated neural network A on the vibration of the ship body 8 or the equipment to be controlled 6 meets the set requirement.
Further, the intelligent vibration control system for the ship equipment with the reinforcement learning function further comprises a first rubber vibration isolator 71, a second rubber vibration isolator 72, a third rubber vibration isolator 73 and a fourth rubber vibration isolator 74;
the device 6 to be controlled is elastically connected with the ship body 8 through a first rubber vibration isolator 71, a second rubber vibration isolator 72, a third rubber vibration isolator 73 and a fourth rubber vibration isolator 74.
Further, the intelligent vibration control system of the ship equipment with the reinforcement learning function also comprises an adapter plate 9;
the adapter plate 9 clamps the motor 5 on the device 6 to be controlled through the first bolt 101 and the second bolt 102, so that the motor 5 directly acts a control force on the device 6 to be controlled, and vibration of the device 6 to be controlled is further suppressed.
Further, when the suppression effect of the updated control current signal output by the neural network a on the vibration of the ship body 8 or the device to be controlled 6 meets the set requirement, the neural network B stops updating the network parameters thereof through reinforcement learning, and the network parameters of the neural network a also stop changing.
Further, the reinforcement learning mode is a DDPG algorithm.
Further, the neural network a and the neural network B are BP neural networks.
Has the advantages that:
1. the invention provides an intelligent vibration control system of ship equipment with a reinforcement learning function, wherein in the actual vibration control process of the ship equipment, a neural network control module is divided into a neural network A and a neural network B, the neural network A directly maps a control current signal required by a current controller according to an acceleration signal, a speed signal and a displacement signal output by a signal acquisition module and transmits the control current signal to the current controller, the current controller generates a corresponding current and transmits the current to a motor, and the motor generates a corresponding control force to inhibit the vibration of the equipment to be controlled or a ship body; the neural network B updates the parameters thereof only by reinforcement learning after the vibration working condition changes, so that the neural network A maintains a better vibration control effect; therefore, the intelligent vibration control system does not need to establish a mathematical model of a controlled object, and also does not need parameters of the controlled equipment, such as the mass, inertia, geometric dimension, rigidity and the like of the controlled equipment.
2. The invention provides an intelligent vibration control system of ship equipment with a reinforcement learning function, wherein a neural network A acquires control force for inhibiting vibration according to current equipment response in real time, and a neural network B updates network parameters of the neural network A and a ship body in a reinforcement learning mode only when the vibration working condition of the ship body or equipment to be controlled is changed; therefore, the working frequencies of the neural network A and the neural network B are different, and calculation resources can be greatly saved, for example, the neural network A calculates a control current signal once every 0.001 second according to equipment response, and the neural network B updates the network parameters of the neural network A and the neural network B once every 10s after the vibration working condition is changed.
3. The intelligent vibration control system for the ship equipment, provided by the invention, has the advantages of simple structure, convenience in implementation and wide application range, and achieves a good technical effect.
Drawings
Fig. 1 is a schematic structural diagram of an intelligent vibration control system for ship equipment with a reinforcement learning function provided by the invention;
FIG. 2 is a flow chart of the neural network control module reinforcement learning provided by the present invention;
the control system comprises an acceleration sensor 1, a signal acquisition module 2, a neural network control module 3, a current controller 4, a motor 5, equipment to be controlled 6, a ship body 8, a first rubber vibration isolator 71, a second rubber vibration isolator 72, a third rubber vibration isolator 73, a fourth rubber vibration isolator 74, an adapter plate 9, a first bolt 101 and a second bolt 102.
Detailed Description
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.
Referring to fig. 1, the intelligent vibration control system for ship equipment with reinforcement learning function of the present invention includes an acceleration sensor 1, a signal acquisition module 2, a neural network control module 3, a current controller 4, and an execution motor 5. The acceleration sensor 1 can be arranged on the ship body 8 and used for measuring an acceleration signal of the ship body 8, the acceleration sensor 1 can also be arranged on the device 6 to be controlled and used for measuring an acceleration signal of the device 6 to be controlled, the device 6 to be controlled is elastically connected with the ship body 8 through the first rubber vibration isolator 71, the second rubber vibration isolator 72, the third rubber vibration isolator 73 and the fourth rubber vibration isolator 74, or the ship body 8 and the device 6 to be controlled are simultaneously provided with a plurality of acceleration sensors 1. The acceleration sensor 1 is connected with the signal acquisition module 2, transmits an acceleration signal to the signal acquisition module 2, and the signal acquisition module 2 conditions the acceleration signal and obtains a speed signal and a displacement signal through integration. The signal acquisition module 2 is connected with the neural network module 3 and transmits the acceleration signal, the speed signal and the displacement signal to the neural network control module 3. The neural network module 3 is connected with the current controller 4, and the neural network control module 3 directly maps a control current signal required by the current controller 4 according to the acceleration signal, the speed signal and the displacement signal transmitted by the signal acquisition module 2 and transmits the control current signal to the current controller 4. The current controller 4 is connected to the motor 5, and the current controller 4 supplies a desired current to the motor 5, and the motor 5 generates a control force. The motor 5 is arranged on the ship equipment, and the motor 5 is used for controlling the vibration of the ship equipment.
The motor 5 is fixedly connected to the ship equipment through the adapter plate 9, the first bolt 101 and the second bolt 102. The adapter plate 9 clamps the motor 5 on the device 6 to be controlled through the first bolt 101 and the second bolt 102, so that the motor 5 directly acts a control force on the device 6 to be controlled.
Referring to fig. 2, the neural network control module 3 has a reinforcement learning function, and the neural network module 3 runs a neural network a and a neural network B having the same structure. The neural network module 3 needs the following parameters for reinforcement learning: the device responds 11 with a corresponding control current signal 12. The device response 11 comprises all or part of the acceleration, velocity and displacement signals from the signal acquisition module 2, i.e. any combination of acceleration, velocity and displacement signals. The control current signal 12 is the control current signal 12 output by the neural network control module 3. It can be seen that the intelligent vibration control algorithm in the present invention does not require parameters of the controlled device itself, such as mass, inertia, geometry, stiffness, etc. of the controlled device.
The reinforcement learning of the neural network vibration control module 3 comprises the following steps:
a. the neural network control module 3 is initialized with parameters, and the initialized parameters can be randomly assigned. The neural network module 3 runs a neural network A and a neural network B which have the same structure, and the initialization parameters of the neural network A and the neural network B are the same.
b. The neural network A calculates a corresponding control current signal 12 according to the equipment response 11, the control current signal 12 is transmitted to the current controller 4, and the current controller 4 generates current to drive the motor 5 to generate control force to act on the equipment 6 to be controlled or the ship body 8. While the neural network a is working, the neural network B updates its own parameters by reinforcement learning using the device response 11 and the corresponding control current signal 12, and assigns the updated parameters to the neural network a. The neural network A achieves better vibration control by continuously improving self parameters. When the vibration control effect of the neural network module 3 reaches the set target, the neural network B stops learning and updating the parameters thereof, and the parameters of the neural network A also stop changing along with the learning and updating. Wherein, the reinforcement learning algorithm may be a DDPG algorithm.
c. And when the vibration working condition of the equipment is changed and the vibration control effect of the neural network module 3 is deteriorated, the neural network B starts reinforcement learning to update the parameters of the neural network B, and assigns the updated parameters to the neural network A until the vibration control effect of the neural network module 3 is improved again.
Therefore, the invention provides an intelligent vibration control system of ship equipment with a reinforcement learning function, which has two remarkable advantages: firstly, a mathematical model of a controlled object is not needed; secondly, the self-adaptive capacity is strong, when the vibration working condition changes, the system updates the neural network parameters through reinforcement learning, and a better active vibration reduction effect can be maintained.
Further, the invention patent "ocean platform intelligent vibration control device CN 102768493B" discloses an ocean platform intelligent vibration control device, wherein platform parameters required for training a BP neural network vibration control module include a platform mass matrix, a stiffness matrix, and a damping matrix; meanwhile, the active control algorithm adopted in the invention patent CN102768493B is an LQR optimal control algorithm, and according to the public knowledge, establishing the LQR algorithm first requires establishing a mathematical model of a controlled object. The intelligent vibration control system disclosed by the invention does not need to establish a mass matrix, a rigidity matrix, a damping matrix and a mathematical model of the controlled object.
The invention discloses a tension leg type wave energy and wind energy comprehensive power generation device and an intelligent vibration control system CN110397561B thereof, and discloses the tension leg type wave energy and wind energy comprehensive power generation device, wherein a BP neural network vibration control module can directly map out a control current signal according to a displacement signal and an acceleration signal, and the BP neural network does not have the function of updating self parameters on line and has no self-adaptability. The neural network vibration control module can update the neural network parameters through reinforcement learning, and has self-adaptability.
The present invention is capable of other embodiments, and various changes and modifications may be made by one skilled in the art without departing from the spirit and scope of the invention.

Claims (6)

1. The intelligent vibration control system of the ship equipment with the reinforcement learning function is characterized by comprising an acceleration sensor (1), a signal acquisition module (2), a neural network control module (3), a current controller (4) and a motor (5), wherein the neural network control module (3) comprises a parameter extraction unit, a neural network A and a neural network B which are identical in structure;
the acceleration sensor (1) is used for acquiring an acceleration signal of the ship body (8) or a device (6) to be controlled, which is arranged on the ship body (8); the signal acquisition module (2) is used for sequentially carrying out filtering conditioning and integration on the acceleration signal to obtain a speed signal and a displacement signal;
the neural network A directly outputs a control current signal required for inhibiting the current vibration of the ship body (8) or the equipment to be controlled (6) according to the currently input equipment response, wherein the equipment response is any combination of an acceleration signal, a speed signal and a displacement signal; the current controller is used for driving the motor (5) to generate a control force acting on the ship body (8) or the equipment (6) to be controlled according to the control current signal so as to realize the suppression of the vibration of the ship body (8) or the equipment (6) to be controlled;
the neural network B is used for updating the network parameters of the neural network B based on the current equipment response and the control current signal in a reinforcement learning mode when the vibration working condition of the ship body (8) or the equipment (6) to be controlled changes;
the parameter extraction unit is used for extracting updated network parameters of the neural network B and assigning the updated network parameters to the neural network A when the vibration working condition of the ship body (8) or the equipment (6) to be controlled changes, so that the suppression effect of the control current signals output by the updated neural network A on the vibration of the ship body (8) or the equipment (6) to be controlled meets the set requirement.
2. The intelligent vibration control system for ship equipment with the reinforcement learning function as recited in claim 1, further comprising a first rubber vibration isolator (71), a second rubber vibration isolator (72), a third rubber vibration isolator (73) and a fourth rubber vibration isolator (74);
the equipment (6) to be controlled is elastically connected with the ship body (8) through the first rubber vibration isolator (71), the second rubber vibration isolator (72), the third rubber vibration isolator (73) and the fourth rubber vibration isolator (74).
3. The intelligent vibration control system for ship equipment with reinforcement learning function as claimed in claim 1, further comprising an adapter plate (9);
the adapter plate (9) clamps the motor (5) on the device (6) to be controlled through the first bolt (101) and the second bolt (102), so that the motor (5) directly acts control force on the device (6) to be controlled, and vibration of the device (6) to be controlled is further suppressed.
4. The intelligent vibration control system with reinforcement learning function for ship equipment according to claim 1, wherein when the suppression effect of the updated control current signal output by the neural network a on the vibration of the ship body (8) or the equipment to be controlled (6) reaches the set requirement, the neural network B stops updating the network parameters thereof through reinforcement learning, and the network parameters of the neural network a also stop changing.
5. The intelligent vibration control system for ship equipment with reinforcement learning function as claimed in claim 1, wherein the reinforcement learning mode is DDPG algorithm.
6. The intelligent vibration control system of ship equipment with reinforcement learning function as claimed in claim 1, wherein the neural network a and the neural network B are BP neural networks.
CN202110916702.4A 2021-08-11 2021-08-11 Intelligent vibration control system with reinforcement learning function for ship equipment Pending CN113759715A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110916702.4A CN113759715A (en) 2021-08-11 2021-08-11 Intelligent vibration control system with reinforcement learning function for ship equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110916702.4A CN113759715A (en) 2021-08-11 2021-08-11 Intelligent vibration control system with reinforcement learning function for ship equipment

Publications (1)

Publication Number Publication Date
CN113759715A true CN113759715A (en) 2021-12-07

Family

ID=78788941

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110916702.4A Pending CN113759715A (en) 2021-08-11 2021-08-11 Intelligent vibration control system with reinforcement learning function for ship equipment

Country Status (1)

Country Link
CN (1) CN113759715A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114489167A (en) * 2021-12-17 2022-05-13 中国船舶重工集团公司第七一九研究所 Warship rotary mechanical equipment feedforward vibration control system based on supervised learning

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101609305A (en) * 2008-06-18 2009-12-23 中国科学院自动化研究所 A kind of active vibration isolation control method based on voice coil motor
CN102768493A (en) * 2012-07-02 2012-11-07 江苏科技大学 Intelligent vibration control device of ocean platform
CN111487863A (en) * 2020-04-14 2020-08-04 东南大学 Active suspension reinforcement learning control method based on deep Q neural network
CN112277558A (en) * 2019-07-22 2021-01-29 本田技研工业株式会社 Damper control system, vehicle, information processing device, control method for damper control system, control method for information processing device, and recording medium
CN112698572A (en) * 2020-12-22 2021-04-23 西安交通大学 Structural vibration control method, medium and equipment based on reinforcement learning

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101609305A (en) * 2008-06-18 2009-12-23 中国科学院自动化研究所 A kind of active vibration isolation control method based on voice coil motor
CN102768493A (en) * 2012-07-02 2012-11-07 江苏科技大学 Intelligent vibration control device of ocean platform
CN112277558A (en) * 2019-07-22 2021-01-29 本田技研工业株式会社 Damper control system, vehicle, information processing device, control method for damper control system, control method for information processing device, and recording medium
CN111487863A (en) * 2020-04-14 2020-08-04 东南大学 Active suspension reinforcement learning control method based on deep Q neural network
CN112698572A (en) * 2020-12-22 2021-04-23 西安交通大学 Structural vibration control method, medium and equipment based on reinforcement learning

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
奔跑的林小川: "学习笔记:DDPG", HTTPS://BLOG.CSDN.NET/QQ_31239495/ARTICLE/DETAILS/80313803, pages 1 *
韩向敏 等: "一种基于深度强化学习的自适应巡航控制算法", 计算机工程, vol. 47, no. 4, pages 34 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114489167A (en) * 2021-12-17 2022-05-13 中国船舶重工集团公司第七一九研究所 Warship rotary mechanical equipment feedforward vibration control system based on supervised learning

Similar Documents

Publication Publication Date Title
CN100524132C (en) Method for testing of a combined dynamic positioning and power management system
CN107387382B (en) Control method, compressor assembly and the refrigeration equipment of compressor amplitude
CN101976560B (en) Method for improving performance of feedforward narrow-band active noise control system
CN103092225B (en) Controller, system and method for actively controlling vibration of arm support
CN113759715A (en) Intelligent vibration control system with reinforcement learning function for ship equipment
CN102979636B (en) A kind of method and device reducing power assembly system vibration under idling operation
CN110865539A (en) Unmanned ship tracking error constraint control method under random interference
CN112809676B (en) Joint actuator, robot, storage medium, and electronic device
CN113954617B (en) Model-based feedforward and feedback combined active control method for vibration of automobile engine
WO2020040963A2 (en) System and method for minimizing fuel usage and emissions of a marine vessel
CN107102658B (en) Method and device for identifying off-line secondary channel
JP5839154B1 (en) Drivetrain testing system
CN110126986A (en) A kind of intelligence control system and its control method that liquid tank system is swung
CN110316118A (en) A kind of noise control method and device
CN114489167A (en) Warship rotary mechanical equipment feedforward vibration control system based on supervised learning
CN116146342A (en) Engine active suspension hybrid control method, device, equipment and storage medium
CN113859252A (en) Vehicle weight determining method and device
CN111123705B (en) Design method for active vibration control of propeller and transmission shaft system
CN102518743B (en) Vibration control method for coupling vibration of tower crane-cable tower structure
CN110880893B (en) Magnetic gear motor control method and control circuit
CN105631179A (en) Carrier transient response indication method based on star satellite load couple analysis result
CN110083127A (en) A kind of servo-driver control method and system for articulated robot
CN116198665A (en) Buoy anti-rolling device and anti-rolling control method thereof
CN109677396A (en) A kind of active damping method for engine-electric machine coupled system twisting vibration
Hung et al. Remote control system of a 6 DOF underwater robot

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20211207

WD01 Invention patent application deemed withdrawn after publication