CN117226847A - Control method and system of teleoperation equipment - Google Patents
Control method and system of teleoperation equipment Download PDFInfo
- Publication number
- CN117226847A CN117226847A CN202311455204.XA CN202311455204A CN117226847A CN 117226847 A CN117226847 A CN 117226847A CN 202311455204 A CN202311455204 A CN 202311455204A CN 117226847 A CN117226847 A CN 117226847A
- Authority
- CN
- China
- Prior art keywords
- information
- robot
- working
- pose
- control
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000004891 communication Methods 0.000 claims description 48
- 238000004590 computer program Methods 0.000 claims description 15
- 238000007781 pre-processing Methods 0.000 claims description 10
- 230000000007 visual effect Effects 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 9
- 230000003993 interaction Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 230000002159 abnormal effect Effects 0.000 claims description 4
- 238000004088 simulation Methods 0.000 claims description 4
- 230000001502 supplementing effect Effects 0.000 claims description 4
- 230000005540 biological transmission Effects 0.000 abstract description 8
- 238000010586 diagram Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 238000005286 illumination Methods 0.000 description 6
- 230000033001 locomotion Effects 0.000 description 6
- 230000001133 acceleration Effects 0.000 description 4
- 239000003795 chemical substances by application Substances 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 241000282472 Canis lupus familiaris Species 0.000 description 1
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 1
- 238000005452 bending Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 229910021389 graphene Inorganic materials 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Manipulator (AREA)
Abstract
The application relates to a control method and device of teleoperation equipment, computer equipment and storage medium. The method comprises the following steps: the remote control system is applied to a remote server, and the remote server is connected with at least two robots in the teleoperation equipment; acquiring control information corresponding to each robot based on a distributed operating system; wherein the distributed operating system is carried on all robots; determining the operation information of teleoperation equipment, and calculating the operation pose of each robot according to the operation information and the control information; based on the working pose, generating a control instruction corresponding to the robots, and based on the control instruction, cooperatively controlling at least two robots to respectively move and adjust to the corresponding working pose. The method can solve the problem of disordered data transmission sequence of different robot terminals; the intelligent control of the different robots based on the cooperation of the distributed operating system and the remote server is realized, and the working efficiency and the working quality of the operation are improved.
Description
Technical Field
The present application relates to the field of cooperative control of multiple robots, and in particular, to a method, a system, a computer device, a storage medium, and a computer program product for controlling a teleoperation device.
Background
In the age of rapid development of information technology today, the demand of robots is increasing. To meet these demands, robotic systems have become increasingly complex, ranging from initial basic motion control to the operation today capable of achieving a variety of precision industrial manipulations and extreme environmental conditions. At the same time, the software scale of the robot control system is also expanding.
Teleoperation is usually, in fact, a remote operation. Correspondingly, a robot that remotely performs an operation is called a teleoperational device. However, when the traditional robot control system performs teleoperation multi-terminal data transmission, because the data receiving rate of the data storage module has an upper limit, when the teleoperation multi-terminal data is received, the data transmission sequence of different terminals is disordered, the transmission line is easy to be blocked, and the data transmission is affected. This presents great difficulties for the implementation, verification and testing of the robot control system.
Therefore, a control method of teleoperation equipment is needed for intelligently controlling all the intelligent agents (such as robots, unmanned aerial vehicles, robot dogs and the like) in the operation area, so that the technical problem that multiple intelligent agents are difficult to cooperate is solved.
Disclosure of Invention
Based on this, it is necessary to provide a control method, a system, a computer device and a computer-readable storage medium of a teleoperation device in view of the above technical problems.
In a first aspect, the present application provides a method for controlling a teleoperation device, applied to a remote server, where the remote server is connected to at least two robots in the teleoperation device; the method comprises the following steps:
based on a distributed operating system, acquiring control information corresponding to each robot and auxiliary information of the robots; the distributed operation systems are carried on all robots, and the control information is collected through distributed soft buses of the distributed operation systems to perform end-side interconnection and interaction of the robots; the auxiliary information comprises visual auxiliary information, lighting auxiliary information and auxiliary decision information;
determining the operation information of the teleoperation equipment, and calculating the operation pose of each robot according to the operation information and the control information;
and generating a control instruction corresponding to the robots based on the working pose, and cooperatively controlling the at least two robots to move and adjust to the corresponding working pose according to the auxiliary information based on the control instruction.
In one embodiment, the robot further comprises at least one sensor, the distributed operating system being deployed with a first class thread and a second class thread; based on the distributed operating system, the method for acquiring the control information corresponding to each robot comprises the following steps:
establishing a communication connection channel between the teleoperation device and the remote server based on the first class thread;
based on the second class thread, synchronously acquiring sensing information acquired by the sensor in real time for the robot through the communication connection channel, and calculating corresponding control information according to the sensing information.
In one embodiment, the establishing a communication connection channel between the teleoperation device and the remote server based on the second class thread includes:
based on the second class thread, determining a communication connection protocol corresponding to each robot in the teleoperation equipment, and establishing a corresponding communication connection channel according to the communication connection protocol; and the remote server unifies the clock of each robot through each communication connection channel.
In one embodiment, the calculating the working pose of each robot according to the working information and the manipulation information includes:
performing data preprocessing on the control information and the operation information to obtain processed information; the data preprocessing comprises the following steps: judging and extracting abnormal data, supplementing missing data and unifying the formats of the data;
and calculating the working pose of each robot according to the processed information.
In one embodiment, the calculating the working pose of each robot according to the working information and the manipulation information includes:
constructing a working area model of each robot according to the working information; the operation area model comprises an operation area and operation contents of the robot;
and importing the control information into the operation area model, and calculating the operation pose of each robot until the teleoperation equipment finishes all operation contents.
In one embodiment, the importing the manipulation information into the operation area model, calculating the operation pose of each robot includes:
determining the operation progress of the current operation stage according to the operation information, and acquiring the operation steps to be executed in the next operation stage according to the operation content and the operation progress of the current operation stage;
based on the control information and the operation steps to be executed in the next operation stage, performing simulation calculation in the operation area model to obtain the operation pose indicating each robot to complete the operation steps to be executed in the next stage;
and under the condition that each robot is determined to move and adjust to the working pose, determining the working progress of the next working stage according to the working information, and repeating the steps according to the working progress of the next working stage to update the working pose until the teleoperation equipment is detected to finish the working content.
In one embodiment, based on the control instruction, the at least two robots are cooperatively controlled to move and adjust to corresponding working positions according to the auxiliary information, including:
the control instructions are sent to each corresponding robot in parallel, and each robot is instructed to execute the control instructions at the same time;
after receiving the control instruction, each robot executes the control instruction based on the mounted distributed operating system and adjusts cooperatively according to the auxiliary information until the robot moves and adjusts to the corresponding operation pose.
In a second aspect, the application further provides a control system of the teleoperation device. The system comprises:
the information acquisition unit is used for acquiring control information corresponding to at least two robots in the teleoperation equipment and auxiliary information of the robots based on the distributed operation system; and determining the operation information of the teleoperation device; wherein the distributed operating system is carried on all robots;
the data processing unit is used for calculating the working pose of each robot according to the working information and the control information;
and the cooperative control unit is used for generating a control instruction corresponding to the robot based on the operation pose, and cooperatively controlling the teleoperation equipment to move and adjust to the corresponding operation pose based on the control instruction according to the auxiliary information.
In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor which when executing the computer program performs the steps of:
based on a distributed operating system, acquiring control information corresponding to each robot and auxiliary information of the robots; wherein the distributed operating system is carried on all robots;
determining the operation information of the teleoperation equipment, and calculating the operation pose of each robot according to the operation information and the control information;
and generating a control instruction corresponding to the robots based on the working pose, and cooperatively controlling the at least two robots to move and adjust to the corresponding working pose according to the auxiliary information based on the control instruction.
In a fourth aspect, the present application also provides a computer-readable storage medium. The computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:
based on a distributed operating system, acquiring control information corresponding to each robot and auxiliary information of the robots; wherein the distributed operating system is carried on all robots;
determining the operation information of the teleoperation equipment, and calculating the operation pose of each robot according to the operation information and the control information;
and generating a control instruction corresponding to the robots based on the working pose, and cooperatively controlling the at least two robots to move and adjust to the corresponding working pose according to the auxiliary information based on the control instruction.
According to the control method, the control system, the computer equipment and the storage medium of the teleoperation equipment, the control information corresponding to each robot is acquired based on the distributed operation system, so that the problem of disordered data transmission sequence of different robot terminals is solved; and the at least two robots are cooperatively controlled to move respectively and adjust to corresponding operation positions based on the control instruction, so that the cooperative intelligent control of different robots based on a distributed operating system and a remote server is realized, the working efficiency and the working quality of the operation are improved, and the technical problem of cooperative matching of multiple intelligent agents is solved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application. It is evident that the drawings in the following description are only some embodiments of the present application and that other drawings may be obtained from these drawings without inventive effort for a person of ordinary skill in the art.
FIG. 1 shows an application environment diagram of a method of controlling a teleoperated device in one embodiment of the present application;
FIG. 2 shows a flow chart of a method of controlling a teleoperated device in accordance with one embodiment of the present application;
FIG. 3 shows a schematic diagram of a unified clock in one embodiment of the application;
FIG. 4 shows a schematic diagram of uniform scheduling of communication resources in one embodiment of the application;
FIG. 5 is a flow chart of a method of controlling a teleoperated device in accordance with a preferred embodiment of the present application;
FIG. 6 shows a block diagram of a control system for a teleoperated device in one embodiment of the application;
FIG. 7 shows an internal block diagram of a computer device in one embodiment of the application.
Detailed Description
In order that the above objects, features and advantages of the application will be readily understood, a more particular description of the application will be rendered by reference to the appended drawings. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. The present application may be embodied in many other forms than described herein and similarly modified by those skilled in the art without departing from the spirit of the application, whereby the application is not limited to the specific embodiments disclosed below. It should be noted that variations and modifications could be made by those skilled in the art without departing from the inventive concept. All other embodiments obtained without inventive effort fall within the scope of protection of the present application.
The control method of the teleoperation device provided by the embodiment of the application can be applied to an application environment shown in figure 1. Wherein the terminal 110 communicates with the remote server 120 through a network. The terminal 110 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, internet of things devices and portable wearable devices with distributed operating systems, where the internet of things devices may be smart speakers, smart televisions, smart air conditioners, smart vehicle devices, and the like. The portable wearable device may be a smart watch, smart bracelet, headset, or the like. The remote server 120 may be implemented as a stand-alone server or as a server cluster of multiple servers. Remote server 120 is coupled to teleoperational device 130 via a communication link that may employ a variety of communication protocols, such as UWB and 5G. Teleoperated device 130 includes a variety of different types and numbers of robots, either unmanned as illustrated or robotic arms, industrial robots, underwater robots, etc. By the control method of the teleoperation device provided by the embodiment of the application and the terminal 110, the communication between the terminal 110 and the remote server 120 can be realized, the control of the teleoperation device participated by the remote server 120 is completed, and the control of the teleoperation device 130 can be monitored and adjusted in the whole process of the terminal 110.
In one embodiment, as shown in fig. 2, a method for controlling a teleoperation device is provided, and the method is applied to the remote server 120 of fig. 1 for illustration, and includes the following steps:
step S210, based on a distributed operating system, acquiring control information corresponding to each robot and auxiliary information of the robots; the distributed operation systems are carried on all robots, and the control information is collected through distributed soft buses of the distributed operation systems to perform end-side interconnection and interaction of the robots; the auxiliary information comprises visual auxiliary information, lighting auxiliary information and auxiliary decision information;
the distributed operating system comprises a distributed operating system such as hong Meng and the like, and can efficiently communicate with other devices carrying the same distributed operating system through a distributed soft bus. Data interconnection is realized by collecting the data of each sensor, and end-side ad hoc network interconnection is realized; meanwhile, end-side interaction can be carried out, and the problem that a centralized controller is needed to make a decision among sensors of the original robots is solved.
Furthermore, the distributed operating system also improves the operability and friendliness of maintenance work. The teleoperation device with the distributed operating system can realize data interconnection by simply contacting with the terminal with the distributed operating system, and the teleoperation device is controlled by the terminal. For example, the mobile phone is used for contacting the unmanned aerial vehicle, the mobile phone can display and acquire control information of the unmanned aerial vehicle, meanwhile, a control page is displayed, the mobile phone is connected with the unmanned aerial vehicle, the mobile phone controls the unmanned aerial vehicle to fly, system software is updated, and the like. The near-field operation and maintenance of teleoperation equipment are realized, and the efficiency of equipment maintenance work is improved.
The control information comprises information such as position information, movement information, angle information and the like for controlling the robot, for example, the position information is a space three-dimensional coordinate, the movement information is movement speed and acceleration, and the angle information is a rotation angle of the unmanned aerial vehicle relative to a certain preset azimuth.
Step S220, determining the operation information of the teleoperation equipment, and calculating the operation pose of each robot according to the operation information and the control information; the operation information comprises operation content, operation progress and other information for describing the operation state of the teleoperation equipment; the working pose represents two aspects of a working position and a working pose, for example, the working pose indicates the whole bending and stretching state of the mechanical arm and the rotation angle of a joint aiming at the mechanical arm.
Step S230, generating a control instruction corresponding to the robot based on the working pose, and cooperatively controlling the at least two robots to move and adjust to the corresponding working pose according to the auxiliary information based on the control instruction; the distributed operating system of the robot can identify and execute control instructions to move and adjust the robot to the corresponding working pose.
Based on the assistance information, the multi-robot enables coordinated control of functions including, but not limited to:
visual cooperative control is realized according to the visual auxiliary information; the mechanical arm collects visual auxiliary information through the camera, when the visual auxiliary information indicates that an obstacle exists nearby, or the camera vision is insufficient to clearly support next operation, the mechanical arm is cooperated with the unmanned aerial vehicle nearby at once, so that the unmanned aerial vehicle provides more clear video data at a position, and supports next operation. When the visual auxiliary information indicates that the own video data has satisfied the requirement, the cooperative control is suspended.
Auxiliary illumination synergy realized according to the illumination auxiliary information; and acquiring illumination auxiliary information, and when the illumination auxiliary information indicates that the light of a working area of the working robot is insufficient, flying the unmanned aerial vehicle near the cooperative mechanical arm to the working position and opening an unmanned aerial vehicle illuminating lamp to provide working illumination. When the illumination assist information indicates that the light is sufficient, the cooperative control is suspended.
Auxiliary decision synergy realized according to the auxiliary decision information; the mechanical arm and unmanned aerial vehicles around the mechanical arm collect auxiliary decision information including wind speed, rain sensor data and the like and are timely synchronized. And cooperatively deciding whether the current weather condition meets the operation requirement or not through the auxiliary decision information, and if the current weather condition does not meet the operation requirement, immediately stopping the operation by the mechanical arm. If the requirements are met, the job task continues to be executed.
In the control method of the teleoperation device, the control information corresponding to each robot is acquired based on the distributed operation system, so that the problem of disordered data transmission sequence of different robot terminals is solved; and the at least two robots are cooperatively controlled to move respectively and adjust to corresponding operation positions based on the control instruction, so that the cooperative intelligent control of different robots based on a distributed operating system and a remote server is realized, the working efficiency and the working quality of the operation are improved, and the technical problem of cooperative matching of multiple intelligent agents is solved.
In one embodiment, the robot further comprises at least one sensor, the distributed operating system being deployed with a first class thread and a second class thread; based on the distributed operating system, the method for acquiring the control information corresponding to each robot comprises the following steps:
establishing a communication connection channel between the teleoperation device and the remote server based on the first class thread;
based on the second class thread, synchronously acquiring sensing information acquired by the sensor in real time for the robot through the communication connection channel, and calculating corresponding control information according to the sensing information.
The sensors included in the robot may be of various types, such as acceleration sensors, gyroscopes, radars, etc., which collect different sensing information (movement speed and acceleration information, rotation angle information, position information), respectively; the first class of threads establish a communication connection channel between the teleoperational device and the remote server by opening a wireless communication function of the teleoperational device.
In one embodiment, as shown in fig. 3 and 4, the establishing a communication connection channel between the teleoperation device and the remote server based on the second class thread includes:
based on the second class thread, determining a communication connection protocol corresponding to each robot in the teleoperation equipment, and establishing a corresponding communication connection channel according to the communication connection protocol; and the remote server unifies the clock of each robot through each communication connection channel.
The communication connection protocol comprises wireless communication protocols such as UWB, 5G, zigBee and the like, after corresponding communication connection channels are established, the remote server realizes unified management of heterogeneous communication resources through the second class threads, schedules communication time and frequency spectrum of the heterogeneous communication resources, achieves the purposes of reducing interference and improving speed, and realizes concurrent transmission of multiple information. For example, as shown in fig. 4, the communication resources may be uniformly managed and scheduled using heterogeneous networking technology.
Because of the differences in hardware and software performance of the individual teleoperational devices, time-out of sync results. And the clocks of the robots are unified through the communication connection channels, so that the unfolding operation and business of each robot can be coordinated.
In one embodiment, the calculating the working pose of each robot according to the working information and the manipulation information includes:
performing data preprocessing on the control information and the operation information to obtain processed information; the data preprocessing comprises the following steps: judging and extracting abnormal data, supplementing missing data and unifying the formats of the data;
and calculating the working pose of each robot according to the processed information.
Through data preprocessing, the calculation speed of the pose of the subsequent operation can be improved, and the working efficiency and the working quality of the operation are finally improved.
In one embodiment, the calculating the working pose of each robot according to the working information and the manipulation information includes:
constructing a working area model of each robot according to the working information; the operation area model comprises an operation area and operation contents of the robot; for example, an area map model is constructed according to the area size outline characteristics of the working area, and the working content executed by the robot in the area map model is constructed; in particular to unmanned aerial vehicles, a map model of a flight area can be provided, wherein the operation content is photography.
And importing the control information into the operation area model, and calculating the operation pose of each robot until the teleoperation equipment finishes all operation contents. The modeling can better adapt to the requirements of practical application, and the accuracy of the operation pose is ensured. For example, the work pose indicates that the unmanned aerial vehicle is launched from a certain point in the flight area, and flies to a preset place at a certain angle, speed and acceleration.
In one embodiment, the importing the manipulation information into the operation area model, calculating the operation pose of each robot includes:
determining the operation progress of the current operation stage according to the operation information, and acquiring the operation steps to be executed in the next operation stage according to the operation content and the operation progress of the current operation stage;
based on the control information and the operation steps to be executed in the next operation stage, performing simulation calculation in the operation area model to obtain the operation pose indicating each robot to complete the operation steps to be executed in the next stage;
and under the condition that each robot is determined to move and adjust to the working pose, determining the working progress of the next working stage according to the working information, and repeating the steps according to the working progress of the next working stage to update the working pose until the teleoperation equipment is detected to finish the working content.
Under the condition that the work of the current stage is completed, based on the current working process and the working steps to be executed of the next stage, the method for updating the position and cooperatively controlling the different working stages in the total working content in real time is realized by circularly calculating the working position and the position, and the accuracy of controlling the teleoperation equipment is improved.
In one embodiment, cooperatively controlling the at least two robots to move and adjust to the corresponding working positions according to the auxiliary information instruction includes:
the control instructions are sent to each corresponding robot in parallel, and each robot is instructed to execute the control instructions at the same time;
after receiving the control instruction, each robot executes the control instruction based on the mounted distributed operating system and adjusts cooperatively according to the auxiliary information until the robot moves and adjusts to the corresponding operation pose.
In a preferred embodiment, as shown in fig. 5, there is provided a control method of a teleoperated device, comprising the steps of:
step S510, based on the multithreading technology of the hong Mongolian system, acquiring the operation position, operation content, movement speed, angle and position information of the mechanical arm of the robot, and acquiring the operation position, form speed, angle and position information of the unmanned aerial vehicle.
Wherein the multithreading technique comprises: first thread: calculating the moving speeds of the unmanned aerial vehicle and the robot in real time through the accelerometer; a second thread: monitoring data through a gyroscope and calculating to obtain angles of the unmanned aerial vehicle and the robot; third thread: and acquiring the position information of the unmanned aerial vehicle, the robot and the mechanical arm through a positioning technology.
Step S520, data preprocessing is performed on the acquired information.
And step S530, starting a wireless communication module through a fourth thread of the multithreading technology, generating communication connection channels of the unmanned aerial vehicle, the cloud server, the robot and the remote server, and connecting the unmanned aerial vehicle and the robot with the remote server in a communication way.
In step S540, the remote server dynamically calculates the control command of the unmanned aerial vehicle and the robot by using the written cooperative control algorithm based on the above information.
Wherein, the cooperative control algorithm comprises:
and in the preset area, initial data, such as position information, of the robot and the unmanned aerial vehicle are acquired. And constructing an area map model according to the area size outline characteristics of the preset area. And acquiring a next working step based on the working content and the current working process.
And importing the robot/unmanned aerial vehicle position and the next stage of working content information into a regional map model for analysis. Based on the working content of the next stage, the working position of the robot of the next stage is analyzed, and the optimal auxiliary angle of the work is analyzed. And carrying out next-stage position analysis of the unmanned aerial vehicle based on the next-stage working content and the real-time image information again to obtain a control command for the unmanned aerial vehicle to reach the optimal auxiliary position and angle.
In step S550, the remote server sends the control command to the robot and the unmanned plane, and finally all the devices accept and cooperatively execute the control command.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the application also provides a control system of a teleoperation device for realizing the control method of the teleoperation device. The implementation of the solution provided by the system is similar to the implementation described in the above method, so the specific limitation in the embodiment of the control system of one or more teleoperation devices provided below may refer to the limitation of the control method of one teleoperation device hereinabove, and will not be repeated here.
In one embodiment, as shown in fig. 6, there is provided a control system 600 for a teleoperated device, comprising: an information acquisition unit 610, a data processing unit 620, and a cooperative control unit 630, wherein:
an information obtaining unit 610, configured to obtain, based on a distributed operating system, control information corresponding to at least two robots in a teleoperation device, and auxiliary information of the robots; and determining the operation information of the teleoperation device; wherein the distributed operating system is carried on all robots;
a data processing unit 620 configured to calculate a working pose of each robot according to the working information and the manipulation information;
and the cooperative control unit 630 is configured to generate a control instruction corresponding to the robot based on the working pose, and cooperatively control the teleoperation device to move and adjust to the corresponding working pose according to the auxiliary information based on the control instruction.
In one embodiment, the robot further comprises at least one sensor, the distributed operating system being deployed with a first class thread and a second class thread; the information acquisition unit 610 is further configured to: establishing a communication connection channel between the teleoperation device and the remote server based on the first class thread;
based on the second class thread, synchronously acquiring sensing information acquired by the sensor in real time for the robot through the communication connection channel, and calculating corresponding control information according to the sensing information.
In one embodiment, the information obtaining unit 610 is further configured to: based on the second class thread, determining a communication connection protocol corresponding to each robot in the teleoperation equipment, and establishing a corresponding communication connection channel according to the communication connection protocol; and the remote server unifies the clock of each robot through each communication connection channel.
In one embodiment, the data processing unit 620 is further configured to: performing data preprocessing on the control information and the operation information to obtain processed information; the data preprocessing comprises the following steps: judging and extracting abnormal data, supplementing missing data and unifying the formats of the data;
and calculating the working pose of each robot according to the processed information.
In one embodiment, the data processing unit 620 is further configured to: constructing a working area model of each robot according to the working information; the operation area model comprises an operation area and operation contents of the robot;
and importing the control information into the operation area model, and calculating the operation pose of each robot until the teleoperation equipment finishes all operation contents.
In one embodiment, the data processing unit 620 is further configured to: determining the operation progress of the current operation stage according to the operation information, and acquiring the operation steps to be executed in the next operation stage according to the operation content and the operation progress of the current operation stage;
based on the control information and the operation steps to be executed in the next operation stage, performing simulation calculation in the operation area model to obtain the operation pose indicating each robot to complete the operation steps to be executed in the next stage;
and under the condition that each robot is determined to move and adjust to the working pose, determining the working progress of the next working stage according to the working information, and repeating the steps according to the working progress of the next working stage to update the working pose until the teleoperation equipment is detected to finish the working content.
In one embodiment, the cooperative control unit 630 is further configured to: the control instructions are sent to each corresponding robot in parallel, and each robot is instructed to execute the control instructions at the same time;
and after receiving the control instruction, each robot executes the control instruction based on the mounted distributed operating system until the robot moves and adjusts to the corresponding operation pose.
The various units in the control system of a teleoperated device described above may be implemented in whole or in part by software, hardware, and combinations thereof. The units can be embedded in hardware or independent of a processor in the computer equipment, and can also be stored in a memory in the computer equipment in a software mode, so that the processor can call and execute the operations corresponding to the units.
In one embodiment, a computer device is provided, which may be a terminal, and the internal structure of which may be as shown in fig. 7. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program, when executed by a processor, implements a method of controlling a teleoperated device. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by those skilled in the art that the structure shown in FIG. 7 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
In one embodiment, a computer device is provided comprising a memory and a processor, the memory having stored therein a computer program, the processor when executing the computer program performing the steps of:
based on a distributed operating system, acquiring control information corresponding to each robot and auxiliary information of the robots; the distributed operation systems are carried on all robots, and the control information is collected through distributed soft buses of the distributed operation systems to perform end-side interconnection and interaction of the robots; the auxiliary information comprises visual auxiliary information, lighting auxiliary information and auxiliary decision information;
determining the operation information of the teleoperation equipment, and calculating the operation pose of each robot according to the operation information and the control information;
and generating a control instruction corresponding to the robots based on the working pose, and cooperatively controlling the at least two robots to move and adjust to the corresponding working pose according to the auxiliary information based on the control instruction.
It should be noted that, the specific examples in this embodiment may refer to the examples described in the foregoing embodiments and the optional implementation manners, and the description of this embodiment is omitted again.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of:
based on a distributed operating system, acquiring control information corresponding to each robot and auxiliary information of the robots; the distributed operation systems are carried on all robots, and the control information is collected through distributed soft buses of the distributed operation systems to perform end-side interconnection and interaction of the robots; the auxiliary information comprises visual auxiliary information, lighting auxiliary information and auxiliary decision information;
determining the operation information of the teleoperation equipment, and calculating the operation pose of each robot according to the operation information and the control information;
and generating a control instruction corresponding to the robots based on the working pose, and cooperatively controlling the at least two robots to move and adjust to the corresponding working pose according to the auxiliary information based on the control instruction.
It should be noted that, the specific examples in this embodiment may refer to the examples described in the foregoing embodiments and the optional implementation manners, and the description of this embodiment is omitted again.
The user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or sufficiently authorized by each party.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, or the like, but is not limited thereto.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples illustrate only a few embodiments of the application and are described in detail herein without thereby limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.
Claims (10)
1. A control method of teleoperation equipment, characterized by being applied to a remote server, wherein the remote server is connected with at least two robots in the teleoperation equipment; the method comprises the following steps:
based on a distributed operating system, acquiring control information corresponding to each robot and auxiliary information of the robots; the distributed operation systems are carried on all robots, and the control information is collected through distributed soft buses of the distributed operation systems to perform end-side interconnection and interaction of the robots; the auxiliary information comprises visual auxiliary information, lighting auxiliary information and auxiliary decision information;
determining the operation information of the teleoperation equipment, and calculating the operation pose of each robot according to the operation information and the control information;
and generating a control instruction corresponding to the robots based on the working pose, and cooperatively controlling the at least two robots to move and adjust to the corresponding working pose according to the auxiliary information based on the control instruction.
2. The method of claim 1, wherein the robot further comprises at least one sensor, the distributed operating system being deployed with a first class thread and a second class thread; based on the distributed operating system, the method for acquiring the control information corresponding to each robot comprises the following steps:
establishing a communication connection channel between the teleoperation device and the remote server based on the first class thread;
based on the second class thread, synchronously acquiring sensing information acquired by the sensor in real time for the robot through the communication connection channel, and calculating corresponding control information according to the sensing information.
3. The method of claim 2, wherein the establishing a communication connection channel between the teleoperational device and the remote server based on the second class thread comprises:
based on the second class thread, determining a communication connection protocol corresponding to each robot in the teleoperation equipment, and establishing a corresponding communication connection channel according to the communication connection protocol; and the remote server unifies the clock of each robot through each communication connection channel.
4. The method of claim 1, wherein calculating the working pose of each robot from the working information and the manipulation information comprises:
performing data preprocessing on the control information and the operation information to obtain processed information; the data preprocessing comprises the following steps: judging and extracting abnormal data, supplementing missing data and unifying the formats of the data;
and calculating the working pose of each robot according to the processed information.
5. The method of claim 1, wherein calculating the working pose of each robot from the working information and the manipulation information comprises:
constructing a working area model of each robot according to the working information; the operation area model comprises an operation area and operation contents of the robot;
and importing the control information into the operation area model, and calculating the operation pose of each robot until the teleoperation equipment finishes all operation contents.
6. The method of claim 5, wherein the importing the manipulation information into the work area model calculates a work pose of each robot, comprising:
determining the operation progress of the current operation stage according to the operation information, and acquiring the operation steps to be executed in the next operation stage according to the operation content and the operation progress of the current operation stage;
based on the control information and the operation steps to be executed in the next operation stage, performing simulation calculation in the operation area model to obtain the operation pose indicating each robot to complete the operation steps to be executed in the next stage;
and under the condition that each robot is determined to move and adjust to the working pose, determining the working progress of the next working stage according to the working information, and repeating the steps according to the working progress of the next working stage to update the working pose until the teleoperation equipment is detected to finish the working content.
7. The method according to any one of claims 1 to 6, wherein cooperatively controlling the at least two robots to move and adjust to the corresponding working positions, respectively, based on the control instructions, according to the auxiliary information, comprises:
the control instructions are sent to each corresponding robot in parallel, and each robot is instructed to execute the control instructions at the same time;
after receiving the control instruction, each robot executes the control instruction based on the mounted distributed operating system and adjusts cooperatively according to the auxiliary information until the robot moves and adjusts to the corresponding operation pose.
8. A control system for a teleoperated device, the system comprising:
the information acquisition unit is used for acquiring control information corresponding to at least two robots in the teleoperation equipment and auxiliary information of the robots based on the distributed operation system; and determining the operation information of the teleoperation device; wherein the distributed operating system is carried on all robots;
the data processing unit is used for calculating the working pose of each robot according to the working information and the control information;
and the cooperative control unit is used for generating a control instruction corresponding to the robot based on the operation pose, and cooperatively controlling the teleoperation equipment to move and adjust to the corresponding operation pose based on the control instruction according to the auxiliary information.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 7 when the computer program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311455204.XA CN117226847B (en) | 2023-11-02 | Control method and system of teleoperation equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311455204.XA CN117226847B (en) | 2023-11-02 | Control method and system of teleoperation equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117226847A true CN117226847A (en) | 2023-12-15 |
| CN117226847B CN117226847B (en) | 2024-05-28 |
Family
ID=
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE69604249D1 (en) * | 1995-07-20 | 1999-10-21 | Raytheon Co | Fault-tolerant distributed control system |
| CN101286058A (en) * | 2008-04-24 | 2008-10-15 | 上海交通大学 | Robot modularized distribution type adaptive control system and method |
| CN101359225A (en) * | 2008-08-29 | 2009-02-04 | 北京大学 | Cooperation control system for underwater multi-robot |
| CN109426273A (en) * | 2017-08-29 | 2019-03-05 | 深圳市道通智能航空技术有限公司 | A kind of flying vehicles control method and device |
| CN110766983A (en) * | 2019-11-01 | 2020-02-07 | 深圳市科卫泰实业发展有限公司 | Distributed unmanned aerial vehicle system command and control system |
| CN111712773A (en) * | 2019-07-09 | 2020-09-25 | 深圳市大疆创新科技有限公司 | Control method, electronic equipment and system for cooperative work of unmanned aerial vehicle |
| CN113021330A (en) * | 2019-12-24 | 2021-06-25 | 沈阳智能机器人创新中心有限公司 | Multi-robot synchronous follow-up control method under distributed network |
| CN113510720A (en) * | 2021-06-16 | 2021-10-19 | 清华大学 | Real-time distributed cooperative robot control system |
| CN114536339A (en) * | 2022-03-03 | 2022-05-27 | 深圳市大族机器人有限公司 | Method and device for controlling cooperative robot, cooperative robot and storage medium |
| CN115302504A (en) * | 2022-07-18 | 2022-11-08 | 北京空间飞行器总体设计部 | Support shoulder wrist to exchange distributed control system of portable arm |
| EP4191361A1 (en) * | 2020-10-15 | 2023-06-07 | I-On Communications Co., Ltd. | Multi-robot control system and method |
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE69604249D1 (en) * | 1995-07-20 | 1999-10-21 | Raytheon Co | Fault-tolerant distributed control system |
| CN101286058A (en) * | 2008-04-24 | 2008-10-15 | 上海交通大学 | Robot modularized distribution type adaptive control system and method |
| CN101359225A (en) * | 2008-08-29 | 2009-02-04 | 北京大学 | Cooperation control system for underwater multi-robot |
| CN109426273A (en) * | 2017-08-29 | 2019-03-05 | 深圳市道通智能航空技术有限公司 | A kind of flying vehicles control method and device |
| CN111712773A (en) * | 2019-07-09 | 2020-09-25 | 深圳市大疆创新科技有限公司 | Control method, electronic equipment and system for cooperative work of unmanned aerial vehicle |
| CN110766983A (en) * | 2019-11-01 | 2020-02-07 | 深圳市科卫泰实业发展有限公司 | Distributed unmanned aerial vehicle system command and control system |
| CN113021330A (en) * | 2019-12-24 | 2021-06-25 | 沈阳智能机器人创新中心有限公司 | Multi-robot synchronous follow-up control method under distributed network |
| EP4191361A1 (en) * | 2020-10-15 | 2023-06-07 | I-On Communications Co., Ltd. | Multi-robot control system and method |
| CN113510720A (en) * | 2021-06-16 | 2021-10-19 | 清华大学 | Real-time distributed cooperative robot control system |
| CN114536339A (en) * | 2022-03-03 | 2022-05-27 | 深圳市大族机器人有限公司 | Method and device for controlling cooperative robot, cooperative robot and storage medium |
| CN115302504A (en) * | 2022-07-18 | 2022-11-08 | 北京空间飞行器总体设计部 | Support shoulder wrist to exchange distributed control system of portable arm |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10168690B2 (en) | Methods and systems for multirobotic management | |
| CN106951926B (en) | Deep learning method and device of hybrid architecture | |
| US9399290B2 (en) | Enhancing sensor data by coordinating and/or correlating data attributes | |
| US11012814B2 (en) | Monitoring system control method and electronic device for supporting same | |
| Luan et al. | The paradigm of digital twin communications | |
| JP2020038699A (en) | Method for generating general-purpose learned model | |
| US20210264188A1 (en) | Image prediction system | |
| US20200272308A1 (en) | Shake Event Detection System | |
| US20190205786A1 (en) | Method and system for classifying time-series data | |
| CN113742366B (en) | Data processing method, device, computer equipment and storage medium | |
| CN115204375A (en) | NPU, edge device and operation method thereof | |
| JP2022013648A (en) | Method and device for processing image, electronic apparatus, storage medium, and computer program | |
| CN108415386A (en) | Augmented reality system and its working method for intelligent workshop | |
| CN112462348A (en) | Method and device for amplifying laser point cloud data and storage medium | |
| CN116127821A (en) | Three-dimensional visual presentation method and platform for operation and maintenance data | |
| CN111066058A (en) | System and method for low power real-time object detection | |
| CN117226847B (en) | Control method and system of teleoperation equipment | |
| EP2941718A2 (en) | Monitor-mine-manage cycle | |
| CN112527104A (en) | Method, device and equipment for determining parameters and storage medium | |
| US11604959B2 (en) | Artificial intelligence-based apparatus and method for providing wake-up time and bed time information | |
| CN117226847A (en) | Control method and system of teleoperation equipment | |
| CN113703704B (en) | Interface display method, head-mounted display device, and computer-readable medium | |
| CN115907041A (en) | Model training method and device | |
| WO2020207294A1 (en) | Service processing method and apparatus, and storage medium and electronic device | |
| CN112905328A (en) | Task processing method and device and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |