WO2020151468A1 - Vehicle remote driving system established by primary and secondary wireless devices by means of internet of things connection - Google Patents

Vehicle remote driving system established by primary and secondary wireless devices by means of internet of things connection Download PDF

Info

Publication number
WO2020151468A1
WO2020151468A1 PCT/CN2020/000015 CN2020000015W WO2020151468A1 WO 2020151468 A1 WO2020151468 A1 WO 2020151468A1 CN 2020000015 W CN2020000015 W CN 2020000015W WO 2020151468 A1 WO2020151468 A1 WO 2020151468A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
processor
arm
fault
manipulator
Prior art date
Application number
PCT/CN2020/000015
Other languages
French (fr)
Chinese (zh)
Inventor
韩磊
韩宛蕙
Original Assignee
岳秀兰
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 岳秀兰 filed Critical 岳秀兰
Publication of WO2020151468A1 publication Critical patent/WO2020151468A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04W—WIRELESS COMMUNICATION NETWORKS
    • H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
    • H04W4/30—Services specially adapted for particular environments, situations or purposes
    • H04W4/40—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P]
    • H04W4/44—Services specially adapted for particular environments, situations or purposes for vehicles, e.g. vehicle-to-pedestrians [V2P] for communication between vehicles and infrastructures, e.g. vehicle-to-cloud [V2C] or vehicle-to-home [V2H]
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/0011—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot associated with a remote control arrangement
    • G05D1/0022—Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot associated with a remote control arrangement characterised by the communication link
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network arrangements or protocols for supporting network services or applications
    • H04L67/01—Protocols
    • H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
    • H04L67/125—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks involving control of end-device applications over a network

Definitions

  • the invention relates to a primary wireless device and a secondary wireless device connected through the Internet of Things to establish a remote driving system for a manned remotely driven vehicle, in particular to the technical field in which a remote operator uses a computer terminal to remotely drive a networked vehicle through the Internet of Things.
  • GPS is used to provide accurate map information, which is currently obtained through manual driving; cameras and radars are used to perceive the environment around the car, identify lane lines, signal lights, and surrounding obstacles; high-performance processors are used to process GPS, cameras, and radars Wait for the collected information and issue instructions to executive components such as the accelerator, brake, and steering wheel.
  • This kind of automatic driving system is costly and requires the establishment of powerful and accurate map data.
  • the map cannot record buildings and road renovations in time, cannot make judgments when encountering construction and accidents, and cannot recognize traffic police gestures and language.
  • Machines are good at simple, definite and repetitive actions, but are not good at handling complex, uncertain, and changeable actions. In the short term, machines cannot reach the perception and judgment capabilities of humans, and cannot independently respond to special situations encountered during driving.
  • the present invention provides a remote operator using a computer terminal to conduct Internet-connected vehicles through the Internet of Things.
  • the solution of the remote driving system is: the remote operator 171 controls the robot 170 to drive the vehicle 260 through the following connections, the remote console 169 is connected to the remote control center 298, and the remote control center 298 is connected to the wired and wireless LAN 295, wired and wireless
  • the wireless local area network 295 is connected to the switch 291, the switch 291 is connected to the ground network 264, the ground network 264 is connected to the wireless carrier system 262, the wireless carrier system 262 is connected to the communication network access system 278, and the communication network access system 278 is connected to the telematics unit 269 connection, telematics unit 269 is connected to vehicle bus 276, vehicle bus 276 is connected to robot 170, robot 170 is connected to first manipulator 182, robot 170 is connected to second manipulator 183,
  • remote control console 169 is connected to remote control center 298, remote control center 298 is connected to switch 291, switch 291 is connected to ground network 264, ground network 264 is connected to uplink transmission station 290, and uplink transmission station 290 It is connected to the communication satellite 289, the communication satellite 289 is connected to the communication network access system 278, the communication network access system 278 is connected to the telematics unit 269, the telematics unit 269 is connected to the robot 170, and the robot 170 is connected to the steering wheel 235. 170 is connected to the gear 400, the robot 170 is connected to the brake pedal 401, and the robot 170 is connected to the accelerator pedal 402,
  • Main radar video image transmission line the vehicle vision system 502 is connected to the processor 280, the processor 280 is connected to the communication network access system 278, the communication network access system 278 is connected to the wireless carrier system 262, and the wireless carrier system 262 is connected to the ground network 264
  • the ground network 264 is connected to the switch 291, the switch 291 is connected to the remote control center 298, the remote control center 298 is connected to the second processor 215, and the second processor 215 is connected to the visual display 255,
  • the vehicle vision system 502 is connected to the processor 280, the processor 280 is connected to the communication network access system 278, the communication network access system 278 is connected to the communication satellite 289, and the communication satellite 289 is connected to the uplink transmitting station 290 is connected, the uplink transmitting station 290 is connected to the ground network 264, the ground network 264 is connected to the switch 291, the switch 291 is connected to the remote control center 298, the remote control center 298 is connected to the second processor 215, and the second processor 215 is connected to The visual display 255 is connected.
  • the radar 110 and the video acquisition device 120 of the vehicle vision system 502 are fused by the radar video information fusion system 130, and the 443 scanning radar video information fusion system 130 transmits the image to the compression storage unit 444, and the compression storage unit 444 transmits the image to the first judgment unit 445 ,
  • the first judging unit 445 transmits the image to the compressed data generating unit 446, and the compressed data generating unit 446 transmits the compressed image to the sending module 447.
  • the sending module 447 sends the compressed image to the communication network access system 278.
  • the communication network access system 278 is transmitted to the switch switch 291 through the wireless carrier system 262 and the ground communication network 264, the switch 291 is transmitted to the remote control center 298, the remote control center 298 is transmitted to the second processor 215, and the second processor 215 is connected to the receiving module 263.
  • the processor 215 transmits the received image to the receiving module 263, the receiving module 263 transmits the received image to the compressed data scanning unit 449, the compressed data scanning unit 449 transmits to the compression logic acquisition unit 450, and the compression logic acquisition unit 450 transmits the
  • the decompression reading unit 451, the decompression reading unit 451 transmits to the second judgment unit 452, the second judgment unit 452 transmits to the original byte data recovery unit 453, and the original byte data recovery unit 453 transmits the image to the visual display 255 .
  • the invention has the beneficial effects of establishing a remote driving system for a vehicle remotely driven by a remote operator, adapting to the requirements of all road conditions, without a driver in the vehicle, realizing unmanned driving in a true sense, and providing technical support for the development of the sharing economy.
  • Figure 1 is a schematic diagram of a primary wireless device and a secondary wireless device connected through the Internet of Things;
  • FIG. 2 is a circuit diagram of the CAN bus module of the multi-protocol communication network access system of the present invention
  • FIG 3 is the third part of the circuit diagram of the processor of the multi-protocol communication network access system of the present invention.
  • FIG. 4 is a circuit diagram of the communication interface of the multi-protocol communication network access system of the present invention.
  • Figure 5 is a control principle diagram of the multi-protocol communication network access system of the present invention.
  • FIG. 6 is the first part of the circuit diagram of the processor of the multi-protocol communication network access system of the present invention.
  • Figure 8 is a circuit diagram of the RS232 signal communication chip of the multi-protocol communication network access system of the present invention.
  • Figure 11 is the third part of the RS485 signal communication circuit diagram of the multi-protocol communication network access system of the present invention.
  • FIG. 12 is a circuit diagram of the Ethernet module of the multi-protocol communication network access system of the present invention.
  • FIG. 13 is a block diagram of the radar video composite data detection and processing system of the present invention.
  • Embodiment 14 is a system structure diagram of data compression provided by Embodiment 5 of the present invention.
  • Figure 16 is a diagram of the coordinate relationship between the environment coordinate system and the pixel coordinate system
  • Figure 17 is a data compression and decompression interface information structure diagram provided by the second embodiment of the present invention.
  • Embodiment 18 is a scene diagram of data compression for wireless communication network transmission provided by Embodiment 1 of the present invention.
  • FIG. 19 is a flowchart of a data compression method provided by Embodiment 1 of the present invention.
  • Embodiment 20 is a flowchart of a method for data compression and storage provided by Embodiment 2 of the present invention.
  • FIG. 21 is a flowchart of a method for data compression with added interface information provided in the second embodiment of the present invention.
  • FIG. 22 is a flowchart of a data decompression method provided by Embodiment 3 of the present invention.
  • FIG. 23 is a flowchart of a method for decompressing and reading data according to Embodiment 4 of the present invention.
  • Fig. 24 is a system connection diagram of the dual-mode driving mode of the second and third embodiments of the present invention.
  • 25 is a logic diagram of dual-mode driving work switching logic diagram of the second embodiment of the present invention.
  • 26 is a logic diagram of dual-mode driving work switching logic diagram of the third embodiment of the present invention.
  • Figure 27 is a structural diagram of the remote control center of the present invention.
  • Figure 28 is a top view of the vehicle
  • Figure 29 is a schematic diagram of a remote driving system and a robot control system
  • Figure 30 is a schematic diagram of a remote console and an operator
  • Figure 31 is a perspective view of the robot system
  • Figure 32 is a perspective view of a remotely controlled robot in the cab of a car
  • Figure 33 is a perspective view of a robot manipulator arm controlling a car steering wheel
  • Figure 34 and Figure 35 are the robot manipulating arm switch and the robot manipulating arm;
  • Figure 36 is the use of the distal part of the cannula and instrument and the switch
  • Figure 37 is a simplified block diagram of a fully constrained inverse Jacobian master/slave speed controller
  • Figure 38 is a refinement of the simplified master/slave control in Figure 38;
  • Figure 39 is a simplified diagram of a modified master/slave controller
  • Figure 40 is a schematic diagram of a modified part of the controller
  • Figure 41 is an exemplary inverse Jacobian controller of a fully constrained master/slave robot control system
  • Figure 42, Figure 43 and Figure 44 are schematic block diagrams of a reference frame for motion control
  • Figures 45 and 46 are block diagrams of two systems of the end effector reference system and the remote center reference system;
  • Figure 47 is a block diagram of fault response, fault isolation and fault weakening in the robot system
  • Figures 48-52 are flowcharts that provide fault response, fault isolation, and fault weakening methods.
  • the remote operator 171 drives the vehicle 260 through the remote control robot 170: the remote console 169 is connected to the remote control center 298, the remote control center 298 is connected to the wired and wireless LAN 295, the wired and wireless LAN 295 is connected to the switch 291, and the switch 291 is connected to The ground network 264 is connected, the ground network 264 is connected to the wireless carrier system 262, the wireless carrier system 262 is connected to the communication network access system 278, the communication network access system 278 is connected to the telematics unit 269, and the telematics unit 269 is connected to the vehicle bus 276 connections,
  • FIG 26 is a schematic diagram of the remote control center 298.
  • Each seat in the end group formulates a plan to issue control instructions according to the corresponding authority and work requirements of the seat.
  • the control instruction data is transmitted to the network switch, and the network switch transmits the control instruction data to the main server
  • the processed data is transmitted to the graphics splicing controller.
  • the graphics splicing controller intelligently realizes the splicing and combination of various data, and finally It is displayed on the large LCD screen.
  • the graphics splicing controller intelligently realizes the splicing and combination of various data, and finally displays it on the large LCD screen.
  • the PDA controller issues control instructions, which are transmitted to the network switch, and the network switch transmits the control instruction data to the main server and the secondary server, after the logic processing of the primary server and the secondary server, and then transmits the processed data to the graphics
  • the graphic splicing controller intelligently realizes the splicing and combination of various data, and finally displays it on the large LCD screen.
  • the large-screen LCD screen can promptly display the data information of various terminals and cameras and other components, which is convenient for the personnel in each seat of the terminal to view, so that people can obtain the information and data of the vehicle 260, and then conduct appropriate coordinated operations.
  • the remote control center 298 has the number of data terminals, the number of voice terminals, graphics workstations, and PDA controllers.
  • the display control module of the large-screen display control host has 16 display control modes. Selection and switching of 16 display modes of the LCD screen; the large LCD screen is a large screen display.
  • the remote control center 298 includes: large-screen LCD display, large-screen display control host, network switch, graphics splicing controller, graphics workstation, graphics workstation group control host, server group and terminal group; network switches are respectively connected with graphics workstations and graphics splicing Controller, graphic workstation group control host, server, terminal one-to-one corresponding electrical communication connection; large-screen LCD screen is used to display the graphics, video, and audio data after the graphic splicing controller is spliced; the graphic splicing controller is used from the graphic workstation Retrieve graphics or video or audio and complete the combination and splicing work; graphics workstation group control is used to control the storage, movement, display, and deletion of graphics or video or audio in the graphics workstation; the network switch realizes the graphics workstation and graphics splicing controller ,
  • the graphics workstation group controls the corresponding data communication between the host, server, and terminal; the server is composed of a main server and a secondary server, and the terminal is composed of a data terminal and a voice terminal.
  • the main server is used to receive and control the data information of the data terminal.
  • the level server is used to receive and control the voice information of the voice terminal;
  • the large-screen display control host is also connected to a wireless receiver for electronic communication, and the wireless receiver is connected to the PDA controller through wireless communication, and the data command information sent by the data terminal passes through the network
  • the switch is transmitted to the main server, through the main server for logical operation processing, the data information and processing results are displayed on the large-screen LCD display and the LCD display of the data terminal;
  • the voice command information issued by the voice terminal is transmitted to the secondary level through the network switch Server, through the secondary server for logical operation processing, the voice information and processing results are displayed on the large-screen LCD display and the LCD display of the voice terminal;
  • the data and voice command information sent by the PDA controller are transmitted to the wireless through wireless communication Receiver, wireless receiver transmits data and voice information to the graphics splicing controller through the large-screen display control host, and processes the data, voice information and processing results through the large-screen LCD
  • the second processor 215 of the remote console 169 is composed of hardware, software, and firmware. It is executed by one unit or divided into several sub-units. Each sub-unit can further use hardware, software, and Realized by any combination of firmware, the second processor 215 can cross-connect the control logic and the controller, the second processor 215 can also be distributed as a subunit in the entire vehicle remote driving system 258, and the second processor 215 can execute A machine-readable instruction of a transitory machine-readable medium, which activates the second processor 215 to perform actions corresponding to the instruction, the second processor 215 executes various instructions input by the remote operator 171, and the second processor 215 executes The remote operator 171 uses the instructions input by the left input device 177 and the right input device 178 to activate the respective joints of the first manipulator 182 and the second manipulator 183, the second processor 215 of the remote console 169 and the visual display 255, left input The device 177 is connected to the right input device 178, the first foot pedal 214 and the
  • the robot 170 directly controls the steering wheel 235, the accelerator pedal 402, the brake pedal 401, the gear position 400, the door 403, the air conditioner 404 and the audio 405 of the car in the cab of the car.
  • the car seat 173 includes a seat back 216, an anti-submarine beam 217, an anti-submersible link mechanism 219, a fifth link 218, a sixth link 230, a seventh link 231, and a column 256.
  • the robot 170 is fixed to the car seat On the chair 173, a first manipulator 182, a second manipulator 183, a third manipulator 184, and a fourth manipulator 185 are installed on the column 256.
  • the three manipulators 184 and the fourth manipulator 185 can move up and down, and can manipulate the attached instruments 300, 301, 302, and 303.
  • the remote operator 171 grasps the left input device 177 with his left hand.
  • the left input device 177 can cause the movement of the first manipulator 182 and is connected to the steering wheel 235; grasps the right input device 178 with the right hand, and the right input device 178 can cause the second manipulator 183
  • the right input device 178 can also cause the second manipulator 183 to move and connect with the gear 400.
  • the remote operator 171 connects with the second foot base 233 with his left foot, and the remote operator 171 connects with his right foot
  • the first foot pedal 214, the first foot pedal 214 can cause the robot's third manipulator 184 to move, the third manipulator 184 can cause the accelerator pedal 402 to move, and the second foot base 233 can cause the fourth manipulator 185 to move.
  • the movement of the manipulator 185 can be connected to the brake pedal 401.
  • Figure 1 shows that the remote driving system 258 includes a vehicle 260, a wireless carrier system 262, a ground communication network 264, a computer 266, and a call center 265.
  • vehicle 260 is a motorcycle, truck, bus, sports utility vehicle (SUV), and recreational vehicle ( RV), ships, aircraft and ultra-high-speed pipeline trains.
  • Vehicle electronic equipment 263 includes telematics unit 269, microphone 270, buttons and control input 271, audio system 272, visual display 273, GPS and BDS satellite navigation module 274, multiple vehicle system modules (VSM virtual switching matrix) 275,
  • VSM virtual switching matrix multiple vehicle system modules
  • CAN Controller Area Network
  • MOST Media Oriented System Transmission
  • LIN Local Interconnect Network
  • LAN Local Area Network
  • other connections Ethernet or conform to ISO, SAE and IEEE standards and specifications
  • telematics unit 269 Installed on the vehicle 260, the wireless carrier system 262 enables the vehicle 260 to perform wireless voice and data communication with the call center 265 through wireless networking.
  • the telematics unit 269 uses radio transmission to establish a communication channel with the wireless carrier system 262.
  • the communication channel includes a voice channel and a data channel, and the communication channel can send and receive voice and data transmissions.
  • the communication network access system 278 includes: a processor, a microwave communication unit, a satellite communication unit and a mobile communication unit; wherein the processor module is adapted to receive data information of the microwave communication unit, the satellite communication unit and the mobile communication unit.
  • the telematics unit 269 uses cellular communication according to the GSM or CDMA standard, and includes a standard cellular chipset 279 for voice communication, a wireless modem for data transmission, an electronic processing device 280, multiple digital storage devices 281, and a communication network interface.
  • the access system 278 includes: a processor, a microwave communication unit, a satellite communication unit, and a mobile communication unit; wherein the processor module is adapted to receive data information of the microwave communication unit, the satellite communication unit, and the mobile communication unit.
  • the modem can be realized by software stored in the telematics unit 269 and executed by the processor 280, or the modem can be a discrete hardware component located inside or outside the telematics unit 269.
  • the modem can use any number of different standards or protocols such as EVDO, CDMA, GPRS and EDGE to operate.
  • the telematics unit 269 can be used to implement wireless networking between the vehicle and other networked devices.
  • the telematics unit 269 can be configured to perform wireless communication according to any one of one or more wireless protocols such as IEEE 422.11 protocol, WiMAX or Bluetooth.
  • the telematics unit can be configured with a static IP address or can be set to automatically receive an assigned IP address from another device on the network such as a router or from a network address server.
  • the smart phone 284 is a wireless device among networked devices that communicate with the telematics unit 269.
  • the smart phone 284 includes computer processing capabilities, a transceiver capable of communicating using a short-range wireless protocol, and a visual smart phone display 286 that includes a touch screen graphical user interface.
  • the smart phone 284 is configured to communicate using the traditional Wi-Fi protocol.
  • the smart phone 284 includes the ability to communicate via cellular communication using the wireless carrier system 262, including processing capabilities, a display 286, and the ability to communicate over short-range wireless communication links.
  • the smart phone 282 may use the WiFi direct protocol to establish a short-range wireless link.
  • the smart phone 282 includes devices that do not have cellular communication capabilities.
  • the processor 280 is a device capable of processing electronic instructions, including a microprocessor, a microcontroller, a main processor, a controller, a vehicle communication processor, and an application specific integrated circuit (ASIC). It is a dedicated processor for the telematics unit 269 , Can be shared with other vehicle systems.
  • the processor 280 executes various types of digital storage instructions, such as software or firmware programs stored in the memory 281, so that the telematics unit can provide various types of services, and the processor 280 can execute programs or process data.
  • the telematics unit 269 provides wireless communication services from the vehicle. These services include: the service provided by the satellite navigation module 274; the airbag deployment notification service provided by the combination of the collision sensor interface module and the body control module; the diagnosis report using the diagnosis module; and Information, music, webpages, movies, TV shows, video games, when the modules are implemented as VSM 275 located outside the telematics unit 269, they can use the vehicle bus 276 to exchange data and commands with the telematics unit 269 .
  • the satellite navigation module 274 can determine the location of the vehicle based on the radio signals received from the satellite 285, and provide navigation and other location-related services to the vehicle driver. Navigation information can be presented and voiced on the display 273. The location information can be provided to a call center 265 or other remote computer system, such as computer 266. The new or updated map data is downloaded from the call center 265 to the satellite navigation module 274 through the telematics unit 269. Satellite navigation includes GPS satellite navigation system and Beidou (BDS) satellite navigation system. A communication satellite 289 and an uplink transmitter station 290 are used to implement two-way communication. The program content is received by the transmitter station 290, packaged for upload, and then sent to the satellite 289. The satellite 289 broadcasts programs to users.
  • BDS Beidou
  • the two-way communication uses the satellite of the satellite 289 Telephone service to relay telephone communication between the vehicle 260 and the station 290.
  • the vehicle 260 includes a vehicle system module (VSM) 275 that is located in the vehicle and receives input from sensors and uses the sensed input to perform diagnosis, monitoring, control, and reporting.
  • VSM 275 is connected to other VSMs and to the telematics unit 269 through the vehicle bus 276, and can be programmed to run vehicle system and subsystem diagnostic tests.
  • One VSM 275 can be an engine control module (ECM), which controls all aspects of engine operation, including fuel ignition and ignition timing, and the other VSM 275 can be a power system control module, which regulates the operation of components of the vehicle’s powertrain.
  • ECM engine control module
  • a VSM 275 can be a body control module that manages the electronic components in the vehicle including the electric door locks and headlights of the vehicle.
  • the engine control module is equipped with on-board diagnostics (OBD) features, which provide information such as various sensors including vehicle emission sensors. Receive various real-time data, and provide a standardized series of diagnostic trouble codes (DTC).
  • OBD on-board diagnostics
  • the vehicle electronic equipment 263 includes multiple vehicle user interfaces, devices for providing and receiving information, including a microphone 270, buttons 271, an audio system 272, and a visual display 273.
  • the microphone 270 provides audio input to the telematics unit 269 to enable the vehicle occupant It can provide voice commands and implement hands-free calls through the wireless carrier system 262, and can connect to the vehicle-mounted automatic voice processing unit using human-machine interface (HMI) technology.
  • Button 271 allows manual user input of telematics unit 269 to initiate wireless telephone calls and provide other data, response, or control input. Separate buttons can be used to initiate emergency calls and regular service assistance calls to the call center 265.
  • the audio system 272 provides audio output to vehicle occupants.
  • the audio system 272 is connected to the vehicle bus 276 and the entertainment bus 277 and can provide AM, FM, satellite radio, CD, DVD, and other multimedia functions.
  • the visual display 273 is a graphic display.
  • the wireless carrier system 262 is a cellular telephone system that includes a cellular tower 287, a mobile switching center (MSC) 288, and any other networking components required to connect the wireless carrier system 262 with the terrestrial network 264.
  • the cell tower 287 includes transmitting and receiving antennas and base stations. Base stations from different cell towers are directly connected to the MSC 288 or connected to the MSC 288 through an intermediate device of the base station controller.
  • the cellular system 262 can implement communication technologies, including AMPS analog technologies and CDMA and GSM/GPRS digital technologies.
  • the terrestrial network 264 is a terrestrial telecommunications network that connects wired telephones and connects the wireless carrier system 262 to the call center 265.
  • the terrestrial network 264 includes the Public Switched Telephone Network (PSTN), which is used to provide hard-line telephones, packet-switched data communications, and Internet infrastructure.
  • PSTN Public Switched Telephone Network
  • the call center 265 is directly connected to the wireless carrier system 262.
  • the computer 266 is a web server accessed by the vehicle through the telematics unit 269 and the wireless carrier 262.
  • the computer 266 uploads diagnostic information from the vehicle 20 through the telematics unit 269; the computer 266 provides Internet connection, provides DNS services and acts as a network address server, which uses DHCP or other appropriate protocols to assign an IP address to the vehicle 260.
  • the call center 265 provides system back-end functions to the vehicle electronic equipment 263. These back-end functions include a switch 291, a server 283, a database 292, a remote control center 298, and an automatic voice response system (VRS) 294 that are connected together through a wired and wireless LAN 295 .
  • VRS automatic voice response system
  • the switch 291 is a private exchange (PBX) switch that routes incoming signals so that the voice transmission is usually sent to the remote control center 298 via a regular telephone and to the automatic voice response system 294 using VoIP.
  • the telephone of the remote control center 298 can also use VoIP, and the VoIP and other data communication through the switch 291 are implemented through a modem connected between the switch 291 and the network 295.
  • the data transmission is transmitted to the server 283 and the database 292 via the modem.
  • the database 292 can store account information, user authentication information, and vehicle identification. It can also transmit data via wireless system 422.11x and GPRS. It is used to connect to the manual call center 265 through the remote control center 298, the call center 265 uses the VRS 294 as an automatic instructor, and the call center 265 uses the VRS 294 to connect to the remote control center 298.
  • the vehicle bus 276 is connected to the robot 170, the robot 170 is connected to the first robot 182, the robot 170 is connected to the second robot 183, the robot 170 is connected to the third robot 184, the robot 170 is connected to the fourth robot 185, and the first robot 182 is connected to the steering wheel.
  • 235 is connected, the second manipulator 183 is connected to the steering wheel 235, the second manipulator 183 can also be connected to the gear 400, 184 is connected to the accelerator pedal 402, the fourth manipulator 185 is connected to the brake pedal 401, and the power cord of the robot 170 is connected to the vehicle 260 Connect the power cords together.
  • Backup driving system remote control station 169 and remote control center 298, remote control center 298 and switch 291, switch 291 and ground network 264, ground network 264 and uplink transmission station 290 connected, uplink transmission station 290 and communication satellite 289 connection, communication satellite 289 and communication network access system 278, communication network access system 278 and telematics unit 269, telematics unit 269 and robot 170, robot 170 and steering wheel 235, robot 170 and gear 400
  • the robot 170 is connected to the brake pedal 401 and the robot 170 is connected to the accelerator pedal 402, the remote operator 171 drives the vehicle 260;
  • the main radar video image transmission line the vehicle vision system 502 is connected to the processor 280,
  • the vehicle 260 includes a vehicle vision system 502, which is configured to capture an image in a 360° area around the vehicle.
  • the first imaging device 500 of the vehicle vision system 502 is installed behind the front windshield, the vehicle grille, the front instrument panel, and the position closer to the front edge of the vehicle, and is used to capture the image of the forward field of view (FOV) 506 of the vehicle 260 A front-view camera
  • the second imaging device 508 of the vehicle vision system 502 is installed at the rear of the vehicle and a rear-view camera used to capture the vehicle's backward field of view (FOV) 510
  • the third imaging device 512 of the vehicle vision system 502 is installed at The left side of the vehicle is used to capture the side view of the vehicle's lateral field of view (FOV) 514
  • the fourth imaging device 504 of the vehicle vision system 502 is installed on the right side of the vehicle to capture the lateral view of the vehicle (FOV) 519 Side-view camera.
  • the fifth imaging device 406 is installed on the first manipulator 182, the sixth imaging device 407 is installed on the second manipulator 183, the seventh imaging device 408 is installed on the third manipulator 184, and the eighth imaging device 409 is installed on the fourth manipulator 185.
  • a ninth imaging device 410 is installed on 256, and the imaging systems of the first imaging device to the ninth imaging device are all composed of a video acquisition device 120 and a radar 110, and the radar 110 is composed of a lidar or a millimeter wave radar.
  • the radar 110 is used to detect the target and collect the target data and environmental coordinates of the target.
  • the radar 110 adopts the FMCW system with one transmitter and two receivers and 2D-FFT data processing technology.
  • the detected target data includes the target's radial distance, radial velocity and angle information .
  • the radial distance and angle information are converted into the target's horizontal distance and vertical distance information according to the geometric relationship.
  • the horizontal distance and the vertical distance information constitute the target's environmental coordinates relative to the video capture device.
  • the target data detected by the radar will be different each time. In order to obtain more accurate target information and eliminate false targets as much as possible, it is necessary to adopt data association and target tracking technology to integrate the target information detected by the radar multiple times.
  • the radar Perform data association and adaptive filtering prediction.
  • the video trigger signal is output, which triggers the camera to perform image acquisition and target extraction, and converts the target detected by the radar into environmental coordinate data relative to the camera and transmits it to the radar video
  • the information fusion system 130 performs information fusion.
  • the video acquisition device 120 is used to collect the image information and pixel coordinates of the target after the radar achieves tracking of the target.
  • the video acquisition device 120 is composed of a camera, which acquires target characteristic data by processing the image after collecting graphic information, and transmits the pixel coordinate data of the target to the radar video information fusion system 130.
  • the radar and video acquisition equipment connected to the input end of the radar video information fusion system 130 are used to perform information fusion on the target data and image information of the target, which specifically includes the coordinate conversion of the target data collected by the radar 110 from the environment coordinates. Converted to pixel coordinates corresponding to the image, the radar 110 detects the target position and the image information or video data collected by the video acquisition device 120 for time registration, first data association and decision making, and the target fusion result is displayed on the display screen.
  • the detection and processing method of the radar video composite data detection and processing system includes the following steps:
  • the radar detects the target and collects the target data and environmental coordinates of the target.
  • the radar detects the target and processes the echo data to obtain target data.
  • the target data includes the radial distance, radial velocity and angle information of the target.
  • the radar converts the radial distance and angle information into the horizontal distance and the vertical distance of the target according to the geometric relationship through the data feature transformation, and the horizontal distance and the vertical distance of the target constitute the environmental coordinates of the target relative to the video acquisition device.
  • the radar After the radar obtains the target data of the target, it performs the second data association on the radar information.
  • the methods for the radar to perform the second data association on the target data obtained at the current moment include: track bifurcation method, nearest neighbor method, and joint probability Data Association Algorithm (JPDA).
  • JPDA joint probability Data Association Algorithm
  • the radar judges the number of targets detected by the radar. If the number of targets detected by the radar is less than the preset number threshold, and the number of targets is small or sparse, the track bifurcation method or the nearest neighbor method is used for data association, and the calculation is simple and real-time.
  • JPDA joint probabilistic data association algorithm
  • the radar performs adaptive filtering prediction on the target data acquired at the current moment, and the adaptive filtering prediction can use Kalman filter tracking to perform target tracking and prediction, and then target the target.
  • the video acquisition device collects the image information and pixel coordinates of the target.
  • the video capture device captures the image information of the target.
  • the video acquisition device performs image processing on the image information to obtain target feature data, and transmits the target feature number and pixel coordinate data to the radar video information fusion system.
  • the S3 radar video information fusion system integrates the target data and image information of the target; information fusion includes: coordinate transformation, time registration, data decision-making and first data association.
  • the radar video information fusion system converts the target data obtained by the radar from the environment coordinates to the pixel coordinates corresponding to the video information, which specifically includes; the environment coordinate system Ow-XwYwZw, whose origin is the intersection point of the video acquisition device perpendicular to the ground It is the origin Ow (can also be set at any position, generally set according to the actual situation), the Yw axis points to the horizontal front of the video captured by the video capture device, the Zw axis points upwards perpendicular to the horizontal plane, and the Xw axis lies on the horizontal plane and perpendicular to the Yw axis .
  • the environment coordinate system Ow-XwYwZw whose origin is the intersection point of the video acquisition device perpendicular to the ground It is the origin Ow (can also be set at any position, generally set according to the actual situation)
  • the Yw axis points to the horizontal front of the video captured by the video capture device
  • the Zw axis points upwards per
  • the pixel coordinate system Oo-UV, the U axis and the Y axis form an imaging plane, the imaging plane is perpendicular to the Yw axis of the environment coordinate system, the upper left corner of the imaging plane is the coordinate origin Oo, and the unit of the pixel coordinate system is a pixel.
  • the height of the video capture device is H meters above the ground, the relationship between the environmental coordinates and the pixel coordinates is as shown in formula (1):
  • u is the U-axis coordinate of the target in the pixel coordinate system
  • v is the V-axis coordinate of the target in the pixel coordinate system
  • ax and az are the equivalent focal lengths of the Xw axis and Zw axis of the video capture device
  • u0, v0 is the coordinate of the pixel center of the image information
  • xw, yw, and zw are respectively the environmental coordinate values of points within the physical range of the camera's irradiation.
  • the radar video information fusion system performs time registration on the radar target data and the image information of the video acquisition device.
  • the data refresh frequency of radar and video cameras are different. It is necessary to integrate the information of the radar detection target and the extracted information of the video target in time to ensure the synchronization of the paired data, and to play the role of complementing the advantages of radar and video.
  • the data refresh frequency of radar is faster than that of cameras, and the time registration algorithm based on least squares criterion can be used, which specifically includes: different types of sensors C and R, the sampling period of sensor C is ⁇ , and the sampling period of sensor R is T , The proportional coefficient of the sampling period is an integer n.
  • the latest target state estimation time from sensor C is recorded as (k-1) ⁇
  • the number of times R estimates the target state is n.
  • the idea of least squares time registration is to fuse the n measurements collected by the sensor R into a virtual measurement, and use it as the measurement value of the sensor R at the current moment.
  • the measurement value is fused with the measurement value of sensor C to eliminate the purpose of unsynchronized target state measurement value caused by time deviation, and eliminate the influence of time mismatch on the accuracy of multi-sensor information fusion.
  • the acquisition period of the video acquisition device is ⁇
  • the acquisition period of the radar is T
  • the scale factor of the acquisition period is an integer n
  • the latest target state estimation time of the video acquisition device is recorded as (k-1) ⁇
  • n is the number of times that the radar detects the target in one cycle of the video acquisition device
  • the n measured values collected by the radar are merged into a virtual measurement and used as the current measurement value of the radar.
  • S n [S1, S2,..., Sn] T is the collection of data of a certain target position detected by radar from (k-1) ⁇ to k ⁇ , sn corresponds to the video collection data at k ⁇ , if expressed by S1, S2,..., Sn is a column vector composed of the measured values and their derivatives after fusion, and the virtual measured value si of the radar detection data is expressed as:
  • V n [v1, v2,..., vn] T has a mean value of zero
  • the covariance matrix is:
  • the measured value of the radar at the current moment and the measured value of the video acquisition device are fused using the nearest neighbor data association method.
  • the radar video information fusion system makes data decisions on the target data of the radar and the image information of the video acquisition device, which specifically includes: the radar video information fusion system determines whether the image quality of the image information collected by the video acquisition device at the current moment is greater than the preset If yes, use the target number information extracted from the image information, if not, use the target number information extracted from the target data collected by radar.
  • the radar video information fusion system performs the first data association between the radar target data and the image information of the video acquisition device.
  • the first data association uses the nearest neighbor data association method, which specifically includes: First, set up tracking gates to limit potential
  • the tracking gate is a subspace in the tracking space.
  • the tracking gate is set with video processing or radar detection target position as the center. Its size should ensure a certain probability of correct matching. Therefore, the larger residual will be eliminated first. If the number of targets detected by the radar in the tracking gate is greater than 1, the one with the smallest residual is regarded as the target.
  • the radar video information fusion system displays the target fusion result information through the display screen; the processor 280 is connected to the communication network access system 278,
  • the communication network access system 278 in FIG. 5 includes a processor, a microwave communication unit, a satellite communication unit, and a mobile communication unit; wherein the processor module is adapted to receive data information of the microwave communication unit, the satellite communication unit, and the mobile communication unit.
  • the microwave communication unit includes: a directional antenna and a radio frequency unit; wherein the directional antenna is adapted to send the received radio frequency signal to the radio frequency unit, and the radio frequency unit is adapted to modulate the radio frequency signal and send it to the processor module for demodulation into data information; or After the data information is modulated by the processor module, it is sent through the radio frequency unit and the directional antenna.
  • the satellite communication unit includes: a transceiver and a Ka-band modem; the transceiver is connected to a UHF UHF antenna for UHF-band signals, the processor is used to convert the received UHF-band signals into Ka-band signals, and the Ka-band modem is connected to the Ka antenna Used to send converted Ka-band signals to satellites; or Ka-band modems receive Ka-band signals sent by satellites through connected Ka antennas.
  • the processor is used to convert received Ka-band signals into UHF-band signals, and the transceiver is suitable for passing The UHF UHF antenna transmits the converted UHF band signal.
  • the mobile communication unit is a 4G and 5G communication module; the processor is suitable for receiving or sending 4G and 5G signals.
  • the wired communication unit includes: a serial communication circuit, a CAN bus module, and an Ethernet module; the processor is adapted to receive the data information sent by the serial communication circuit, the CAN bus module and the Ethernet module, and then convert the above data information into a Ka-band signal and UHF frequency band signal; or extract data information from Ka frequency band signal and UHF frequency band signal, and send it out through serial communication circuit, CAN bus module, and Ethernet module.
  • the multi-protocol communication network access system combines wireless communication and wired communication, that is, the communication signal of the corresponding protocol is sent to the processor. After the processor performs the corresponding protocol conversion, it is sent through the corresponding communication method to realize the multi-protocol Conversion between.
  • the processor adopts STM32 series single-chip microcomputer.
  • the 21, 22, 25, 26, 27 and 28 pins of the STM32F10XC processor are respectively connected to the 36 of the Ethernet module.
  • 37, 32, 33, 34, 35 pins are connected for communication.
  • the CAN bus module uses the SN65HVD230 chip, the CAN bus module 1 and 4 pins are electrically connected to the processor 46, 45 pins, and the CAN bus module realizes the cascading of multiple processors to realize the The processor is expanded to meet the needs of controlling communication between multiple processors.
  • the serial communication circuit includes: a communication interface, an RS485 signal communication circuit electrically connected to the processor, and an RS232 signal communication chip; the communication interface is provided with RS485 signal communication The input terminal of the circuit and the input terminal of the RS232 signal communication chip, the input terminal of the RS485 signal communication circuit and the input terminal of the RS232 signal communication chip send data information to the processor; the processor is suitable for converting the RS232 signal into the RS485 signal.
  • the pins 9 and 10 of the communication interface are electrically connected to pins 30 and 31 of the processor.
  • the 3 and 4 pins of the communication interface are connected to the 6 and 7 pins of the RS485 signal communication circuit, and the 5 and 6 pins of the communication interface are connected to the 7 and 8 pins of the RS232 signal communication chip.
  • the pins 1, 2, and 4 of the RS485 signal communication circuit are respectively connected to the pins 14, 15, and 16 of the processor, and the pins 10 and 9 of the RS232 signal communication chip are respectively connected to the pins 12 and 13 of the processor.
  • the processor module is equipped with an information classification database; the processor module is suitable for extracting the key content of the data information, and comparing it in the information classification database, and after classifying according to the comparison result, transmitting according to the corresponding transmission mode of the classification , During classified transmission, the corresponding communication protocol to be transmitted will be loaded into the data information to meet the corresponding communication requirements, thereby realizing automatic configuration between multiple protocols.
  • the serial communication circuit also includes: a communication indication circuit electrically connected to the communication interface; the communication indication circuit is provided with a first indicator light and a second indicator light. When the RS485 signal communication circuit connected to the communication interface works normally, the first indicator light The indicator is green, and when the RS232 signal communication chip connected to the communication interface works normally, the second indicator is green.
  • the multi-protocol communication network access system also includes: a DC-DC step-down circuit; the DC-DC step-down circuit is suitable for powering and stabilizing equipment.
  • the communication network access system 278 is connected to the wireless carrier system 262, the wireless carrier system 262 is connected to the ground network 264, the ground network 264 is connected to the switch 291, the switch 291 is connected to the remote control center 298, and the remote control center 298 is connected to the second processor 215 Connected, the second processor 215 is connected to the visual display 255.
  • the vehicle vision system 502 is connected to the processor 280, the processor 280 is connected to the communication network access system 278, the communication network access system 278 is connected to the communication satellite 289, and the 289 is connected to the uplink transmitting station 290 ,
  • the uplink transmitting station 290 is connected to the ground network 264, the ground network 264 is connected to the switch 291, the switch 291 is connected to the remote control center 298, the remote control center 298 is connected to the second processor 215, and the second processor 215 is connected to the visual display 255 connections.
  • the radar 110 of the vehicle vision system 502 and the video acquisition device 120 are fused by the radar video information fusion system 130, and the image of the 443 scanning radar video information fusion system 130 is transmitted to the compression storage unit 444 and the compression storage unit 444.
  • the image is transmitted to the first judging unit 445, the first judging unit 445 transmits the image to the compressed data generating unit 446, and the compressed data generating unit 446 transmits the compressed image to the sending module 447, which sends the compressed image to the communication network interface.
  • the communication network access system 278 transmits to the switch switch 291 through the wireless carrier system 262 and the ground communication network 264, the switch 291 transmits to the remote control center 298, and the remote control center 298 transmits to the second processor 215, the second processing
  • the receiver 215 is connected to the receiving module 263, the second processor 215 transmits the received image to the receiving module 263, the receiving module 263 transmits the received image to the compressed data scanning unit 449, and the compressed data scanning unit 449 transmits to the compression logic acquiring unit 450, the compression logic acquisition unit 450 transmits to the decompression reading unit 451, the decompression reading unit 451 transmits to the second judgment unit 452, and the second judgment unit 452 transmits to the original byte data recovery unit 453, and the original byte data is restored
  • the unit 453 transmits the image to the visual display 255.
  • Figures 17-23 are data compression and decompression methods and systems.
  • Figure 17 is a data compression and decompression interface information structure diagram.
  • the interface information can include the compression type and the size of the original byte data.
  • the definition of the compression type can be represented by 0 No compression, use 1 to indicate compressed.
  • Further data compression is performed on the continuously increasing byte data in the original byte data and the discontinuously identical and discontinuously increasing byte data in the original byte data, which further reduces the redundancy of the data and improves the data transmission Efficiency: By adding interface information to the compressed data, it can ensure that the receiving module correctly completes the decompression process.
  • the transmitted data is transmitted between the wireless node 411 and the wireless node 414 through the wireless channel after data compression.
  • the wireless node 411 includes a sending module 412; the wireless node 414 includes a receiving module 413; the data compression function is deployed in On the sending module 413, the data decompression function is deployed on the receiving module 413.
  • the data compression method includes the following steps: 415. Scan the original byte data. The scanning starts from the first byte of the original byte data, and scans sequentially. According to the scanning results, the redundant components of the original byte data are determined, and then the next step of compression processing is carried out according to the characteristics of the redundant data. 416 compresses and stores the original byte data.
  • any one byte data of consecutive identical byte data is stored as another byte data. For example, when there are 3 consecutive identical byte data in the original byte data, and they are 0x05, 0x05, and 0x05, the first logical operation is to use 0x80 and the number of consecutive identical byte data 0x03.
  • the logical OR operation obtains the first logical operation value 0x83, and then the obtained first logical operation value 0x83 is stored as one byte of data, and the continuous identical byte data 0x05 is stored as another byte of data.
  • 417 Determine whether the original byte data has been scanned. If the scan is completed, go to step 418; if the scan is not completed, go back to step 415 to continue scanning. 418 generates compressed data based on the stored byte data.
  • step 416 compression processing is performed for the consecutive identical byte data in the original byte data, so that the byte data that originally occupied 3 bytes of data only occupies 2 bytes of data after data compression processing.
  • 0x83 and 0x05 are compressed data.
  • step a scan the original byte data b: if there are consecutive identical byte data in the original byte data, perform the first logical operation with the number of consecutive identical byte data to obtain the first A logical operation value, the first logical operation value is stored as one byte of data, and any byte data of the same continuous byte data is stored as another byte data c: Determine whether the original byte data has been scanned, if so, Go to step d, if not, return to step a; d: generate compressed data according to the stored byte data.
  • step 419 first determine the data characteristics in the original byte data according to step 419; if step 420, that is, when it is determined that there is continuously increasing byte data in the original byte data, step 421 can be performed, which is different from the continuous increase
  • the second logical operation is performed on the number of byte data to obtain the second logical operation value; then step 422 is executed, that is, the second logical operation value is stored as one byte of data, the first byte of continuously increasing byte data
  • the data is stored as another byte of data.
  • the second logic operation is to use 0xC0 and the number of continuously increasing byte data to perform an "OR" operation.
  • step 424 may be performed, that is, the number of discontinuously identical and discontinuously increasing byte data is carried out.
  • Three logic operations are performed to obtain the third logic operation value; then step 425 is performed, that is, the third logic operation value is stored as one byte of data, and each byte of data that is discontinuously identical and discontinuously increasing is sequentially stored as Another byte of data.
  • the third logical operation is to perform an "OR" operation using 0x00 and the number of discontinuously identical and discontinuously increasing byte data.
  • the stored byte data is obtained according to step 426, and compressed data is generated according to the stored byte data.
  • step 419 add interface information to the compressed data to generate a compressed data packet.
  • Figure 23 is a method of data decompression.
  • the method of data decompression may include the following steps: 427 scanning compressed data, the compressed data is the data obtained after the original byte data is compressed by the data compression method in the first embodiment. 428 performs a compression logic judgment operation on the n-th byte data of the compressed data to obtain a compression logic judgment value and a compression logic, where n is a natural number greater than or equal to 1. 429. Decompress and read the compressed data according to the compression logic judgment value and the compression logic.
  • the compression logic judgment value is equal to the first preset value, it is judged that there are consecutive identical byte data in the original byte data corresponding to the compression logic; the first logical number operation is performed on the nth byte data to obtain the number of data i , Where i is a natural number greater than or equal to 2, and the n+1th byte data of i compressed data is repeatedly read. 430 It is judged whether the scanning of the compressed data is completed. If the scan is completed, go to step 431; if the scan is not completed, go back to step 427 to continue scanning. 431 restores the original byte data according to the read byte data. In step 429, the original byte data is the read byte data 0x05, 0x05, 0x05.
  • step 427 on the basis of step 427, that is, on the basis of decompressing and reading the compressed data according to the compression logic judgment value and the compression logic, it is further provided that the compression logic judgment value is equal to the second preset value, That is, there is continuously increasing byte data in the original byte data corresponding to the compression logic; and the judgment value of the compression logic is not equal to the first preset value and not equal to the second preset value, that is, there is discontinuity in the original byte data corresponding to the compression logic In the case of the same and discontinuously increasing byte data, the compressed data is decompressed and read.
  • step 434 determines that the compression logic corresponds to the original byte data.
  • step 435 is further performed to perform the second logical number operation on the n-th byte data of the compressed data to obtain the data number j, where j is a natural number greater than or equal to 2; finally, according to step 436, Starting from the n+1th byte of compressed data, read j bytes of data in sequence.
  • the second preset value is 0xC0; the second logical number operation is to use 0x38 and the nth byte of data to perform an "OR" operation.
  • step 438 it can be determined that there are discontinuously identical and discontinuously increasing original byte data corresponding to the compression logic
  • step 439 is further performed to perform a third logical number operation on the n-th byte data of the compressed data to obtain the number of data k, where k is a natural number greater than or equal to 2; finally, according to step 440, Starting from the n+1th byte of compressed data, read k bytes of data in sequence. Among them, the third logical number operation is an OR operation using 0x00 and the nth byte data. Finally, according to step 441, the read byte data is obtained, and the original byte data is restored.
  • Figure 15 is a data compression system 442 including: radar 110, video acquisition device 120, radar video information fusion system 130, radar 110 is composed of lidar and millimeter wave radar, raw byte data scanning unit 443, compression storage unit 444, A judging unit 445 and compressed data generating unit 446, sending module 447, original byte data scanning unit 443, used to scan the original byte data; compression storage unit 444, used to compress and store the original byte data; The judging unit 445 is used to judge whether the original byte data has been scanned; the compressed data generating unit 446 is used to generate compressed data according to the stored byte data, and the compressed data is transmitted to the sending module 447.
  • 16 is a data decompression system 448 including a receiving module 263, a compressed data scanning unit 449, a compression logic judgment value and compression logic acquisition unit 450, a decompression reading unit 451, a second judgment unit 452, and an original byte data recovery unit 453.
  • the compressed data scanning unit 449 is used to scan compressed data; the compression logic judgment value and compression logic acquisition unit 450 is used to perform compression logic judgment operations on the nth byte of compressed data to obtain the compression logic judgment value and compression logic , Where n is a natural number greater than or equal to 1; the decompression reading unit 451 is used to decompress and read the compressed data according to the compression logic judgment value and the compression logic; the second judgment unit 452 is used to judge whether the compressed data is Scanning is complete; the original byte data recovery unit 453 is used to recover the original byte data according to the read byte data.
  • the display 255 displays the original byte data recovered by the original byte data recovery unit 453
  • the remote operator 171 uses the following system to remotely drive the vehicle 260 to walk:
  • Figures 33 to 46 are remote driving systems.
  • the distal end of the first link 139 is connected to the proximal end of the second link 137 at a joint that provides a horizontal pivot axis 138.
  • the proximal end of the third link 124 is connected to the distal end of the second link 137 at the rolling joint, so that the third link generally lies around an axis extending along the axes of both the second link and the third link.
  • the rotation or rolling at the joint 123 is performed distally after the pivot joint 125.
  • the distal end of the fourth link 136 is connected to the instrument holder 136 through a pair of pivot joints 135, 134, and the pivot joints 135, 134 together define
  • the instrument holder 121, the translation of the robot 170 manipulator arm assembly 133 or the prismatic joint 132 facilitates the axial movement of the instrument 126, enabling the instrument holder 131 to be attached to the cannula through which the instrument 126 is slidably inserted ,
  • the second instrument 126 includes additional degrees of freedom, the actuation of the second instrument 126 degrees of freedom is driven by the motor of the robot manipulator arm assembly 133, the second instrument 126 and the robot manipulator arm
  • the interface between the components 133 can be arranged closer or farther along the kinematic chain of the manipulator arm assembly 133.
  • the second instrument 126 includes a rotary joint 130 on the proximal side of the pivot point PP, which is arranged at a desired location. At this point, the distal side of the second instrument 126 allows the end effector 128 to pivot about the instrument wrist axis 129, 127.
  • the angle ⁇ between the end effector jaws 231 can be controlled independently of the position and orientation of the end effector 128.
  • the left hand-held input device 177 and the right main input device 178 are connected to and separated from the console 169 through wireless communication, the left hand-held input device 177 is connected to the second processor 215, and the right hand-held input device 178 is connected to the second processor.
  • the remote operator 171 starts to perform remote driving work after the remote console 169 activates the second processor 215, the left hand of the remote operator 171 controls the left hand input device 177, and the left hand input device 177 is controlled by the second processor 215
  • the movement of the arm end 197, the right hand of the remote operator 171 controls the right hand input device 178, the right hand input device 178 controls the movement of the arm end 197 through the second processor 215, and the arm end 197 uses the first contact in the end effector 193
  • the end 194 and the second contact end 196 are in contact with the steering wheel 235 and held tightly.
  • the steering wheel 235 can be rotated by moving the left hand input device 177 and the right hand input device 178 in opposite directions.
  • the remote operator 171 uses the The second processor 215 software controls the first manipulator 182 and the second manipulator 183 of the robot 170.
  • the remote operator 171 determines the application on the first manipulator 182 and the second manipulator 183 of the robot 170 through measurement, model estimation, measurement and modeling.
  • the first manipulator 182 and the second manipulator 183 provide tactile feedback to the remote operator 171 through the remote console 169.
  • This tactile feedback can simulate the manual manipulation of the arm end 197 for the remote operator 171 to control the steering wheel 235.
  • the reaction force corresponding to the steering wheel 235 experienced by the first manipulator 182 of the robot 170 can be simulated for the remote operator 171.
  • the first contact end 194 and the second contact end 196 of the end effector 193 pivot relative to each other so as to define a pair of end effector jaws 231.
  • the jaws 231 are actuated by squeezing the grip members of the input devices 177 and 178, and the robot 170 manipulates the first manipulator 182 and the second manipulator 183 to move the transmission assembly 195 on the upper part of the steering wheel 235, so that the shaft 187 is extended and retracted , Provide the desired movement of the end effector 193.
  • the robot 170 manipulates the first manipulator 182, the second manipulator 183, the third manipulator 184, and the fourth manipulator 185 to move at the steering wheel 235, the gear position 400, the brake 401 pedal, and the accelerator pedal 402 during remote driving.
  • the first manipulator 182 and the second manipulator 183 grip the steering wheel 235 to change the direction of the car.
  • the first manipulator 182 is connected with an instrument holder 180, the instrument holder 180 and the instrument 186 and the arm end 197
  • the instrument holder 180 is connected to the first manipulator 182 by a motorized joint.
  • the instrument holder 180 includes an instrument holder frame 188, a clamp 189 and an instrument holder bracket 190.
  • the clamp 189 is fixed to the instrument holder frame 188 At the distal end, the clamp 189 can be connected to and separated from the arm end 197, the instrument holder bracket 190 is connected with the instrument holder frame 188, and the linear translation of the instrument holder bracket 190 along the instrument holder frame 188 is determined by the second processor 215 Controlled motorized translational movement.
  • the instrument 186 includes a transmission assembly 195, an elongated shaft 187 and an end effector 193, and the transmission assembly 195 is connected to the instrument holder bracket 190.
  • the shaft 187 extends distally from the transmission assembly 195.
  • the end effector 193 is provided at the distal end of the shaft 187.
  • the shaft 187 defines a longitudinal axis 192 that coincides with the longitudinal axis of the arm end 197 and coincides with the longitudinal axis defined by the arm end 197.
  • the end effector 193 can be extended and retracted from the working space.
  • the remote operator 171 sends instructions through the second processor 215, the three-state switch 202 receives the activation signal, and the remote operator 171 uses the second processor 215 to drive the vehicle remotely.
  • the system 258 connects the robot 170 to manipulate the arm end 197 of the first manipulator 182 to grip and move away from the steering wheel 235.
  • the remote operator 171 uses the second processor 215 and the vehicle remote driving system 258 to connect the robot 170 to manipulate the arm end 197 of the second manipulator 183 Make a grip and move away from the steering wheel 235.
  • the first contact end 194 and the second contact end 196 in the end effector 193 apply force to the steering wheel 235 to rotate the steering wheel 235, release the three-state switch 202 to stop the movement of the arm end 197, and when the arm end 197 is connected to the steering wheel 235 ,
  • the remote operator 171 sends an activation second direction signal, the first direction is opposite to the second direction, the tri-state switch 202 receives the activation second direction signal, and the arm end 197 moves to the steering wheel 235.
  • 151 is a simplified controller schematic diagram of the master/slave controller 153 connecting the master input device 152 to the slave manipulator 154.
  • subscripts can be appended to these vectors to identify specific structures so that Is the position of the main input device in the associated main working space or coordinate system, and x s represents the position of the follower in the working space.
  • the velocity vector associated with the bearing vector is represented by the dot above the vector or the word "dot" between the vector and the subscript, such as xdot m of the main velocity vector, where the velocity vector is mathematically defined as the change of the bearing vector over time .
  • the controller 153 contains the inverse Jacobian speed controller, in In the case of the orientation of the master input device and the speed of the master input device, the controller 153 calculates the power command for transmission to the slave manipulator 154 to realize the movement of the slave end effector corresponding to the input device from the master speed. The controller 153 can calculate the force reflection signal applied to the main input device and from there to the hand of the remote operator 171 from the slave position x s and the slave speed.
  • the first module 159 contains an inverse Jacobian speed controller, which has an output from a calculation performed using an inverse Jacobian matrix modified according to the virtual slave path 163.
  • the vector associated with the virtual follower is usually represented by the v subscript, so that the virtual follower velocity in the joint space qdot v is integrated to provide q v , and the inverse motion module 162 is used to process q v to generate the virtual From the joint position signal x v .
  • the virtual slave position and the master input command x m are combined and processed using the forward motion 161.
  • the use of the virtual follower is helpful for smooth control and force reflection when approaching the hard limit of the system, when surpassing the soft limit of the system, etc., indicated by the first control module 159 and the second control module 160 and the control diagram
  • the structure of other components of 165 and other controllers includes data processing hardware, software, and firmware. Such a structure includes reprogrammable software and data, which is embodied in machine-readable code and stored in a tangible medium to
  • the second processor 215 for the remote console 169 uses machine-readable codes to store in a variety of different configurations, including random access memory, non-volatile memory, write-once memory, magnetic recording media, and optical recording media.
  • the second processor 215 includes one or more data processors of the remote console 169, including one or more local data processing circuits of manipulators, instruments, separate and remote processing structures and locations, and the module includes a single common processor Board, multiple individual boards, one or more of the modules are scattered on multiple boards, some of which also run some and all calculations of another module.
  • the software code of the module is written as a single integrated software code, and each module is divided into separate subroutines, or part of the code of one module is combined with some or all of the code of another module.
  • the data and processing structure includes any of a variety of centralized or distributed data processing and programming architectures.
  • the output of the controller which will often try to solve for a specific manipulator joint configuration vector q, is used to generate commands for these highly configurable slave manipulator mechanisms.
  • Manipulator linkages usually have enough degrees of freedom to occupy a series of joint states for a given end effector state.
  • These structures are sometimes referred to as having excess, extra or redundant degrees of freedom, and these terms usually cover kinematic chains in which the middle link can move without changing the orientation of the end effector.
  • the main joint controller of the first module often tries to determine or solve the virtual joint speed vector qdot v , which can be used to make the end effector accurate
  • the joint of the slave manipulator 164 is driven in a manner of following the master command x m .
  • the inverse Jacobian matrix usually does not completely define the joint vector solution.
  • the mapping from the Cartesian command xdot to the joint movement qdot is a one-to-many mapping. Because the mechanism is redundant, there are countless solutions in mathematics. , Which is represented by the subspace of inverse survival.
  • the controller uses Jacobian matrices with more columns than rows to reflect this relationship, and maps multiple joint velocities to relatively few Cartesian velocities.
  • the concept of the remote motion center 298 constrained by software is determined.
  • different modes characterized by system compliance or stiffness can be selectively implemented.
  • the estimated pivot point After calculating the estimated pivot point, different system modes on a certain range of pivot points/centers are realized.
  • the estimated pivot point can be compared with the desired pivot point to generate an error output that can be used to drive the pivot of the instrument to a desired position.
  • the estimated pivot point can be used for error detection and therefore for safety, because the estimated pivot point position The change indicates that it is separated from the steering wheel or the sensor is malfunctioning, giving the system an opportunity to take corrective actions.
  • the processor 157 includes a first controller module 157 and a second controller module 160.
  • the first module 157 can include a main joint controller and an inverse Jacobian master-slave controller.
  • the main joint controller of the first module 157 can be configured to generate the desired manipulator assembly motion in response to input from the main input device 156.
  • the manipulator linkage has a series of alternative configurations for a given end effector orientation in space. Commands for the end effector to assume a given orientation can cause a variety of different joint motions and configurations.
  • the second module 160 can be configured to help drive the manipulator assembly to the desired configuration, and manipulate the manipulator during the master-slave movement. The filter is driven toward the preferred configuration, and the second module 160 will contain configuration-related filters.
  • Both the main joint controller of the first module 157 and the configuration-related filters of the second module 160 can include filters used by the processor 157 to transfer the control authority of the linear combination of joints to the realization of one or more goals or tasks.
  • F(X) can be a filter that controls the joint to i) provide the desired end effector movement, and ii) provide the pivoting movement of the instrument shaft at the hole site.
  • the main joint controller of the first module 157 may include the filter F(X).
  • (1-F-1F)(X) can describe configuration-related subspace filters that give control actuation authority for linear combinations of joint speeds orthogonal to the goal of achieving the main joint controller.
  • This configuration-related filter can be used by the second module 160 of the controller 157 to achieve the second goal.
  • the two filters can be further subdivided into more filters corresponding to more specific tasks.
  • the filter F(X) can be divided into F1(X) and F2(X), which are used to control the end effector and control the movement of the pivot axis respectively, any of which can be selected as the highest priority task of the processor.
  • the robot processor and control technology will often utilize the primary joint controller configured for the first controller task, as well as configuration related filters that utilize the underconstrained solution generated by the primary joint controller for the first controller task.
  • the main joint controller will be described with reference to the first module, and the configuration related filters will be described with reference to the second module, which can also include additional functions and additional modules of various priorities.
  • the hardware and programming codes of the first module and the second module are fully integrated, and partly integrated and can be completely separated.
  • the controller 157 can use the functions of two modules at the same time, and can have a variety of different modes, in which one or two modules are used separately or in different ways.
  • the first module 157 can be used with little or no influence from the second module 160, and when the end effector is not driven by the robot, the second module 160 has more features during system assembly.
  • the big effect is that both modules can be active most or all of the time when the robot movement is enabled, by setting the gain of the first module to zero, by setting X s to x s.actual and by reducing the inverse Jacobian
  • the matrix rank in the controller makes it impossible to control too much and allows the configuration-related filters to have more control authority, which can reduce or eliminate the influence of the first module on the state of the manipulator component, thereby changing the mode of the processor 157 to capture Support mode.
  • the first module 157 can contain some form of Jacobian controller with a Jacobian correlation matrix.
  • the second module 160 can receive a signal from the slave manipulator 158 that indicates the orientation or speed of the follower at least in part resulting from the manual articulation of the slave manipulator linkage.
  • the second module 160 can generate a power command suitable for driving the joint of the follower in order to allow manual articulation of the slave linkage while configuring the follower in a desired joint configuration.
  • the controller can use the second module 160 to help derive power commands based on different signals bqdot o .
  • This alternative input signal to the second module 160 of the controller 157 can be used to drive the manipulator linkage to maintain or move the minimally invasive hole pivot position along the manipulator structure, thereby avoiding collisions between multiple manipulators, Thereby increasing the range of motion of the manipulator structure and avoiding singularities in order to produce the desired posture of the manipulator.
  • Fig. 41 the input from the MTM controller is used to actively control a block diagram 231 of the remote motion center (RC), arm end 197 (C), and instrument end effector (E) reference frame.
  • RC remote motion center
  • C arm end 197
  • E instrument end effector
  • the instrument end effector (E) system is actively controlled using input from the primary manipulator controller, while the secondary input device is used to control the block diagram 232 of the remote center (RC) and arm end 197 (C) system.
  • the secondary input device uses arbitrary references, not necessarily the target system (EYE system).
  • the reference frame transformation EYETREF can be directly measured or calculated from indirect measurements.
  • the signal conditioning unit combines these inputs in an appropriate common system for use by the slave manipulator controller.
  • the posture specifications of the remote center frame of reference and the arm end 197 frame of reference come from one or a combination of the following sources: (i) The MTM controller designates these frames/frames in the EYE system, namely EYE T RC and EYE T C , (ii ) The secondary device commands these postures in a convenient reference frame, namely REF T RC and REF T C (where EYE T REF can be determined), and (iii) the slave side controller assigns these postures to the base system of the slave arm ( base frame), namely W T RC and W T C.
  • 45 and 46 are schematic block diagrams of the systems 212 and 213.
  • the systems 212 and 213 are used to use the second processor 215 of the computer-aided vehicle remote driving system 258 to control the instrument end effector 193 reference frame and the remote control center 298
  • the relationship between frames of reference It is assumed that the reference frame of the arm end 197 and the reference frame of the remote control center 298 coincide.
  • the arm end 197 frame of reference and the remote control center 298 are physically constrained to move relative to the instrument end effector 193 only along the arm end 197 and the longitudinal axis of the instrument.
  • Two different strategies are adopted to control the relationship between the frame of reference (E system) of the instrument end effector 193 and the frame of reference (RC system) of the remote control center 298.
  • One strategy for actively controlling the relative distance (d) between the two frames of reference whether the E frame is fixed or moving uses input from a force/torque sensor or a three-state switch.
  • the block diagram is used to implement the control subsystem for this mode, which can be described as a'relative posture controller'.
  • slv_cart_delta S*slv_cart_vel*Ts (where Ts is the sampling time of the controller).
  • S takes the value [1, -1, 0], which depends on its commands to control the movement of the steering wheel 235, the gear 400, the brake pedal 401, and the accelerator pedal 402.
  • Figure 46 is a general block diagram of the distance between the force/torque or pressure sensor and the tip of the control arm 197.
  • the calculated estimation value can be used as input F to command incremental movement, and the signal F can be any other measured or calculated amount of force based on the user's interaction with the manipulator. It can independently control the trajectories of RC system and E system, where the control input to manage these trajectories may all come from the main manipulator.
  • the block diagram of the control subsystem of this additional strategy can be called an'independent posture controller', which can be summarized as inserted (I /O) Movement to allow lateral movement of the remote control center 298 or arm end 197 relative to the instrument tip E.
  • the remote control center 298 or arm end 197 will need to pivot about the tip while driving the instrument to compensate for the movement of E. This will allow movement of the RC and arm end 197 in the cab.
  • Figures 47-52 are methods for providing fault response, fault isolation, and fault weakening of the remote system.
  • the components of the robot 170 cooperate and interact to perform various aspects of fault reaction, fault isolation and fault weakening in the robot 170.
  • the first robot 182, the second robot 183, the third robot 184, and the fourth robot 185 each include a plurality of nodes. Each node controls multiple motors, which drive the joint and linkage mechanism in the robotic arm to affect the freedom of movement of the robotic arm, and each node also controls multiple brakes for stopping the rotation of the motor.
  • the first manipulator 182 has motors 307, 309, 311, and 313; multiple brakes 308, 310, 312, and 314, and multiple nodes 315, 316, and 317, each node 315, 316 controls a single motor/brake pair; node 317 controls Two motor/brake pairs, the sensor processing unit 318 is included to provide motor displacement sensor information to the node 317 for control purposes.
  • the second manipulator 183, the third manipulator 184, and the fourth manipulator 185 are similarly configured to the first manipulator 182. With motors, brakes and nodes.
  • Each robotic arm is operatively coupled to the arm processor.
  • the arm processor 328 is operatively coupled to the node of the first manipulator 182
  • the arm processor 325 is operatively coupled to the node of the second manipulator 183
  • the arm processor 323 is operatively coupled to the node of the third manipulator 184
  • the arm The processor 321 is operatively coupled to the node of the fourth manipulator 185.
  • Each arm processor also includes a joint position controller for converting the desired joint position of the operatively coupled robot arm into a current command for driving the motor in the operatively coupled robot arm to adjust it accordingly The joint is driven to the desired joint position.
  • the system management processor 320 is operatively coupled to the arm processors 328, 325, 323, 321; the system management processor 320 also translates the user manipulation input device associated with the robotic arm to the desired joint position although shown as a separate unit,
  • the arm processors 328, 325, 323, and 321 are also implemented as part of the system management processor 320 through program codes.
  • the arm management processor 319 is operatively coupled to the system management processor 320 and the arm processors 328, 325, 323, 321.
  • the arm management processor 319 initiates, controls, and monitors certain coordinated activities of the arm in order to save the system management processor 320 from having to do so.
  • the arm manager 319 is also implemented as a part of the system management processor 320 through program code.
  • Each of the processor and the node is configured to perform various tasks herein through any combination of hardware, firmware, and software programming. Their functions are executed by one unit or distributed among many subunits, and each subunit is implemented by any combination of hardware, firmware, and software programming.
  • the system management processor 320 is allocated as sub-units throughout the robot 170, such as the remote console 169 and the base 173 of the robot 170.
  • the system management processor 320, the arm management processor 319, and each arm processor 328, 325, 323, 321 include multiple processors to perform various processor and controller tasks and functions.
  • Each node and sensor processing unit includes a transmitter/receiver (TX/RX) pair to facilitate communication with other nodes of its robotic arm and an arm processor operatively coupled to its robotic arm.
  • TX/RX are connected to the network in a daisy chain.
  • the RX of each node receives an information packet from the TX of a neighboring node, it verifies the destination field in the packet to determine whether the packet is for its node. If the packet is used for its node, the node processes the packet. If the packet is used for another node, the node's TX transfers the received packet to the neighboring node's RX in the opposite direction to where it came from.
  • the fault response logic (FRL) circuit is provided in each robot arm, and the fault notification is quickly transmitted by hand.
  • the first manipulator 182 includes an FRL circuit coupled to each of the nodes 315, 316, and 317 of the arm processor 328 and the manipulator 315.
  • the arm processor 328 and one of the nodes 315, 316, and 317 detects a fault affecting it, the arm processor or node pulls up the FRL line 329 to quickly transmit a fault notification to other components coupled to the line 329.
  • the arm processor 328 when the arm processor 328 is about to transmit the recovery notification to the node of the first manipulator 182, it pulls down the FRL line 329 to quickly transmit the recovery notification to other components coupled to the line 329.
  • the virtual FRL line 329 is used instead by specifying that one or more fields in the packet include such failure notification and recovery notification.
  • the method detects a failure in a failed arm of a plurality of robot arms, where the robot arm becomes a "failed arm” due to the detected failure.
  • the method then puts the failed arm into a safe state, where the "safe state” refers to the state of isolating the detected malfunctioning arm by preventing further movement of the arm.
  • the method determines whether the failure should be regarded as a system failure or a partial failure, where "system failure” refers to a failure that affects the performance of at least one other of the multiple robot arms, and "partial failure” Refers to a malfunction that affects the performance of only the failed arm.
  • a local fault causes only the failed arm to be kept in a safe state until the fault is cleared, it is not a type of fault that causes unsafe operation of a non-failed mechanical arm.
  • the fault is the type that causes the unsafe operation of the non-failure arm, then the method should produce a determination that the detected fault is a system fault, in which all the robot arms in the system will be placed in a safe state.
  • the method puts the non-failed arm of the multiple arms into a safe state only when the fault will be regarded as a system failure, where the "non-failed arm" refers to the one in which no failure has been detected The robotic arm among multiple robotic arms.
  • the method determines whether the detected fault is classified as a recoverable system fault or an unrecoverable system fault.
  • the fault is classified as a recoverable system fault, and the method provides the system user with recovery options.
  • the fault is classified as an unrecoverable system fault, and the method waits for the system to shut down.
  • the determination in 329 is that the fault will be treated as a partial fault, and then in 334, the method determines whether the fault is classified as a recoverable partial fault or an unrecoverable partial fault.
  • the fault is classified as a recoverable partial fault, and the method provides the system user with recovery options and weakened operation options.
  • the fault is classified as an unrecoverable partial fault, and the method only provides a weakened operation option.
  • FIG. 47 is a flowchart of aspects of a method of performing fault reaction, fault isolation, and fault weakening, which are performed by each node 315, 316, and 317 of the multiple robot arms of the robot 170.
  • each node continuously monitors the signals and information in the node to use conventional fault detection methods to detect faults affecting the node.
  • This type of detected fault is referred to herein as a "local fault” because it is limited to nodes.
  • the node also monitors the FRL circuit for fault notifications sent by its arm processor or another node in its robotic arm.
  • This type of detected fault is referred to herein as a "remote fault” because it is not limited to the node.
  • the detected fault is hardware, firmware, software, environment, or related communications.
  • failed node The node where the fault has been detected is called “failed node” in this article, and its robotic arm is called “failed arm” in this article.
  • a node in which no fault has been detected is referred to herein as a “non-failure node”, and a mechanical arm in which no fault has been detected is referred to herein as a “non-failure arm”.
  • a failure is detected in 337, and the node puts itself into a safe state. This is done by deactivating one or more controlled motors of the node, which is done by engaging one or more controlled brakes of the node.
  • the node determines whether the detected fault is a local fault or a remote fault. As previously referred to 337, the source of the fault determines whether it will be regarded as a local fault or a remote fault. If the failure is determined to be a partial failure, the node is a failed node. In the first case, the failed node continues by executing the following 343-346 and 341-342. If the failure is determined to be a remote failure, the node is a non-failed node.
  • the non-failed node continues by performing the following 340-342.
  • the failed node transmits a failure notification to neighboring nodes in the failed robot arm in the upstream and downstream directions.
  • the "downstream” direction refers to the direction in which the packet travels away from the arm processor of the node and the "upstream” direction refers to the direction the packet travels toward the arm processor of the node.
  • One way for a node to complete the process is through Pull the FRL line to the high state.
  • the failed node then diagnoses the fault and sends an error message to the system management processor 320.
  • the error message preferably includes failure information, its error code, error type, and origin. Each type of error that may occur that affects the node is assigned an error code.
  • the error code is classified as an error class. There are at least four types of errors: recoverable arm failure, unrecoverable arm failure, recoverable system failure, and unrecoverable system failure.
  • recoverable means that the user is provided with the option to try to recover from the failure.
  • Using "unrecoverable” means that the user is not provided with the option to try to recover from the failure.
  • the origin of the fault includes information about the identity of the node and optional additional information about the source of the fault in the node.
  • the failed node determines whether the detected fault is a recoverable partial fault. The determination in 345 is no, then in 346, the failed node remains in its safe state and ignores any recovery notifications it may subsequently receive on the FRL line. If the determination in 345 is yes, the failed node proceeds to 341.
  • the determination in 339 is that the detected fault will be regarded as a remote fault, then in 340, the virtual FRL line is used, and the non-failed node transmits the received fault notification in the opposite direction from where the fault notification came from In the case of a real FRL line, the non-failure node does not need to take any action for this transmission of the failure notification.
  • both the failed node and the non-failed node wait for the recovery notification to be received.
  • the node once the recovery notification is received, the node returns itself from the safe state to its normal operating state. This is done by reversing the actions taken in 338 while avoiding sudden changes. The node returns to perform the fault detection task referred to in 337.
  • each arm processor 321, 323, 325 and 328 The each arm processor is operatively coupled to the robotic arm of the robot 170.
  • each arm processor continuously monitors its own operation while performing its normal operation tasks and pays attention to the failure notifications transmitted by the failed node in the robot arm to which it is operatively coupled.
  • the fault is referred to herein as a "local fault”.
  • a fault is detected by receiving a fault notification from a failed node in the robotic arm to which it is operatively coupled, and the fault is called a "remote fault.”
  • the remote fault is a fault notification sent along the FRL line by the failed node in the robotic arm operatively coupled to the arm processor.
  • the fault has been detected.
  • the arm processor puts its joint position controller into a safe state by locking the joint position controller's output motor current command to zero. This is used to strengthen the security status of their corresponding nodes.
  • the arm processor determines whether the detected fault is a local fault or a remote fault.
  • the source of the fault determines whether the fault will be considered a local fault or a remote fault.
  • the fault is determined to be a partial fault, and the arm processor is regarded as a failed node.
  • the arm processor continues by executing the following 353-356 and 351-352.
  • the fault is determined as a remote fault, the arm processor is regarded as a non-failure node, and the arm processor continues by executing 350-352.
  • the arm processor transmits the failure notification downstream to all nodes of the robotic arm to which it is operatively connected. One way for the arm processor to complete this process is by pulling the FRL line to a high state.
  • the arm processor diagnoses the fault and sends an error message to the system management processor 320.
  • the error message includes fault information, error code, error type, and origin.
  • Each type of error that occurs that affects the arm processor is assigned an error code.
  • the error code is classified as an error type, and there are at least four error types: recoverable processor failure, unrecoverable processor failure, recoverable system failure, and unrecoverable system failure.
  • the origin of the fault includes information on the identity of the arm processor and optional additional information on the source of the fault in the arm processor.
  • the arm processor determines whether the detected fault is a recoverable partial fault, and the determination is completed by the error category of the fault. If the determination in 355 is no, then in 356, the joint position of the failed arm processor is controlled The arm processor remains in its safe state and the arm processor ignores any recovery notifications it may subsequently receive on the FRL line. The determination in 355 is yes, then the arm processor proceeds to 350.
  • the determination in 349 is that the detected fault will be regarded as a remote fault, then in 350, the arm processor waits for the recovery notification received from the system management processor 320. In 351, once the recovery notification is received, the arm processor transmits the recovery notification to all nodes in the robotic arm to which it is operatively coupled by, for example, pulling its FRL line low. In 352, the arm processor then returns its joint position controller from the safe state to its normal operating state. This process is accomplished by releasing the output motor current command of the joint position controller so that they can once again reflect the desired joint position of the robotic arm to which they are operatively coupled while avoiding sudden changes. The arm processor then returns to perform its fault detection task with reference to 347.
  • 47 and 51 are flowcharts of various aspects of methods for performing fault reaction, fault isolation, and fault weakening, which are executed by the system management processor 320 of the robot 170.
  • the system management processor also waits to receive an error message transmitted from another component of the robot 170 while performing its normal operation tasks.
  • the error message is received in 357, then in 358, the system management processor for safety purposes, for example, by instructing the joint position controllers of all arm processors 328, 325, 323, and 321 in the robot system to lock their respective current values To stop the system. No new current command input is provided to the robotic arm until the output of the joint position controller is unlocked. This locking of the output of the joint position controller is referred to herein as a "soft lock" joint position controller.
  • the system management processor determines whether the detected fault should be regarded as a system fault or an arm fault.
  • the system management processor completes this step by checking the error type information provided in the error message.
  • System faults include all faults classified as either recoverable system faults or unrecoverable system faults, because these faults can be applied to more than just failed robots.
  • arm faults include all faults classified as either recoverable partial faults or unrecoverable partial faults, because these faults can only be applied to a failed robot arm.
  • the system management processor provides the remote operator 171 of the robot 170 with the option of accepting the weakened operation of the robot 170.
  • the system management processor also provides the user with an option to recover from the failure.
  • information about the detected failure is also provided by the system management processor to assist the remote operator 171 in determining whether to accept the option.
  • the option and fault information are provided on the visual display 255 of the remote console 169.
  • the system management processor waits for the remote operator 171 to select the option provided in 360. Once the option is selected by the remote operator 171, in 362, the system management processor determines whether the selected option is a weakened operation option or a recovery option. The recovery option is provided and the remote operator 171 selects the recovery option, then in 381, the system management processor sends a recovery notification to the arm processor of the failed robotic arm. The arm processor of the failed robotic arm will process the recovery notification, including sending the recovery notification to all nodes of the failed arm, and the node of the failed arm will then process the recovery notification.
  • system management processor then releases the soft locks of the joint controllers by unlocking the outputs of the joint controllers of all arm processors, so that the joint controllers again issue the desired joint position reflecting their operationally coupled robotic arms Then, the system management processor returns to perform its task with reference to 357.
  • the remote operator 171 selects the weakened operation option, then in 363, the system management processor provides the remote operator 171 with an option to recover from the failure. Recovery from the failure in this case is different from the recovery with reference 381-382 because no attempt is made to recover the failed arm. Recovery should only be used to restore normal operation of the non-failed arm.
  • the system management processor waits for the user to select the option provided in 363. Once the option is selected by the remote operator 171, in 365, the system management processor sends a message to the arm processor of the disabled arm to reinforce the failure.
  • the enhancement of the failure in this case means that additional steps are taken to completely shut down the operation of the failed robot arm.
  • An example of such an enhanced measure is to operatively disconnect the joint position controller of the arm processor from other components of the master/slave control system that generates the desired joint position of the operatively coupled robot arm.
  • One strengthening measure is to turn off the power to the failed robotic arm.
  • the system management processor then releases the soft lock of the joint controller by unlocking the output of the joint controller of the arm processor of all non-failed arms, so that the joint controller again sends out the mechanical coupling reflecting its operation.
  • the motor current at the desired joint position of the arm commands the system management processor and then returns to perform the task with reference to 357.
  • the system management processor validates the system FRL situation for all nodes in the robot 170. This is done by causing the FRL lines 329, 327, 384, and 385 to be pulled high so that the fault notification is provided to the arm processor and the nodes of the first robot 182, second robot 183, third robot 184, and fourth robot 185 at the same time. step.
  • the system management processor determines whether the system failure is a recoverable system failure. Complete this step by checking the error class in the received error message. The determination in 369 is no, then in 363, the system management processor takes no further action and waits for the system to be shut down.
  • the system management processor determines whether the system management processor provides the user with an option to recover from the failure.
  • the management processor waits for the remote operator 171 to select a recovery option. If this option is selected, in 372, the system management processor sends a recovery notification to the arm processors of all the robot arms of the robot 170.
  • the system management processor upon receiving a request or action from 171, releases the soft lock of each joint controller to operate the joint controller arm in its normal operating state, so that the released joint controller is once again Sending a motor current reflecting the desired joint position of the robotic arm to which it is operatively coupled instructs the system management processor to then return to perform the task of reference 357.
  • FIGS. 47 and 52 are flowcharts of various aspects of methods for performing fault reaction, fault isolation, and fault weakening, which are operatively coupled to the system management processor and arm processing of the robot 170
  • the arm management processors of the devices 328, 325, 323, and 321 are executed.
  • the arm management processor 319 starts, controls, and monitors certain coordinated activities of the first robot 170, the first robot 182, the second robot 183, the third robot 184, and the fourth robot 185.
  • the arm manager 319 initiates and monitors the start of the brake test, where the arm manager 319 communicates with each of the arm processors 328, 325, and 323 so that specific braking sequences with different torque values are applied to their respective machines The brake of the arm.
  • the coordination of this activity is performed by the arm manager 319 in this case, because the overhead of coding it to each arm processor is redundant.
  • the maximum torque value calculated by each arm processor is transmitted back to the arm manager 319.
  • the arm manager 319 will notify the failed arm of the transmission failure.
  • the arm manager instructs the arm processor to perform arm activity, monitors the result, and determines whether the result of the activity indicates arm failure.
  • the arm manager monitors the coordinated activities of the robotic arms based on the reports of the respective arm processors of the robotic arms to detect faults in one arm. When the reported measurement exceeds the expected value by a threshold amount, the arm manager determines that a fault has occurred.
  • the detected fault is a fault that is generally not detected by one of the nodes of the robot arm or the arm processor.
  • the arm manager suppresses any further commands of the disabled arm. No further commands will be transmitted from the arm manager to the arm processor of the failed arm until either a recovery notification is received from the system manager or the system is restarted.
  • the arm manager sends a fault notification to the failed arm by pulling the FRL line of the failed arm to a high state. In the case of a virtual FRL line, the arm manager transmits a failure notification in a packet field that is the same as or different from the packet field designated for transmission of the failure notification through one of the nodes of the failed arm or the arm processor.
  • the arm manager sends an error message to the system manager, the error message has the available details of the failure, each type of failure detected by the arm manager is assigned an error code, and the error code is classified as an error class,
  • the origin of the fault includes the identification information of the failed arm and optional additional information on the source of the fault.
  • the system manager in 357 the system manager then starts processing error messages.
  • the arm manager determines whether the detected fault is a recoverable fault. This determination is made according to the error class of the fault. The determination in 378 is no, then in 381, the arm manager continues to suppress any further commands to the disabled arm and ignores any recovery notifications subsequently received from the system manager. The determination in 378 is yes, then in 379, the arm manager waits for the recovery notification received from the management processor. In 380, the recovery notification is received, the arm manager stops suppressing the command to the failed arm and returns to its normal operation mode and performs the fault detection task with reference 374.
  • Fig. 24 is a system connection diagram of the dual-mode driving mode of the second and third embodiments of the present invention.
  • the modular design concept is adopted for the unmanned driving controller 525.
  • the unmanned driving controller 525 is connected to the vehicle vision system 502 for unmanned driving control
  • the device 525 is connected with the positioning and navigation module 274, the planning system module 528, and the unmanned driving controller 525 with the vehicle bus 276, and the control system module 529 is connected with the robot 170. Ethernet and CAN bus communication are adopted between the modules.
  • the unmanned driving controller 525 adopts a reconfigurable computing AI chip and is programmed based on the Linux system platform.
  • the control strategy is customized by integrating vehicle parameters and operating characteristics.
  • the specific control strategies include: 1.
  • the unmanned driving controller 525 is used as control Mode arbitration controller determines whether it is currently in unmanned driving mode and whether it meets the conditions of unmanned driving mode; 2. When in remote driving mode, the control system module shields the control commands issued by the unmanned driving controller and responds to remote operations Command; 3. When in the unmanned driving control mode, according to the information of lidar, millimeter-wave radar and camera received by the environment sensing module received by the unmanned controller 525, the optimized information fused by each sensor, and The received body attitude information sensed by the gyroscope and the latitude and longitude information provided by the DGPS positioning and navigation module are combined with the tracking route learned through deep learning, and adaptive route planning through parallel mapping and positioning technology (SLAM).
  • SLAM parallel mapping and positioning technology
  • Fig. 25 is a logic diagram of dual-mode driving switching in the second embodiment of the present invention.
  • the remote control driving mode can be switched to the intelligent unmanned driving mode.
  • Emergency switching mode the "remote control driving mode" signal interrupts the automatic sending request 517.
  • the unmanned driving controller judges to enter the unmanned driving mode and automatically takes over the remote driving mode, and executes 518. It is not allowed to enter the unmanned driving mode, and the feedback signal 519
  • the external unmanned driving control switch is reset or the pre-planned exit unmanned driving logic is switched to the normal driving mode, and the control instruction 520 is executed.
  • Figure 26 is a logic diagram of dual-mode driving switching logic diagram of the third embodiment of the present invention.
  • the remote control driving mode can be used to switch to the intelligent unmanned driving mode. Normal switching mode: when it is necessary to enter the "intelligent unmanned driving mode" from the "remote control driving mode” ”, press the external driverless control switch to send a request 521 to the driverless controller.
  • the driverless controller judges whether to enter the driverless mode, and allows the driver to enter the driverless mode, execute 522, and not allow Unmanned driving mode, feedback signal 523; when it is necessary to enter the “remote control driving mode” from “unmanned driving mode”, switch to normal driving by resetting the external unmanned driving control switch or pre-planned exit unmanned driving logic Mode, execute the control instruction 524.

Abstract

A vehicle remote driving system established by primary and secondary wireless devices by means of Internet of Things connection, a primary driving system, a standby driving system, a primary radar video image transmission line, and a standby radar video image transmission line. A radar 110 of a vehicle vision system 502 and a video acquisition device 120 are fused by a radar video information fusion system 130; the vehicle remote driving system for a remote operator to remotely drive is established by means of the systems above, and in particular, the remote operator remotely drives networked vehicles by means of the Internet of Things by using a computer terminal, the requirements of the whole road conditions are adapted, no driver is assigned in the vehicle, a real sense of unmanned driving is achieved, and a technical support for the development of sharing economy is provided.

Description

主次无线设备通过物联网连接建立的车辆远程驾驶体系Vehicle remote driving system established by primary and secondary wireless devices through the Internet of Things connection 技术领域Technical field
本发明涉及主无线设备和次无线设备通过物联网连接，建立有人远程驾驶的车辆远程驾驶体系，特别是由远程操作员使用计算机终端通过物联网对联网车辆进行远程驾驶的技术领域。The invention relates to a primary wireless device and a secondary wireless device connected through the Internet of Things to establish a remote driving system for a manned remotely driven vehicle, in particular to the technical field in which a remote operator uses a computer terminal to remotely drive a networked vehicle through the Internet of Things.
背景技术Background technique
因行驶环境复杂，汽车自动驾驶技术无法进入实用阶段，还处于试验阶段，当前的人工智能水平还无法达到人类感知复杂环境的水平。自动驾驶汽车主要关键部件有GPS芯片、摄像头、雷达、高性能处理器等。其中GPS用于提供精确的地图信息，目前通过人工先行驾驶获取；摄像头和雷达用于感知汽车周围的环境，识别车道线、信号灯、周围障碍物等；高性能处理器用于处理GPS、摄像头、雷达等采集的信息并向油门、刹车、方向盘等执行部件发出指令。这种自动驾驶系统成本高，需要建立强大、精确的地图数据，地图无法及时收录建筑物和道路改造、遇到施工和事故无法做出判断以及无法识别交警手势和语言等困难。机器擅长简单的、确定的和重复的动作，不善于处理复杂的、不确定的、多变的动作。短期内机器无法达到人类所具有的感知和判断能力，无法独立应对驾驶过程中遇到的特殊情况。Due to the complex driving environment, auto-driving technology for automobiles cannot enter the practical stage and is still in the experimental stage. The current level of artificial intelligence cannot reach the level of human perception of complex environments. The main key components of autonomous vehicles include GPS chips, cameras, radars, and high-performance processors. Among them, GPS is used to provide accurate map information, which is currently obtained through manual driving; cameras and radars are used to perceive the environment around the car, identify lane lines, signal lights, and surrounding obstacles; high-performance processors are used to process GPS, cameras, and radars Wait for the collected information and issue instructions to executive components such as the accelerator, brake, and steering wheel. This kind of automatic driving system is costly and requires the establishment of powerful and accurate map data. The map cannot record buildings and road renovations in time, cannot make judgments when encountering construction and accidents, and cannot recognize traffic police gestures and language. Machines are good at simple, definite and repetitive actions, but are not good at handling complex, uncertain, and changeable actions. In the short term, machines cannot reach the perception and judgment capabilities of humans, and cannot independently respond to special situations encountered during driving.
发明内容Summary of the invention
本发明为了解决目前的自动驾驶技术无法达到人类所具有的感知和判断能力，无法独立应对驾驶过程中遇到的特殊情况，提供了一种由远程操作员使用计算机终端通过物联网对联网车辆进行远程驾驶的体系的解决方案是：远程操作员171通过以下连接，控制机器人170驾驶车辆260行驶，远程控制台169与远程控制中心298连接，远程控制中心298与有线和无线局域网295连接，有线和无线局域网295与交换机291连接，交换机291与地面网络264连接，地面网络264与无线载波系统262连接，无线载波系统262与通信网络接入系统278连接，通信网络接入系统278与远程信息处理单元269连接，远程信息处理单元269与车辆总线276连接，车辆总线276与机器人170连接，机器人170与第一机械手182连接、机器人170与第二机械手183连接、机器人170与第三机械手184连接、机器人170与第四机械手185连接，第一机械手182与方向盘235连接，第二机械手183与方向盘235连接，第二机械手183还能够与档位400连接，第三机械手184与油门踏板402连接，第四机械手185与刹车踏板401连接，In order to solve the problem that the current automatic driving technology cannot reach the perception and judgment ability of human beings, and cannot independently deal with the special situations encountered during driving, the present invention provides a remote operator using a computer terminal to conduct Internet-connected vehicles through the Internet of Things. The solution of the remote driving system is: the remote operator 171 controls the robot 170 to drive the vehicle 260 through the following connections, the remote console 169 is connected to the remote control center 298, and the remote control center 298 is connected to the wired and wireless LAN 295, wired and wireless The wireless local area network 295 is connected to the switch 291, the switch 291 is connected to the ground network 264, the ground network 264 is connected to the wireless carrier system 262, the wireless carrier system 262 is connected to the communication network access system 278, and the communication network access system 278 is connected to the telematics unit 269 connection, telematics unit 269 is connected to vehicle bus 276, vehicle bus 276 is connected to robot 170, robot 170 is connected to first manipulator 182, robot 170 is connected to second manipulator 183, robot 170 is connected to third manipulator 184, robot 170 is connected to the fourth manipulator 185, the first manipulator 182 is connected to the steering wheel 235, the second manipulator 183 is connected to the steering wheel 235, the second manipulator 183 can also be connected to the gear 400, the third manipulator 184 is connected to the accelerator pedal 402, and the fourth manipulator 184 is connected to the accelerator pedal 402. The manipulator 185 is connected to the brake pedal 401,
备用驾驶系统：远程控制台169与远程控制中心298连接，远程控制中心298与交换机291连接，交换机291与地面网络264连接，地面网络264与上行链路发射站290连接，上行链路发射站290与通信卫星289连接，通信卫星289与通信网络接入系统278连接，通信网络接入系统278与远程信息处理单元269连接，远程信息处理单元269与机器人170连接，机器人170与方向盘235连接、机器人170与档位400连接、机器人170与刹车踏板401连接、机器人170与油门踏板402连接，Backup driving system: remote control console 169 is connected to remote control center 298, remote control center 298 is connected to switch 291, switch 291 is connected to ground network 264, ground network 264 is connected to uplink transmission station 290, and uplink transmission station 290 It is connected to the communication satellite 289, the communication satellite 289 is connected to the communication network access system 278, the communication network access system 278 is connected to the telematics unit 269, the telematics unit 269 is connected to the robot 170, and the robot 170 is connected to the steering wheel 235. 170 is connected to the gear 400, the robot 170 is connected to the brake pedal 401, and the robot 170 is connected to the accelerator pedal 402,
主雷达视频图像传输线路：车辆视觉系统502与处理器280连接，处理器280与通信网络接入系统278连接，通信网络接入系统278与无线载波系统262连接，无线载波系统262与地面网络264连接，地面网络264与交换机291连接，交换机291与远程控制中心298连接，远程控制中心298与第二处理器215连接，第二处理器215与视觉显示器255连接，Main radar video image transmission line: the vehicle vision system 502 is connected to the processor 280, the processor 280 is connected to the communication network access system 278, the communication network access system 278 is connected to the wireless carrier system 262, and the wireless carrier system 262 is connected to the ground network 264 The ground network 264 is connected to the switch 291, the switch 291 is connected to the remote control center 298, the remote control center 298 is connected to the second processor 215, and the second processor 215 is connected to the visual display 255,
备用雷达视频图像传输线路：车辆视觉系统502与处理器280连接，处理器280与通信网络接入系统278连接，通信网络接入系统278与通信卫星289连接，通信卫星289与上行链路发射站290连接，上行链路发射站290与地面网络264连接，地面网络264与交换机291连接，交换机291与远程控制中心298连接，远程控制中心298与第二处理器215连接，第二处理器215与视觉显示器255连接。Backup radar video image transmission line: the vehicle vision system 502 is connected to the processor 280, the processor 280 is connected to the communication network access system 278, the communication network access system 278 is connected to the communication satellite 289, and the communication satellite 289 is connected to the uplink transmitting station 290 is connected, the uplink transmitting station 290 is connected to the ground network 264, the ground network 264 is connected to the switch 291, the switch 291 is connected to the remote control center 298, the remote control center 298 is connected to the second processor 215, and the second processor 215 is connected to The visual display 255 is connected.
车辆视觉系统502的雷达110和视频采集设备120由雷达视频信息融合系统130融合，443扫描雷达视频信息融合系统130图像传输给压缩存储单元444、压缩存储单元444把图像传输给第一判断单元445，第一判断单元445把图像传输给压缩数据生成单元446，压缩数据生成单元446把压缩好的图像传输给发送模块447，由发送模块447发送给通信网络接入系统278，通信网络接入系统278通过无线载波系统262和地面通信网络264传输给交换机交换机291，交换机291传输给远程控制中心298，远程控制中心298传输给第二处理器215，第二处理器215连接接收模块263，第二处理器215把接收到的图像传输给接收模块263，接收模块263把接收到的图像传输给压缩数据扫描单元449，压缩数据扫描单元449传输给压缩逻辑获取单元450，压缩逻辑获取单元450传输给解压缩读取单元451，解压缩读取单元451传输给第二判断单元452，第二判断单元452传输给原始字节数据恢复单元453，原始字节数据恢复单元453把图像传输给视觉显示器255。The radar 110 and the video acquisition device 120 of the vehicle vision system 502 are fused by the radar video information fusion system 130, and the 443 scanning radar video information fusion system 130 transmits the image to the compression storage unit 444, and the compression storage unit 444 transmits the image to the first judgment unit 445 , The first judging unit 445 transmits the image to the compressed data generating unit 446, and the compressed data generating unit 446 transmits the compressed image to the sending module 447. The sending module 447 sends the compressed image to the communication network access system 278. The communication network access system 278 is transmitted to the switch switch 291 through the wireless carrier system 262 and the ground communication network 264, the switch 291 is transmitted to the remote control center 298, the remote control center 298 is transmitted to the second processor 215, and the second processor 215 is connected to the receiving module 263. The processor 215 transmits the received image to the receiving module 263, the receiving module 263 transmits the received image to the compressed data scanning unit 449, the compressed data scanning unit 449 transmits to the compression logic acquisition unit 450, and the compression logic acquisition unit 450 transmits the The decompression reading unit 451, the decompression reading unit 451 transmits to the second judgment unit 452, the second judgment unit 452 transmits to the original byte data recovery unit 453, and the original byte data recovery unit 453 transmits the image to the visual display 255 .
本发明的有益效果：建立远程操作员远程驾驶的车辆远程驾驶体系，适应全路况的要求，车内不配置驾驶员，实现真正意义上的无人驾驶，为共享经济的发展提供技术支持。The invention has the beneficial effects of establishing a remote driving system for a vehicle remotely driven by a remote operator, adapting to the requirements of all road conditions, without a driver in the vehicle, realizing unmanned driving in a true sense, and providing technical support for the development of the sharing economy.
附图说明Description of the drawings
图1是主无线设备和次无线设备通过物联网连接示意图；Figure 1 is a schematic diagram of a primary wireless device and a secondary wireless device connected through the Internet of Things;
图2是本发明多协议通信网络接入系统的CAN总线模块的电路图；2 is a circuit diagram of the CAN bus module of the multi-protocol communication network access system of the present invention;
图3是本发明多协议通信网络接入系统的处理器的电路图的第三部分；3 is the third part of the circuit diagram of the processor of the multi-protocol communication network access system of the present invention;
图4是本发明多协议通信网络接入系统的通讯接口的电路图；4 is a circuit diagram of the communication interface of the multi-protocol communication network access system of the present invention;
图5是本发明多协议通信网络接入系统的控制原理图；Figure 5 is a control principle diagram of the multi-protocol communication network access system of the present invention;
图6是本发明多协议通信网络接入系统的处理器的电路图的第一部分；6 is the first part of the circuit diagram of the processor of the multi-protocol communication network access system of the present invention;
图7是本发明多协议通信网络接入系统的处理器的电路图的第二部分；7 is the second part of the circuit diagram of the processor of the multi-protocol communication network access system of the present invention;
图8是本发明多协议通信网络接入系统的RS232信号通讯芯片的电路图；Figure 8 is a circuit diagram of the RS232 signal communication chip of the multi-protocol communication network access system of the present invention;
图9是本发明多协议通信网络接入系统的RS485信号通讯电路图的第一部分；9 is the first part of the RS485 signal communication circuit diagram of the multi-protocol communication network access system of the present invention;
图10是本发明多协议通信网络接入系统的RS485信号通讯电路图的第二部分；10 is the second part of the RS485 signal communication circuit diagram of the multi-protocol communication network access system of the present invention;
图11是本发明多协议通信网络接入系统RS485信号通讯电路图的第三部分；Figure 11 is the third part of the RS485 signal communication circuit diagram of the multi-protocol communication network access system of the present invention;
图12是本发明多协议通信网络接入系统的以太网模块的电路图；12 is a circuit diagram of the Ethernet module of the multi-protocol communication network access system of the present invention;
图13为本发明雷达视频复合数据探测与处理系统的模块图；Figure 13 is a block diagram of the radar video composite data detection and processing system of the present invention;
图14是本发明实施例五提供的数据压缩的系统结构图；14 is a system structure diagram of data compression provided by Embodiment 5 of the present invention;
图15是本发明实施例六提供的数据解压缩的系统结构图；15 is a system structure diagram of data decompression provided by the sixth embodiment of the present invention;
图16为环境坐标系与像素坐标系的坐标关系图；Figure 16 is a diagram of the coordinate relationship between the environment coordinate system and the pixel coordinate system;
图17是本发明实施例二提供的数据压缩、解压缩接口信息结构图；Figure 17 is a data compression and decompression interface information structure diagram provided by the second embodiment of the present invention;
图18是本发明实施例一提供的无线通信网络传输数据压缩的场景图；18 is a scene diagram of data compression for wireless communication network transmission provided by Embodiment 1 of the present invention;
图19是本发明实施例一提供的数据压缩的方法流程图；FIG. 19 is a flowchart of a data compression method provided by Embodiment 1 of the present invention;
图20是本发明实施例二提供的数据压缩和存储的方法流程图；20 is a flowchart of a method for data compression and storage provided by Embodiment 2 of the present invention;
图21是本发明实施例二提供的增加了接口信息的数据压缩的方法流程图；FIG. 21 is a flowchart of a method for data compression with added interface information provided in the second embodiment of the present invention;
图22是本发明实施例三提供的数据解压缩的方法流程图；FIG. 22 is a flowchart of a data decompression method provided by Embodiment 3 of the present invention;
图23是本发明实施例四提供的数据解压缩和读取的方法流程图；FIG. 23 is a flowchart of a method for decompressing and reading data according to Embodiment 4 of the present invention;
图24是本发明第二和第三实施例双模驾驶模式的系统连接图。Fig. 24 is a system connection diagram of the dual-mode driving mode of the second and third embodiments of the present invention.
图25是本发明第二实施例双模驾驶工作切换逻辑图；25 is a logic diagram of dual-mode driving work switching logic diagram of the second embodiment of the present invention;
图26是本发明第三实施例双模驾驶工作切换逻辑图；26 is a logic diagram of dual-mode driving work switching logic diagram of the third embodiment of the present invention;
图27是本发明远程控制中心的结构图；Figure 27 is a structural diagram of the remote control center of the present invention;
图28是车辆的俯视图；Figure 28 is a top view of the vehicle;
图29是远程驾驶系统和机器人控制系统的示意图；Figure 29 is a schematic diagram of a remote driving system and a robot control system;
图30是远程控制台和操作员的示意图；Figure 30 is a schematic diagram of a remote console and an operator;
图31是机器人系统的立体图；Figure 31 is a perspective view of the robot system;
图32远程控制的机器人在汽车驾驶室内的立体图；Figure 32 is a perspective view of a remotely controlled robot in the cab of a car;
图33是机器人操纵器臂控制汽车方向盘的透视图；Figure 33 is a perspective view of a robot manipulator arm controlling a car steering wheel;
图34和图35是机器人操纵臂开关和机器人操纵臂；Figure 34 and Figure 35 are the robot manipulating arm switch and the robot manipulating arm;
图36是插管和器械的远侧部分和开关的使用；Figure 36 is the use of the distal part of the cannula and instrument and the switch;
图37是完全约束的逆雅可比主/从速度控制器的简化框图；Figure 37 is a simplified block diagram of a fully constrained inverse Jacobian master/slave speed controller;
图38是图38中的简化主/从控制的细化；Figure 38 is a refinement of the simplified master/slave control in Figure 38;
图39是修改的主/从控制器的简化图；Figure 39 is a simplified diagram of a modified master/slave controller;
图40是控制器的修改部分的示意图；Figure 40 is a schematic diagram of a modified part of the controller;
图41是完全约束的主/从机器人控制系统的示例性逆雅可比控制器；Figure 41 is an exemplary inverse Jacobian controller of a fully constrained master/slave robot control system;
图42、图43和图44是用于运动控制参照系的示意框图；Figure 42, Figure 43 and Figure 44 are schematic block diagrams of a reference frame for motion control;
图45和图46是末端执行器参照系和远程中心参照系的两种系统的框图；Figures 45 and 46 are block diagrams of two systems of the end effector reference system and the remote center reference system;
图47是在该机器人系统中的故障反应、故障隔离和故障弱化的方块图；Figure 47 is a block diagram of fault response, fault isolation and fault weakening in the robot system;
图48-图52是提供故障反应、故障隔离和故障弱化方法的流程图。Figures 48-52 are flowcharts that provide fault response, fault isolation, and fault weakening methods.
具体实施方式detailed description
远程操作员171通过远程控制机器人170驾驶车辆260行驶：远程控制台169与远程控制中心298连接，远程控制中心298与有线和无线局域网295连接，有线和无线局域网295与交换机291连接，交换机291与地面网络264连接，地面网络264与无线载波系统262连接，无线载波系统262与通信网络接入系统278连接，通信网络接入系统278与远程信息处理单元269连接，远程信息处理单元269与车辆总线276连接，The remote operator 171 drives the vehicle 260 through the remote control robot 170: the remote console 169 is connected to the remote control center 298, the remote control center 298 is connected to the wired and wireless LAN 295, the wired and wireless LAN 295 is connected to the switch 291, and the switch 291 is connected to The ground network 264 is connected, the ground network 264 is connected to the wireless carrier system 262, the wireless carrier system 262 is connected to the communication network access system 278, the communication network access system 278 is connected to the telematics unit 269, and the telematics unit 269 is connected to the vehicle bus 276 connections,
图26是远程控制中心298的示意图，端组中的各个席位按照该席位相应的权限和工作要求、制定计划发布控制指令，该控制指令数据传输网络交换机，网络交换机将控制指令数据传输至主服务器和二级服务器，经过主服务器和二级服务器的逻辑处理，然后将处理后的数据传输至图形拼接控制器上，图形拼接控制器智能化地实现各种数据的拼接、组合等操作，最后在大屏液晶显示屏中显示出来。图形拼接控制器智能化地实现各种数据的拼接、组合等操作，最后在大屏液晶显示屏中显示出来。PDA控制器发布控制指令，该控制指令数据传输网络交换机，网络交换机将控制指令数据传输至主服务器和二级服务器，经过主服务器和二级服务器的逻辑处理，然后将处理后的数据传输至图形拼接控制器上，图形拼接控制器智能化地实现各种数据的拼接、组合等操作，最后在大屏液晶显示屏中显示出来。在大屏液晶显示屏能够及时地显示出各种终端以及摄像头等部件的数据信息，便于终端各个席位的人员观看，以便于人们得到车辆260的信息数据，然后进行合适地协调操作。Figure 26 is a schematic diagram of the remote control center 298. Each seat in the end group formulates a plan to issue control instructions according to the corresponding authority and work requirements of the seat. The control instruction data is transmitted to the network switch, and the network switch transmits the control instruction data to the main server After the logic processing of the primary server and the secondary server, the processed data is transmitted to the graphics splicing controller. The graphics splicing controller intelligently realizes the splicing and combination of various data, and finally It is displayed on the large LCD screen. The graphics splicing controller intelligently realizes the splicing and combination of various data, and finally displays it on the large LCD screen. The PDA controller issues control instructions, which are transmitted to the network switch, and the network switch transmits the control instruction data to the main server and the secondary server, after the logic processing of the primary server and the secondary server, and then transmits the processed data to the graphics On the splicing controller, the graphic splicing controller intelligently realizes the splicing and combination of various data, and finally displays it on the large LCD screen. The large-screen LCD screen can promptly display the data information of various terminals and cameras and other components, which is convenient for the personnel in each seat of the terminal to view, so that people can obtain the information and data of the vehicle 260, and then conduct appropriate coordinated operations.
在图26中，远程控制中心298有数据终端的数量、语音终端的数量、图形工作站、PDA控制器，大屏显示控制主机的显示控制模块具有16种显示控制模式，通过图形拼接控制器实现大屏液晶显示屏的16种显示模式的选取和切换；大屏液晶显示屏为大屏幕显示器。远程控制中心298包括：大屏液晶显示屏、大屏显示控制主机、网络交换机、图形拼接控制器、图形工作站、图形工作站组控制主机、服务器组和终端组；网络交换机分别与图形工作站、图形拼接控制器、图形工作站组控制主机、服务器、终端一一对应电通信连接；大屏液晶显示屏用于显示图形拼接控制器拼接后的图形、视频、音频资料；图形拼接控制器用于从图形工作站中调取图形或视频或音频并完成组合、拼接工作；图形工作站组控制用于控制图形工作站中图形或视频或音频的存储、移动、显示、删除操作；网络交换机实现与图形工作站、图形拼接控制器、图形工作站组控制主机、服务器、终端之间对应的数据通讯；服务器由主服务器和二级服务器构成，终端由数据终端和语音终端构成，主服务器用于接收和控制数据终端的数据信息，二级服务器用于接收和控制语音终端的语音信息；大屏显示控制主机还电通信连接有无线接收器，无线接收器通过无线通讯方式通信连接有PDA控制器，数据终端发出的数据指令信息通过网络交换机传输给主服务器，通过主服务器进行逻辑运算处理，将数据信息和处理结果通过大屏液晶显示屏和数据终端的液晶显示屏显示出来；语音终端发出的语音指令信息通过网络交换机传输给二级服务器，通过二级服务器进行逻辑运算处理，将语音信息和处理结果通过大屏液晶显示屏和语音终端的液晶显示屏显示出来；PDA控制器发出的数据、语音指令信息通过无线通讯方式传输给无线接收器，无线接收器将数据、语音信息通过大屏显示控制主机传输给图形拼接控制器，通过主服务器、二级服务器的逻辑运算处理，将数据、语音信息和处理结果通过大屏液晶显 示屏和PDA控制器的液晶显示屏显示出来；图形工作站组控制主机通过网络交换机将数据信息传输给主服务器，通过主服务器进行逻辑运算处理，将数据信息和处理结果通过大屏液晶显示屏显示出来。In Figure 26, the remote control center 298 has the number of data terminals, the number of voice terminals, graphics workstations, and PDA controllers. The display control module of the large-screen display control host has 16 display control modes. Selection and switching of 16 display modes of the LCD screen; the large LCD screen is a large screen display. The remote control center 298 includes: large-screen LCD display, large-screen display control host, network switch, graphics splicing controller, graphics workstation, graphics workstation group control host, server group and terminal group; network switches are respectively connected with graphics workstations and graphics splicing Controller, graphic workstation group control host, server, terminal one-to-one corresponding electrical communication connection; large-screen LCD screen is used to display the graphics, video, and audio data after the graphic splicing controller is spliced; the graphic splicing controller is used from the graphic workstation Retrieve graphics or video or audio and complete the combination and splicing work; graphics workstation group control is used to control the storage, movement, display, and deletion of graphics or video or audio in the graphics workstation; the network switch realizes the graphics workstation and graphics splicing controller , The graphics workstation group controls the corresponding data communication between the host, server, and terminal; the server is composed of a main server and a secondary server, and the terminal is composed of a data terminal and a voice terminal. The main server is used to receive and control the data information of the data terminal. The level server is used to receive and control the voice information of the voice terminal; the large-screen display control host is also connected to a wireless receiver for electronic communication, and the wireless receiver is connected to the PDA controller through wireless communication, and the data command information sent by the data terminal passes through the network The switch is transmitted to the main server, through the main server for logical operation processing, the data information and processing results are displayed on the large-screen LCD display and the LCD display of the data terminal; the voice command information issued by the voice terminal is transmitted to the secondary level through the network switch Server, through the secondary server for logical operation processing, the voice information and processing results are displayed on the large-screen LCD display and the LCD display of the voice terminal; the data and voice command information sent by the PDA controller are transmitted to the wireless through wireless communication Receiver, wireless receiver transmits data and voice information to the graphics splicing controller through the large-screen display control host, and processes the data, voice information and processing results through the large-screen LCD screen through the logical operation of the main server and secondary server And the LCD display of the PDA controller is displayed; the graphics workstation group control host transmits data information to the main server through the network switch, and performs logical operation processing through the main server, and displays the data information and processing results through the large-screen LCD display.
在图1和图29-图32中，远程控制台169的第二处理器215由硬件、软件和固件组成，由一个单元执行或者分给若干子单元，每个子单元能够进而通过硬件、软件和固件的任何组合来实现，第二处理器215能够交叉连接控制逻辑和控制器，第二处理器215也能够作为子单元分布在整个车辆远程驾驶体系258中，第二处理器215能够执行来自非暂时性机器可读介质的机器可读指令，其激活第二处理器215以执行与指令相对应的动作，第二处理器215执行远程操作员171输入的各种指令，第二处理器215执行远程操作员171用左输入装置177和右输入装置178输入的指令，以致动第一机械手182和第二机械手183各自的关节，远程控制台169的第二处理器215与视觉显示器255、左输入装置177和右输入装置178、第一脚踏板214和第二脚底板233连接，视觉显示器255由第一显示屏174、第二显示屏175、第三显示屏176和第四显示屏179组成，第九成像装置410采集的驾驶室内部图像，通过压缩被传输到远程控制台169解压缩后显示在视觉显示器255的显示屏上。远程操作员171用双眼查看视觉显示器255上的图像。In FIGS. 1 and 29-32, the second processor 215 of the remote console 169 is composed of hardware, software, and firmware. It is executed by one unit or divided into several sub-units. Each sub-unit can further use hardware, software, and Realized by any combination of firmware, the second processor 215 can cross-connect the control logic and the controller, the second processor 215 can also be distributed as a subunit in the entire vehicle remote driving system 258, and the second processor 215 can execute A machine-readable instruction of a transitory machine-readable medium, which activates the second processor 215 to perform actions corresponding to the instruction, the second processor 215 executes various instructions input by the remote operator 171, and the second processor 215 executes The remote operator 171 uses the instructions input by the left input device 177 and the right input device 178 to activate the respective joints of the first manipulator 182 and the second manipulator 183, the second processor 215 of the remote console 169 and the visual display 255, left input The device 177 is connected to the right input device 178, the first foot pedal 214 and the second foot sole plate 233, and the visual display 255 is composed of a first display screen 174, a second display screen 175, a third display screen 176 and a fourth display screen 179 The image inside the cab collected by the ninth imaging device 410 is compressed and transmitted to the remote console 169 to be decompressed and displayed on the display screen of the visual display 255. The remote operator 171 views the image on the visual display 255 with both eyes.
在图1、图30和图31中，机器人170在汽车的驾驶室内直接控制汽车的方向盘235、油门踏板402、刹车踏板401、档位400，车门403、空调404和音响405。汽车座椅173包括座椅靠背216、防潜梁217、防潜连杆机构219、第五连杆218、第六连杆230、第七连杆231和立柱256，机器人170被固定到汽车座椅173上，在立柱256上安装第一机械手182、第二机械手183、第三机械手184和第四机械手185，机器人170上的机械手多于1个，第一机械手182、第二机械手183、第三机械手184和第四机械手185能够上下移动，并能够操纵附连的器械300、301、302、303。远程操作员171用左手抓握左输入装置177，左输入装置177能够引起第一机械手182的移动并与方向盘235连接；用右手抓握右输入装置178，右输入装置178能够引起第二机械手183的移动并与方向盘235连接，右输入装置178还能够引起第二机械手183的移动并与档位400连接，远程操作员171用左脚连接第二脚底板233，远程操作员171用右脚连接第一脚踏板214，第一脚踏板214能够引起机器人第三机械手184的移动，第三机械手184能够引起油门踏板402移动，第二脚底板233能够引起第四机械手185的移动，第四机械手185的移动能够与刹车踏板401连接。In FIGS. 1, 30 and 31, the robot 170 directly controls the steering wheel 235, the accelerator pedal 402, the brake pedal 401, the gear position 400, the door 403, the air conditioner 404 and the audio 405 of the car in the cab of the car. The car seat 173 includes a seat back 216, an anti-submarine beam 217, an anti-submersible link mechanism 219, a fifth link 218, a sixth link 230, a seventh link 231, and a column 256. The robot 170 is fixed to the car seat On the chair 173, a first manipulator 182, a second manipulator 183, a third manipulator 184, and a fourth manipulator 185 are installed on the column 256. There are more than one manipulator on the robot 170, the first manipulator 182, the second manipulator 183, and the The three manipulators 184 and the fourth manipulator 185 can move up and down, and can manipulate the attached instruments 300, 301, 302, and 303. The remote operator 171 grasps the left input device 177 with his left hand. The left input device 177 can cause the movement of the first manipulator 182 and is connected to the steering wheel 235; grasps the right input device 178 with the right hand, and the right input device 178 can cause the second manipulator 183 The right input device 178 can also cause the second manipulator 183 to move and connect with the gear 400. The remote operator 171 connects with the second foot base 233 with his left foot, and the remote operator 171 connects with his right foot The first foot pedal 214, the first foot pedal 214 can cause the robot's third manipulator 184 to move, the third manipulator 184 can cause the accelerator pedal 402 to move, and the second foot base 233 can cause the fourth manipulator 185 to move. The movement of the manipulator 185 can be connected to the brake pedal 401.
图1是远程驾驶系统258包括车辆260、无线载波系统262、地面通信网络264、计算机266和呼叫中心265，车辆260为摩托车、卡车、客车、运动型多功能车(SUV)、休闲车(RV)、船只、航空器和超高速管道列车。车辆电子设备263包括远程信息处理单元269、麦克风270、按钮和控制输入271、音频系统272、可视显示器273、GPS和BDS卫星导航模块274、多个车辆系统模块(VSM虚拟交换矩阵)275，车辆总线276与娱乐总线277连接。控制器区域网络(CAN)、媒体导向系统传输(MOST)、本地互连网络(LIN)、局域网(LAN)和其它连接，以太网或符合ISO、SAE和IEEE标准和规范，远程信息处理单元269安装在车辆260上，无线载波系统262通过无线联网能够使车辆260与呼叫中心265进行无线语音和数据通信通信。Figure 1 shows that the remote driving system 258 includes a vehicle 260, a wireless carrier system 262, a ground communication network 264, a computer 266, and a call center 265. The vehicle 260 is a motorcycle, truck, bus, sports utility vehicle (SUV), and recreational vehicle ( RV), ships, aircraft and ultra-high-speed pipeline trains. Vehicle electronic equipment 263 includes telematics unit 269, microphone 270, buttons and control input 271, audio system 272, visual display 273, GPS and BDS satellite navigation module 274, multiple vehicle system modules (VSM virtual switching matrix) 275, The vehicle bus 276 is connected to the entertainment bus 277. Controller Area Network (CAN), Media Oriented System Transmission (MOST), Local Interconnect Network (LIN), Local Area Network (LAN) and other connections, Ethernet or conform to ISO, SAE and IEEE standards and specifications, telematics unit 269 Installed on the vehicle 260, the wireless carrier system 262 enables the vehicle 260 to perform wireless voice and data communication with the call center 265 through wireless networking.
远程信息处理单元269使用无线电传输建立与无线载波系统262的通信信道，通信信道包括语音信道和数据信道，通信信道能够发送和接收语音和数据传输。通信网络接入系统278包括：处理器、微波通信单元、卫星通信单元和移动通信单元；其中处理器模块适于接收微波通信单元、卫星通信单元和移动通信单元的数据信息。远程信息处理单元269使用根据GSM或CDMA标准的蜂窝通信，包括用于语音通信的标准蜂窝芯片组279、用于数据传输的无线调制解调器、电子处理设备280、多个数字存储设备281和通信网络接入系统278包括：处理器、微波通信单元、卫星通信单元和移动通信单元；其中处理器模块适于接收微波通信单元、卫星通信单元和移动通信单元的数据信息。能够通过存储在远程信息处理单元269中且通过处理器280执行的软件实现调制解调器，或者调制解调器能够是位于远程信息处理单元269内部或外部的分立硬件组件。调制解调器能够使用任何数量的不同标准或协议诸如EVDO、CDMA、GPRS和EDGE来运行。能够使用远程信息处理单元269实施车辆与其它联网的设备之间的无线联网。远程信息处理单元269能够被配置为根据一个或多个无线协议诸如IEEE 422.11协议、WiMAX或蓝牙中的任一个进行无线通信。当用于TCP/IP的包交换数据通信时，远程信息处理单元能够配置有静态IP地址或者能够设置为自动接收来自网络上的另一设备诸如路由器或者来自网络地址服务器的所分配的IP地址。The telematics unit 269 uses radio transmission to establish a communication channel with the wireless carrier system 262. The communication channel includes a voice channel and a data channel, and the communication channel can send and receive voice and data transmissions. The communication network access system 278 includes: a processor, a microwave communication unit, a satellite communication unit and a mobile communication unit; wherein the processor module is adapted to receive data information of the microwave communication unit, the satellite communication unit and the mobile communication unit. The telematics unit 269 uses cellular communication according to the GSM or CDMA standard, and includes a standard cellular chipset 279 for voice communication, a wireless modem for data transmission, an electronic processing device 280, multiple digital storage devices 281, and a communication network interface. The access system 278 includes: a processor, a microwave communication unit, a satellite communication unit, and a mobile communication unit; wherein the processor module is adapted to receive data information of the microwave communication unit, the satellite communication unit, and the mobile communication unit. The modem can be realized by software stored in the telematics unit 269 and executed by the processor 280, or the modem can be a discrete hardware component located inside or outside the telematics unit 269. The modem can use any number of different standards or protocols such as EVDO, CDMA, GPRS and EDGE to operate. The telematics unit 269 can be used to implement wireless networking between the vehicle and other networked devices. The telematics unit 269 can be configured to perform wireless communication according to any one of one or more wireless protocols such as IEEE 422.11 protocol, WiMAX or Bluetooth. When used for TCP/IP packet exchange data communication, the telematics unit can be configured with a static IP address or can be set to automatically receive an assigned IP address from another device on the network such as a router or from a network address server.
智能电话284是与远程信息处理单元269通信的联网设备中的无线设备。智能电话284包括计算机处理能力、能够使用短程无线协议通信的收发机、以及可视智能电话显示器286，智能电话显示器286包括触屏图形用户接口。智能电话284构造成使用传统Wi-Fi协议通信。智能电话284包括使用无线载波系统262经由蜂窝通信进行通信的能力，包括处理能力、显示器286、在短程无线通信链接上通信的能力。智能电话282可使用WiFi直接协议建立短程无线链接。智能电话282包括不具有蜂窝通信能力的设备。处理器280能够处理电子指令的设备，包括微处理器、微控制器、主处理器、控制器、车辆通信处理器和专用集成电路(ASIC)，是用于远程信息处理单元269的专用处理器，能够与其它车辆系统共享。处理器280执行各种类型的数字存储指令，诸如存储器281中存储的软件或固件程序，使远程信息处理单元能够提供多种类型的服务，处理器280能够执行程序或处理数据。The smart phone 284 is a wireless device among networked devices that communicate with the telematics unit 269. The smart phone 284 includes computer processing capabilities, a transceiver capable of communicating using a short-range wireless protocol, and a visual smart phone display 286 that includes a touch screen graphical user interface. The smart phone 284 is configured to communicate using the traditional Wi-Fi protocol. The smart phone 284 includes the ability to communicate via cellular communication using the wireless carrier system 262, including processing capabilities, a display 286, and the ability to communicate over short-range wireless communication links. The smart phone 282 may use the WiFi direct protocol to establish a short-range wireless link. The smart phone 282 includes devices that do not have cellular communication capabilities. The processor 280 is a device capable of processing electronic instructions, including a microprocessor, a microcontroller, a main processor, a controller, a vehicle communication processor, and an application specific integrated circuit (ASIC). It is a dedicated processor for the telematics unit 269 , Can be shared with other vehicle systems. The processor 280 executes various types of digital storage instructions, such as software or firmware programs stored in the memory 281, so that the telematics unit can provide various types of services, and the processor 280 can execute programs or process data.
远程信息处理单元269提供来自车辆的无线通信服务，这些服务包括：卫星导航模块274提供的服务；结合碰撞传感器接口模块和车身控制模块提供的安全气囊展开通知服务；使用诊断模块的诊断报告；以及信息、音乐、网页、电影、电视节目、视频游戏，在将模块实施为位于远程信息处理单元269外部的VSM 275的情况下，它们能够使用车辆总线276以与远程信息处理单元269交换数据和命令。The telematics unit 269 provides wireless communication services from the vehicle. These services include: the service provided by the satellite navigation module 274; the airbag deployment notification service provided by the combination of the collision sensor interface module and the body control module; the diagnosis report using the diagnosis module; and Information, music, webpages, movies, TV shows, video games, when the modules are implemented as VSM 275 located outside the telematics unit 269, they can use the vehicle bus 276 to exchange data and commands with the telematics unit 269 .
卫星导航模块274根据从卫星285接收的无线电信号，卫星导航模块274能够确定车辆位置，向车辆驾驶员提供导航和其它位置相关服务。导航信息能够在显示器273上呈现和语音提示。位置信息能够提供给呼叫中心265或其它远程计算机系统，诸如计算机266。通过远程信息处理单元269将新的或更新的地图数据从呼叫中心265下载到卫星导航模块274。卫星导航包括GPS卫星导航系统和北斗(BDS)卫星导航系统。使用通信卫星289和上行链路发射站290来实施双向通信，节目内容由发射站290接收、被打包用于上载、然后发送到卫星289，卫星289向用户广播节目，双向通信使用卫星289的卫星电话服务，以在车辆260与站290之间中继电话通信。车辆260包括车辆系统模块(VSM)275，其位于车辆内且从传感器接收输入且使用感测的输入执行诊断、监控、控制、报告。每个VSM 275通过车辆总线276连接到其它 VSM以及连接到远程信息处理单元269，并且能够被编程以运行车辆系统和子系统诊断测试。一个VSM 275能够是发动机控制模块(ECM)，其控制发动机操作的各个方面，包括燃料点火和点火正时，另一VSM 275能够是动力系统控制模块，其调整车辆动力系的组件的操作，另一VSM 275能够是车身控制模块，管理车辆内的电子组件包括车辆的电动门锁和前灯，发动机控制模块配备有车载诊断(OBD)特征件，其提供诸如从包括车辆排放传感器的各种传感器接收到的各种实时数据，并且提供标准化的一系列诊断故障码(DTC)。The satellite navigation module 274 can determine the location of the vehicle based on the radio signals received from the satellite 285, and provide navigation and other location-related services to the vehicle driver. Navigation information can be presented and voiced on the display 273. The location information can be provided to a call center 265 or other remote computer system, such as computer 266. The new or updated map data is downloaded from the call center 265 to the satellite navigation module 274 through the telematics unit 269. Satellite navigation includes GPS satellite navigation system and Beidou (BDS) satellite navigation system. A communication satellite 289 and an uplink transmitter station 290 are used to implement two-way communication. The program content is received by the transmitter station 290, packaged for upload, and then sent to the satellite 289. The satellite 289 broadcasts programs to users. The two-way communication uses the satellite of the satellite 289 Telephone service to relay telephone communication between the vehicle 260 and the station 290. The vehicle 260 includes a vehicle system module (VSM) 275 that is located in the vehicle and receives input from sensors and uses the sensed input to perform diagnosis, monitoring, control, and reporting. Each VSM 275 is connected to other VSMs and to the telematics unit 269 through the vehicle bus 276, and can be programmed to run vehicle system and subsystem diagnostic tests. One VSM 275 can be an engine control module (ECM), which controls all aspects of engine operation, including fuel ignition and ignition timing, and the other VSM 275 can be a power system control module, which regulates the operation of components of the vehicle’s powertrain. A VSM 275 can be a body control module that manages the electronic components in the vehicle including the electric door locks and headlights of the vehicle. The engine control module is equipped with on-board diagnostics (OBD) features, which provide information such as various sensors including vehicle emission sensors. Receive various real-time data, and provide a standardized series of diagnostic trouble codes (DTC).
车辆电子设备263包括多个车辆用户接口，提供和接收信息的装置，包括麦克风270、按钮271、音频系统272和可视显示器273，麦克风270向远程信息处理单元269提供音频输入，使车辆占用者能够通过无线载波系统262提供语音命令和实施免提呼叫，能够利用人机接口(HMI)技术连接到车载自动语音处理单元。按钮271允许远程信息处理单元269的手动用户输入，以启动无线电话呼叫和提供其它数据、响应或控制输入。分立的按钮能够使用以便向呼叫中心265发起紧急呼叫和常规服务援助呼叫。音频系统272向车辆占用者提供音频输出，音频系统272连接到车辆总线276和娱乐总线277，能够提供AM、FM、卫星无线电、CD、DVD和其它多媒体功能。可视显示器273是图形显示器。无线载波系统262是蜂窝电话系统，包括蜂窝塔287，移动交换中心(MSC)288以及将无线载波系统262与地面网络264连接所需的任何其它联网组件。蜂窝塔287包括发送和接收天线以及基站，来自不同蜂窝塔的基站直接地连接到MSC 288或者通过基站控制器的中间设备连接到MSC288。蜂窝系统262能够实施通信技术，包括AMPS的模拟技术和CDMA和GSM/GPRS数字技术。The vehicle electronic equipment 263 includes multiple vehicle user interfaces, devices for providing and receiving information, including a microphone 270, buttons 271, an audio system 272, and a visual display 273. The microphone 270 provides audio input to the telematics unit 269 to enable the vehicle occupant It can provide voice commands and implement hands-free calls through the wireless carrier system 262, and can connect to the vehicle-mounted automatic voice processing unit using human-machine interface (HMI) technology. Button 271 allows manual user input of telematics unit 269 to initiate wireless telephone calls and provide other data, response, or control input. Separate buttons can be used to initiate emergency calls and regular service assistance calls to the call center 265. The audio system 272 provides audio output to vehicle occupants. The audio system 272 is connected to the vehicle bus 276 and the entertainment bus 277 and can provide AM, FM, satellite radio, CD, DVD, and other multimedia functions. The visual display 273 is a graphic display. The wireless carrier system 262 is a cellular telephone system that includes a cellular tower 287, a mobile switching center (MSC) 288, and any other networking components required to connect the wireless carrier system 262 with the terrestrial network 264. The cell tower 287 includes transmitting and receiving antennas and base stations. Base stations from different cell towers are directly connected to the MSC 288 or connected to the MSC 288 through an intermediate device of the base station controller. The cellular system 262 can implement communication technologies, including AMPS analog technologies and CDMA and GSM/GPRS digital technologies.
地面网络264是地面的电信网络，其连接有线电话并且将无线载波系统262连接到呼叫中心265。地面网络264包括公共交换电话网(PSTN)，用于提供硬线电话、包交换数据通信和因特网基础设施。能够通过使用标准有线网络、光纤和其它光学网络、电缆网络、电源线、诸如无线局域网(WLAN)的其它无线网络或者提供宽带无线接入(BWA)的网络或者其任意组合来实施一段或多段地面网络264。呼叫中心265与无线载波系统262直接连接。The terrestrial network 264 is a terrestrial telecommunications network that connects wired telephones and connects the wireless carrier system 262 to the call center 265. The terrestrial network 264 includes the Public Switched Telephone Network (PSTN), which is used to provide hard-line telephones, packet-switched data communications, and Internet infrastructure. Ability to implement one or more sections of ground by using standard wired networks, optical and other optical networks, cable networks, power lines, other wireless networks such as wireless local area networks (WLAN), or networks that provide broadband wireless access (BWA), or any combination thereof Network 264. The call center 265 is directly connected to the wireless carrier system 262.
计算机266通过远程信息处理单元269和无线载波262是由车辆访问的web服务器。计算机266通过远程信息处理单元269从车辆20上载诊断信息；计算机266提供因特网连接，提供DNS服务和作为网络地址服务器，其使用DHCP或其它适当协议向车辆260分配IP地址。呼叫中心265向车辆电子设备263提供系统后端功能，这些后端功能包括交换机291、服务器283、数据库292、远程控制中心298和自动语音响应系统(VRS)294通过有线和无线局域网295连接在一起。交换机291是专用交换(PBX)交换机，路由进入信号，使得语音传输通常通过常规电话发送到远程控制中心298，使用VoIP发送到自动语音响应系统294。远程控制中心298的电话也能够使用VoIP，通过交换机291的VoIP和其它数据通信通过在交换机291与网络295之间连接的调制解调器来实施。数据传输经由调制解调器传到服务器283和数据库292。数据库292能够存储账户信息、用户认证信息、车辆标识。还能够通过无线系统422.11x、GPRS进行数据传输。它通过远程控制中心298连接人工呼叫中心265而使用，呼叫中心265使用VRS 294作为自动指导者，呼叫中心265使用VRS 294与远程控制中心298连接。The computer 266 is a web server accessed by the vehicle through the telematics unit 269 and the wireless carrier 262. The computer 266 uploads diagnostic information from the vehicle 20 through the telematics unit 269; the computer 266 provides Internet connection, provides DNS services and acts as a network address server, which uses DHCP or other appropriate protocols to assign an IP address to the vehicle 260. The call center 265 provides system back-end functions to the vehicle electronic equipment 263. These back-end functions include a switch 291, a server 283, a database 292, a remote control center 298, and an automatic voice response system (VRS) 294 that are connected together through a wired and wireless LAN 295 . The switch 291 is a private exchange (PBX) switch that routes incoming signals so that the voice transmission is usually sent to the remote control center 298 via a regular telephone and to the automatic voice response system 294 using VoIP. The telephone of the remote control center 298 can also use VoIP, and the VoIP and other data communication through the switch 291 are implemented through a modem connected between the switch 291 and the network 295. The data transmission is transmitted to the server 283 and the database 292 via the modem. The database 292 can store account information, user authentication information, and vehicle identification. It can also transmit data via wireless system 422.11x and GPRS. It is used to connect to the manual call center 265 through the remote control center 298, the call center 265 uses the VRS 294 as an automatic instructor, and the call center 265 uses the VRS 294 to connect to the remote control center 298.
车辆总线276与机器人170连接，机器人170与第一机械手182连接、机器人170与第二机械手183连接、机器人170与第三机械手184连接、机器人170与第四机械手185连接，第一机械手182与方向盘235连接，第二机械手183与方向盘235连接，第二机械手183还能够与档位400连接，184与油门踏板402连接，第四机械手185与刹车踏板401连接，机器人170的电源线与车辆260的电源线连接在一起。The vehicle bus 276 is connected to the robot 170, the robot 170 is connected to the first robot 182, the robot 170 is connected to the second robot 183, the robot 170 is connected to the third robot 184, the robot 170 is connected to the fourth robot 185, and the first robot 182 is connected to the steering wheel. 235 is connected, the second manipulator 183 is connected to the steering wheel 235, the second manipulator 183 can also be connected to the gear 400, 184 is connected to the accelerator pedal 402, the fourth manipulator 185 is connected to the brake pedal 401, and the power cord of the robot 170 is connected to the vehicle 260 Connect the power cords together.
备用驾驶系统：远程控制台169与远程控制中心298，远程控制中心298与交换机291，交换机291与地面网络264，地面网络264与上行链路发射站290连接，上行链路发射站290与通信卫星289连接，通信卫星289与通信网络接入系统278，通信网络接入系统278与远程信息处理单元269，远程信息处理单元269与机器人170连接，机器人170与方向盘235连接、机器人170与档位400连接、机器人170与刹车踏板401连接和机器人170与油门踏板402连接后远程操作员171驾驶车辆260行驶；Backup driving system: remote control station 169 and remote control center 298, remote control center 298 and switch 291, switch 291 and ground network 264, ground network 264 and uplink transmission station 290 connected, uplink transmission station 290 and communication satellite 289 connection, communication satellite 289 and communication network access system 278, communication network access system 278 and telematics unit 269, telematics unit 269 and robot 170, robot 170 and steering wheel 235, robot 170 and gear 400 After the connection, the robot 170 is connected to the brake pedal 401 and the robot 170 is connected to the accelerator pedal 402, the remote operator 171 drives the vehicle 260;
主雷达视频图像传输线路：车辆视觉系统502与处理器280连接，The main radar video image transmission line: the vehicle vision system 502 is connected to the processor 280,
在图1和图28中，车辆260包括车辆视觉系统502，车辆视觉系统502被配置成捕获车辆周围360°区域内的图像。车辆视觉系统502的第一成像装置500安装在前挡风玻璃后面、车辆格栅、前仪表板和更接近车辆前边缘的位置，用于捕获车辆260向前视场(FOV)506的图像的前视摄像机，车辆视觉系统502的第二成像装置508安装在车辆的后部用于捕获车辆的向后视场(FOV)510的后视摄像机，车辆视觉系统502的第三成像装置512安装在车辆的左侧用于捕获车辆的侧面视场(FOV)514的侧视图像摄像机，车辆视觉系统502的第四成像装置504安装在车辆的右侧用于捕获车辆的侧面视场(FOV)519的侧视摄像机。In FIGS. 1 and 28, the vehicle 260 includes a vehicle vision system 502, which is configured to capture an image in a 360° area around the vehicle. The first imaging device 500 of the vehicle vision system 502 is installed behind the front windshield, the vehicle grille, the front instrument panel, and the position closer to the front edge of the vehicle, and is used to capture the image of the forward field of view (FOV) 506 of the vehicle 260 A front-view camera, the second imaging device 508 of the vehicle vision system 502 is installed at the rear of the vehicle and a rear-view camera used to capture the vehicle's backward field of view (FOV) 510, and the third imaging device 512 of the vehicle vision system 502 is installed at The left side of the vehicle is used to capture the side view of the vehicle's lateral field of view (FOV) 514, and the fourth imaging device 504 of the vehicle vision system 502 is installed on the right side of the vehicle to capture the lateral view of the vehicle (FOV) 519 Side-view camera.
第一机械手182上安装第五成像装置406、第二机械手183上安装第六成像装置407、第三机械手184上安装第七成像装置408和第四机械手185上安装第八成像装置409，在立柱256上安装第九成像装置410，第一成像装置到第九成像装置的成像系统都由视频采集设备120和雷达110组成，雷达110由激光雷达或毫米波雷达组成。The fifth imaging device 406 is installed on the first manipulator 182, the sixth imaging device 407 is installed on the second manipulator 183, the seventh imaging device 408 is installed on the third manipulator 184, and the eighth imaging device 409 is installed on the fourth manipulator 185. A ninth imaging device 410 is installed on 256, and the imaging systems of the first imaging device to the ninth imaging device are all composed of a video acquisition device 120 and a radar 110, and the radar 110 is composed of a lidar or a millimeter wave radar.
雷达110用于探测目标采集目标的目标数据和环境坐标，雷达110采用一发双收的FMCW体制，2D-FFT数据处理技术，探测的目标数据包含目标的径向距离、径向速度及角度信息。通过数据特征变换，根据几何关系将径向距离与角度信息转换为目标的横向距离及纵向距离信息，横向距离及纵向距离信息组成目标相对视频采集设备的环境坐标。对于运动目标的检测，雷达每次探测到的目标数据都会不同，为了获得更加准确的目标信息，尽可能的剔除虚假目标，需要采用数据关联及目标跟踪技术，将雷达多次探测到的目标信息进行数据关联并进行自适应滤波预测。当雷达获得准确的目标信息，对探测到的目标建立稳定跟踪时输出视频触发信号，触发摄像机进行图像获取及目标提取，并将雷达探测到的目标转换为相对相机的环境坐标数据传输给雷达视频信息融合系统130进行信息融合。The radar 110 is used to detect the target and collect the target data and environmental coordinates of the target. The radar 110 adopts the FMCW system with one transmitter and two receivers and 2D-FFT data processing technology. The detected target data includes the target's radial distance, radial velocity and angle information . Through data feature transformation, the radial distance and angle information are converted into the target's horizontal distance and vertical distance information according to the geometric relationship. The horizontal distance and the vertical distance information constitute the target's environmental coordinates relative to the video capture device. For the detection of moving targets, the target data detected by the radar will be different each time. In order to obtain more accurate target information and eliminate false targets as much as possible, it is necessary to adopt data association and target tracking technology to integrate the target information detected by the radar multiple times. Perform data association and adaptive filtering prediction. When the radar obtains accurate target information and establishes stable tracking of the detected target, the video trigger signal is output, which triggers the camera to perform image acquisition and target extraction, and converts the target detected by the radar into environmental coordinate data relative to the camera and transmits it to the radar video The information fusion system 130 performs information fusion.
视频采集设备120用于在雷达对目标实现跟踪后，采集目标的图像信息和像素坐标。视频采集设备120由摄像机组成，采集图形信息后通过对图像的处理获得目标特征数据，将目标的像素坐标数据等传输给雷达视频信息融合系统130。雷达视频信息融合系统130的输入端通信连接的雷达与视频采集设备，用于对目标的目标数据与图像信息进行信息融合，具体包含将获得的雷达110采集的目标数据进行坐标转换，从环境坐标转换为图像对应的像素坐标，雷达110探测目标位置与视频采集设备120采集的图像信息或视频数据进行时间配准、第一数据关联及决策判决，并将目标融合结果在显示屏上进行显示。The video acquisition device 120 is used to collect the image information and pixel coordinates of the target after the radar achieves tracking of the target. The video acquisition device 120 is composed of a camera, which acquires target characteristic data by processing the image after collecting graphic information, and transmits the pixel coordinate data of the target to the radar video information fusion system 130. The radar and video acquisition equipment connected to the input end of the radar video information fusion system 130 are used to perform information fusion on the target data and image information of the target, which specifically includes the coordinate conversion of the target data collected by the radar 110 from the environment coordinates. Converted to pixel coordinates corresponding to the image, the radar 110 detects the target position and the image information or video data collected by the video acquisition device 120 for time registration, first data association and decision making, and the target fusion result is displayed on the display screen.
在图13和图16中，雷达视频复合数据探测与处理系统的探测与处理方法包含以下步骤：In Figure 13 and Figure 16, the detection and processing method of the radar video composite data detection and processing system includes the following steps:
S1、雷达探测目标采集目标的目标数据和环境坐标。S1. The radar detects the target and collects the target data and environmental coordinates of the target.
S1.1、雷达探测目标，对回波数据进行处理获得目标数据，目标数据包含目标的径向距离、径向速度及角度信息。S1.1. The radar detects the target and processes the echo data to obtain target data. The target data includes the radial distance, radial velocity and angle information of the target.
S1.2、雷达通过数据特征变换，根据几何关系将径向距离与角度信息转换为目标的横向距离及纵向距离，该目标的横向距离及纵向距离组成目标相对视频采集设备的环境坐标。S1.2. The radar converts the radial distance and angle information into the horizontal distance and the vertical distance of the target according to the geometric relationship through the data feature transformation, and the horizontal distance and the vertical distance of the target constitute the environmental coordinates of the target relative to the video acquisition device.
S1.3、雷达获取目标的目标数据后，对雷达信息进行第二数据关联，雷达对当前时刻获取的目标数据进行第二数据关联的方法包含：航迹分叉法、最近邻方法、联合概率数据关联算法(JPDA)。雷达判断雷达探测的目标数，若雷达探测的目标数小于预设的数目阈值，目标数少或稀疏，则采用航迹分叉法或最近邻方法进行数据关联，计算简单实时性好。若雷达探测的目标数大于预设的数目阈值，目标数多且密集，则采用联合概率数据关联算法(JPDA)进行数据关联，该算法在杂波环境下有很好的跟踪性能。假设杂波环境下有多个目标存在，并且每个目标的航迹已经形成，如果回波有多个，则认为在跟踪门所有的回波都可能源于目标，只是每个回波源于目标的概率有所不同。S1.3. After the radar obtains the target data of the target, it performs the second data association on the radar information. The methods for the radar to perform the second data association on the target data obtained at the current moment include: track bifurcation method, nearest neighbor method, and joint probability Data Association Algorithm (JPDA). The radar judges the number of targets detected by the radar. If the number of targets detected by the radar is less than the preset number threshold, and the number of targets is small or sparse, the track bifurcation method or the nearest neighbor method is used for data association, and the calculation is simple and real-time. If the number of targets detected by the radar is greater than the preset number threshold, and the number of targets is large and dense, the joint probabilistic data association algorithm (JPDA) is used for data association, which has good tracking performance in clutter environments. Assuming that there are multiple targets in the clutter environment, and the track of each target has been formed, if there are multiple echoes, it is considered that all the echoes at the tracking gate may originate from the target, but each echo originates from the target The probability is different.
S1.4、雷达对当前时刻获取的目标数据进行自适应滤波预测，自适应滤波预测可采用卡尔曼(kalman)滤波跟踪进行目标跟踪预测，将目标。S1.4. The radar performs adaptive filtering prediction on the target data acquired at the current moment, and the adaptive filtering prediction can use Kalman filter tracking to perform target tracking and prediction, and then target the target.
S2、雷达实现目标跟踪后，视频采集设备采集目标的图像信息和像素坐标。S2. After the radar achieves target tracking, the video acquisition device collects the image information and pixel coordinates of the target.
S2.1、视频采集设备采集目标的图像信息。S2.1. The video capture device captures the image information of the target.
S2.2、视频采集设备对图像信息进行图像处理，获得目标特征数据，将目标特征数和像素坐标数据等传输给雷达视频信息融合系统。S2.2. The video acquisition device performs image processing on the image information to obtain target feature data, and transmits the target feature number and pixel coordinate data to the radar video information fusion system.
S3雷达视频信息融合系统将目标的目标数据与图像信息进行信息融合；信息融合包含：坐标变换、时间配准、数据决策和第一数据关联。The S3 radar video information fusion system integrates the target data and image information of the target; information fusion includes: coordinate transformation, time registration, data decision-making and first data association.
S3.1、雷达视频信息融合系统将雷达获取的目标数据由环境坐标向视频信息对应的像素坐标进行坐标转换，具体包含；环境坐标系Ow-XwYwZw，其原点以视频采集设备垂直于地面的交点为原点Ow(也可设置在任意位置，一般是参照实际情况进行设置)，Yw轴指向视频采集设备采集视频的水平正前方，Zw轴指向垂直于水平面向上，Xw轴位于水平面且垂直于Yw轴。像素坐标系Oo-UV，U轴和Y轴组成成像平面，成像平面垂直于环境坐标系Yw轴，以成像平面左上角为坐标原点Oo，像素坐标系的单位是像素。设视频采集设备离地高度H米时，环境坐标与像素坐标的关系如式(1)：
Figure PCTCN2020000015-appb-000001
Figure PCTCN2020000015-appb-000002
S3.1. The radar video information fusion system converts the target data obtained by the radar from the environment coordinates to the pixel coordinates corresponding to the video information, which specifically includes; the environment coordinate system Ow-XwYwZw, whose origin is the intersection point of the video acquisition device perpendicular to the ground It is the origin Ow (can also be set at any position, generally set according to the actual situation), the Yw axis points to the horizontal front of the video captured by the video capture device, the Zw axis points upwards perpendicular to the horizontal plane, and the Xw axis lies on the horizontal plane and perpendicular to the Yw axis . The pixel coordinate system Oo-UV, the U axis and the Y axis form an imaging plane, the imaging plane is perpendicular to the Yw axis of the environment coordinate system, the upper left corner of the imaging plane is the coordinate origin Oo, and the unit of the pixel coordinate system is a pixel. When the height of the video capture device is H meters above the ground, the relationship between the environmental coordinates and the pixel coordinates is as shown in formula (1):
Figure PCTCN2020000015-appb-000001
Figure PCTCN2020000015-appb-000002
式(1)中，u为目标在像素坐标系的U轴坐标，v为目标在像素坐标系的V轴坐标，ax、az为视频采集设备Xw轴和Zw轴方向的等效焦距，u0，v0为图像信息的像素中心的坐标，xw，yw，zw分别为相机照射物理范围内的点的环境坐标值。In formula (1), u is the U-axis coordinate of the target in the pixel coordinate system, v is the V-axis coordinate of the target in the pixel coordinate system, ax and az are the equivalent focal lengths of the Xw axis and Zw axis of the video capture device, u0, v0 is the coordinate of the pixel center of the image information, and xw, yw, and zw are respectively the environmental coordinate values of points within the physical range of the camera's irradiation.
S3.2、雷达视频信息融合系统对雷达的目标数据和视频采集设备的图像信息进行时间配准。雷达与视频相机数据刷新频率不同，需要将雷达探测目标信息与视频目标提取信息进行时间上的融合，确保配对数据的同步性，发挥好雷达与视频优势互补的作用。一般雷达的数据刷新频率要比摄像机快，可采用基于最小二乘准则的时间配准算法，具体包含：不同种类的传感器C和R，传感器C的采样周期为τ，传感器R的采样周期为T，采样周期的比例系数为整数n。如果距离传感器C的最近一次目标状态估计时刻记为(k-1)τ，则当前时刻表示为kτ＝[(k-1)τ+nT]，意味着在传感器C的一个周期之内，传感器R对目标状态估计的次数为n。最小二乘法时间配准的思路是将传感器R采集到的n次测量值融合为一虚拟测量测，并作为当前时刻传感器R的测量值。利用该测量值与传感器C的测量值进行融合，消除时间偏差引起的目标状态测量值不同步的目的，消除时间不匹配对多传感器信息融合精确度造成的影响。S3.2. The radar video information fusion system performs time registration on the radar target data and the image information of the video acquisition device. The data refresh frequency of radar and video cameras are different. It is necessary to integrate the information of the radar detection target and the extracted information of the video target in time to ensure the synchronization of the paired data, and to play the role of complementing the advantages of radar and video. Generally, the data refresh frequency of radar is faster than that of cameras, and the time registration algorithm based on least squares criterion can be used, which specifically includes: different types of sensors C and R, the sampling period of sensor C is τ, and the sampling period of sensor R is T , The proportional coefficient of the sampling period is an integer n. If the latest target state estimation time from sensor C is recorded as (k-1)τ, then the current time is expressed as kτ=[(k-1)τ+nT], which means that within one period of sensor C, the sensor The number of times R estimates the target state is n. The idea of least squares time registration is to fuse the n measurements collected by the sensor R into a virtual measurement, and use it as the measurement value of the sensor R at the current moment. The measurement value is fused with the measurement value of sensor C to eliminate the purpose of unsynchronized target state measurement value caused by time deviation, and eliminate the influence of time mismatch on the accuracy of multi-sensor information fusion.
设视频采集设备的采集周期为τ，雷达的采集周期为T，采集周期的比例系数为整数n；若视频采集设备最近一次目标状态估计时刻记为(k-1)τ，则当前时刻表示为kτ＝[(k-1)τ+nT]，n为视频采集设备的一个周期之内雷达对目标探测的次数；Suppose the acquisition period of the video acquisition device is τ, the acquisition period of the radar is T, and the scale factor of the acquisition period is an integer n; if the latest target state estimation time of the video acquisition device is recorded as (k-1)τ, the current time is expressed as kτ=[(k-1)τ+nT], n is the number of times that the radar detects the target in one cycle of the video acquisition device;
将雷达采集到的n次测量值融合为一虚拟测量测，并作为当前时刻雷达的测量值。假设S n＝[S1，S2，...，Sn] T为(k-1)τ到kτ时刻雷达探测到的某一目标位置数据的集合，sn与kτ时刻视频采集数据对应，若用表示S1，S2，...，Sn融合以后的量测值及其导数构成的列向量，则雷达探测数据的虚拟量测值si表示成：
Figure PCTCN2020000015-appb-000003
The n measured values collected by the radar are merged into a virtual measurement and used as the current measurement value of the radar. Suppose S n =[S1, S2,..., Sn] T is the collection of data of a certain target position detected by radar from (k-1)τ to kτ, sn corresponds to the video collection data at kτ, if expressed by S1, S2,..., Sn is a column vector composed of the measured values and their derivatives after fusion, and the virtual measured value si of the radar detection data is expressed as:
Figure PCTCN2020000015-appb-000003
其中vi表示量测噪声，将上式改写成向量形式为：S n＝W nU+V n Where vi represents the measurement noise, rewrite the above formula into vector form as: S n =W n U+V n
其中，V n＝[v1，v2，...，vn] T其均值为零，协方差阵为：
Figure PCTCN2020000015-appb-000004
Among them, V n =[v1, v2,..., vn] T has a mean value of zero, and the covariance matrix is:
Figure PCTCN2020000015-appb-000004
而
Figure PCTCN2020000015-appb-000005
为融合以前的位置量测噪声方差， and
Figure PCTCN2020000015-appb-000005
To measure the noise variance of the previous position,
根据最小二乘准则有目标函数：
Figure PCTCN2020000015-appb-000006
According to the least squares criterion, there is an objective function:
Figure PCTCN2020000015-appb-000006
使得J为最小，J两边对
Figure PCTCN2020000015-appb-000007
求导数并令其等于零得：
Figure PCTCN2020000015-appb-000008
Make J the smallest, and the two sides of J are right
Figure PCTCN2020000015-appb-000007
Find the derivative and make it equal to zero:
Figure PCTCN2020000015-appb-000008
从而有：
Figure PCTCN2020000015-appb-000009
Thus:
Figure PCTCN2020000015-appb-000009
相应的误差协方差阵为：
Figure PCTCN2020000015-appb-000010
The corresponding error covariance matrix is:
Figure PCTCN2020000015-appb-000010
将S n的表达式及式W n代入以上两式，可得融合以后的量测值及量测噪声方差分别为： Substituting the expression of S n and the formula W n into the above two formulas, the measured value and measurement noise variance after fusion can be obtained as:
Figure PCTCN2020000015-appb-000011
Figure PCTCN2020000015-appb-000011
其中c1＝-2/n，c2＝6/[n(n+1)]Where c1=-2/n, c2=6/[n(n+1)]
当前时刻雷达的测量值与视频采集设备的测量值采用最近邻数据关联方法进行融合。The measured value of the radar at the current moment and the measured value of the video acquisition device are fused using the nearest neighbor data association method.
S3.3、雷达视频信息融合系统对雷达的目标数据和视频采集设备的图像信息进行数据决策，具体包含：雷达视频信息融合系统判断当前时刻视频采集设备采集的图像信息的图像质量是否大于预设的阈值，若是则采用图像信息提取的目标数目信息，若否则采用雷达采集的目标数据提取的目标数目信息。S3.3. The radar video information fusion system makes data decisions on the target data of the radar and the image information of the video acquisition device, which specifically includes: the radar video information fusion system determines whether the image quality of the image information collected by the video acquisition device at the current moment is greater than the preset If yes, use the target number information extracted from the image information, if not, use the target number information extracted from the target data collected by radar.
S3.4、雷达视频信息融合系统对雷达的目标数据和视频采集设备的图像信息进行第一数据关联，这里第一数据关联采用最近邻数据关联方法，具体包含：首先，设置跟踪门以限制潜在的决策数目，跟踪门是跟踪空间中的一块子空间，以视频处理或雷达探测目标位置为中心来设置跟踪门，其大小应保证具有一定的正确匹配的概率。因此，残差较大的将首先被剔除。若跟踪门内雷达探测目标数大于1，则残差最小者被看作目标。S3.4. The radar video information fusion system performs the first data association between the radar target data and the image information of the video acquisition device. Here, the first data association uses the nearest neighbor data association method, which specifically includes: First, set up tracking gates to limit potential The tracking gate is a subspace in the tracking space. The tracking gate is set with video processing or radar detection target position as the center. Its size should ensure a certain probability of correct matching. Therefore, the larger residual will be eliminated first. If the number of targets detected by the radar in the tracking gate is greater than 1, the one with the smallest residual is regarded as the target.
S3.5、雷达视频信息融合系统显示通过显示屏显示目标融合结果信息；处理器280与通信网络接入系统278连接，S3.5. The radar video information fusion system displays the target fusion result information through the display screen; the processor 280 is connected to the communication network access system 278,
图5的通信网络接入系统278包括：处理器、微波通信单元、卫星通信单元和移动通信单元；其中处理器模块适于接收微波通信单元、卫星通信单元和移动通信单元的数据信息。微波通信单元包括：定向天线、射频单元；其中定向天线适于将接收到射频信号发送至射频单元，射频单元适于将射频信号进行调制后，发送给处理器模块进行解调成数据信息；或将数据信息通过处理器模块进行调制后，经过射频单元、定向天线进行发送。卫星通信单元包括：收发器以及Ka频段调制解调器；其中收发器与一特高频UHF天线连接UHF频段信号，处理器用于将接收到的UHF频段信号转换为Ka频段信号，Ka频段调制解调器相连的Ka天线用于向卫星发送转换后的Ka频段信号；或Ka频段调制解调器通过相连的Ka天线接收卫星发送的Ka频段信号，处理器用于将接收到的Ka频段信号转换为UHF频段信号，收发器适于通过特高频UHF天线发送转换后的UHF频段信号。移动通信单元为4G和5G通信模块；处理器适于接收或发送4G和5G信号。有线通讯单元包括：串口通讯电路、CAN总线模块、以太网模块；其中处理器适于接收串口通讯电路和CAN总线模块和以太网模块发送的数据信息，再将上述数据信息转换为Ka频段信号和UHF频段信号；或从Ka频段信号和UHF频段信号中提取出数据信息，通过串口通讯电路、CAN总线模块、以太网模块向外发送。多协议通信网络接入系统将无线通讯方式和有线通讯方式进行结合，即将相应协议的通讯信号发送给处理器，由处理器进行相应协议转换后，经相应通讯方式进行发送，实现了多协议之间的转换。The communication network access system 278 in FIG. 5 includes a processor, a microwave communication unit, a satellite communication unit, and a mobile communication unit; wherein the processor module is adapted to receive data information of the microwave communication unit, the satellite communication unit, and the mobile communication unit. The microwave communication unit includes: a directional antenna and a radio frequency unit; wherein the directional antenna is adapted to send the received radio frequency signal to the radio frequency unit, and the radio frequency unit is adapted to modulate the radio frequency signal and send it to the processor module for demodulation into data information; or After the data information is modulated by the processor module, it is sent through the radio frequency unit and the directional antenna. The satellite communication unit includes: a transceiver and a Ka-band modem; the transceiver is connected to a UHF UHF antenna for UHF-band signals, the processor is used to convert the received UHF-band signals into Ka-band signals, and the Ka-band modem is connected to the Ka antenna Used to send converted Ka-band signals to satellites; or Ka-band modems receive Ka-band signals sent by satellites through connected Ka antennas. The processor is used to convert received Ka-band signals into UHF-band signals, and the transceiver is suitable for passing The UHF UHF antenna transmits the converted UHF band signal. The mobile communication unit is a 4G and 5G communication module; the processor is suitable for receiving or sending 4G and 5G signals. The wired communication unit includes: a serial communication circuit, a CAN bus module, and an Ethernet module; the processor is adapted to receive the data information sent by the serial communication circuit, the CAN bus module and the Ethernet module, and then convert the above data information into a Ka-band signal and UHF frequency band signal; or extract data information from Ka frequency band signal and UHF frequency band signal, and send it out through serial communication circuit, CAN bus module, and Ethernet module. The multi-protocol communication network access system combines wireless communication and wired communication, that is, the communication signal of the corresponding protocol is sent to the processor. After the processor performs the corresponding protocol conversion, it is sent through the corresponding communication method to realize the multi-protocol Conversion between.
在图3、图6、图7、图11和图12中，处理器采用STM32系列单片机，STM32F10XC型处理器的21、22、25、26、27、28号引脚分别与以太网模块的36、37、32、33、34、35号引脚相连以进行通信。能够从STM32F10XC型处理器的18、19、20、39、40、41、42、43、45、46号引脚中选择任意引脚用于连接射频单元、收发器和Ka频段调制解调器以及4G和5G通信模块。In Fig. 3, Fig. 6, Fig. 7, Fig. 11 and Fig. 12, the processor adopts STM32 series single-chip microcomputer. The 21, 22, 25, 26, 27 and 28 pins of the STM32F10XC processor are respectively connected to the 36 of the Ethernet module. , 37, 32, 33, 34, 35 pins are connected for communication. Able to select any pin from the 18, 19, 20, 39, 40, 41, 42, 43, 45, 46 pins of the STM32F10XC type processor for connecting radio frequency units, transceivers and Ka-band modems as well as 4G and 5G Communication module.
在图2中，CAN总线模块采用SN65HVD230型芯片，CAN总线模块1、4号引脚与处理器46、45号引脚电性相连，通过CAN总线模块实现多块处理器进行级联，实现对处理器的拓展，以满足控制多个处理器之间进行通信的需求。In Figure 2, the CAN bus module uses the SN65HVD230 chip, the CAN bus module 1 and 4 pins are electrically connected to the processor 46, 45 pins, and the CAN bus module realizes the cascading of multiple processors to realize the The processor is expanded to meet the needs of controlling communication between multiple processors.
在图4、图8、图9、图10和图12中，串口通讯电路包括：通讯接口、与处理器电性相连的RS485信号通讯电路、RS232信号通讯芯片；其中通讯接口设置有RS485信号通讯电路的输入端和RS232信号通讯芯片的输入端，RS485信号通讯电路的输入端和RS232信号通讯芯片的输入端将数据信息发送至处理器；处理器适于将RS232信号转换为RS485信号。通讯接口的9、10号引脚与处理器30、31号引脚电性相连。通讯接口的3、4号引脚接入RS485信号通讯电路的6、7号引脚，通讯接口的5、6号引脚接入RS232信号通讯芯片的7、8号引脚。RS485信号通讯电路的1、2、4号引脚分别与处理器14、15、16号引脚相连，RS232信号通讯芯片的10、9号引脚分别与处理器12、13号引脚相连。处理器模块内设置有信息分类数据库；处理器模块适于提取数据信息中的关键内容，并在信息分类数据库中进行比对，且根据比对结果进行分类后，按照分类对应的传输方式进行传输，在分类传输时，会将相应待传输的通讯协议加载在数据信息中，以满足相应通讯要求，进而实现多协议之间自动配置。串口通讯电路还包括：与通讯接口电性相连的通信指示电路；通信指示电路设置有第一指示灯、第二指示灯，当与通讯接口相连的RS485信号通讯电路正常工作时，第一指示灯指示为绿灯亮，以及当与通讯接口相连的RS232信号通讯芯片正常工作时，第二指示灯指示为绿灯亮。多协议通信网络接入系统还包括：DC-DC降压电路；DC-DC降压电路适于对设备供电、稳压。In Figure 4, Figure 8, Figure 9, Figure 10 and Figure 12, the serial communication circuit includes: a communication interface, an RS485 signal communication circuit electrically connected to the processor, and an RS232 signal communication chip; the communication interface is provided with RS485 signal communication The input terminal of the circuit and the input terminal of the RS232 signal communication chip, the input terminal of the RS485 signal communication circuit and the input terminal of the RS232 signal communication chip send data information to the processor; the processor is suitable for converting the RS232 signal into the RS485 signal. The pins 9 and 10 of the communication interface are electrically connected to pins 30 and 31 of the processor. The 3 and 4 pins of the communication interface are connected to the 6 and 7 pins of the RS485 signal communication circuit, and the 5 and 6 pins of the communication interface are connected to the 7 and 8 pins of the RS232 signal communication chip. The pins 1, 2, and 4 of the RS485 signal communication circuit are respectively connected to the pins 14, 15, and 16 of the processor, and the pins 10 and 9 of the RS232 signal communication chip are respectively connected to the pins 12 and 13 of the processor. The processor module is equipped with an information classification database; the processor module is suitable for extracting the key content of the data information, and comparing it in the information classification database, and after classifying according to the comparison result, transmitting according to the corresponding transmission mode of the classification , During classified transmission, the corresponding communication protocol to be transmitted will be loaded into the data information to meet the corresponding communication requirements, thereby realizing automatic configuration between multiple protocols. The serial communication circuit also includes: a communication indication circuit electrically connected to the communication interface; the communication indication circuit is provided with a first indicator light and a second indicator light. When the RS485 signal communication circuit connected to the communication interface works normally, the first indicator light The indicator is green, and when the RS232 signal communication chip connected to the communication interface works normally, the second indicator is green. The multi-protocol communication network access system also includes: a DC-DC step-down circuit; the DC-DC step-down circuit is suitable for powering and stabilizing equipment.
通信网络接入系统278与无线载波系统262连接，无线载波系统262与地面网络264连接，地面网络264与交换机291连接，交换机291与远程控制中心298连接，远程控制中心298与第二处理器215连接，第二处理器215与视觉显示器255连接。备用雷达视频图像传输线路：车辆视觉系统502与处理器280连接，处理器280与通信网络接入系统278 连接，通信网络接入系统278与通信卫星289连接，289与上行链路发射站290连接，上行链路发射站290与地面网络264连接，地面网络264与交换机291连接，交换机291与远程控制中心298连接，远程控制中心298与第二处理器215连接，第二处理器215与视觉显示器255连接。The communication network access system 278 is connected to the wireless carrier system 262, the wireless carrier system 262 is connected to the ground network 264, the ground network 264 is connected to the switch 291, the switch 291 is connected to the remote control center 298, and the remote control center 298 is connected to the second processor 215 Connected, the second processor 215 is connected to the visual display 255. Backup radar video image transmission line: the vehicle vision system 502 is connected to the processor 280, the processor 280 is connected to the communication network access system 278, the communication network access system 278 is connected to the communication satellite 289, and the 289 is connected to the uplink transmitting station 290 , The uplink transmitting station 290 is connected to the ground network 264, the ground network 264 is connected to the switch 291, the switch 291 is connected to the remote control center 298, the remote control center 298 is connected to the second processor 215, and the second processor 215 is connected to the visual display 255 connections.
在图14和图15中，车辆视觉系统502的雷达110和视频采集设备120由雷达视频信息融合系统130融合，443扫描雷达视频信息融合系统130的图像传输给压缩存储单元444、压缩存储单元444把图像传输给第一判断单元445，第一判断单元445把图像传输给压缩数据生成单元446，压缩数据生成单元446把压缩好的图像传输给发送模块447，由发送模块447发送给通信网络接入系统278，通信网络接入系统278通过无线载波系统262和地面通信网络264传输给交换机交换机291，交换机291传输给远程控制中心298，远程控制中心298传输给第二处理器215，第二处理器215连接接收模块263，第二处理器215把接收到的图像传输给接收模块263，接收模块263把接收到的图像传输给压缩数据扫描单元449，压缩数据扫描单元449传输给压缩逻辑获取单元450，压缩逻辑获取单元450传输给解压缩读取单元451，解压缩读取单元451传输给第二判断单元452，第二判断单元452传输给原始字节数据恢复单元453，原始字节数据恢复单元453把图像传输给视觉显示器255。In Figure 14 and Figure 15, the radar 110 of the vehicle vision system 502 and the video acquisition device 120 are fused by the radar video information fusion system 130, and the image of the 443 scanning radar video information fusion system 130 is transmitted to the compression storage unit 444 and the compression storage unit 444. The image is transmitted to the first judging unit 445, the first judging unit 445 transmits the image to the compressed data generating unit 446, and the compressed data generating unit 446 transmits the compressed image to the sending module 447, which sends the compressed image to the communication network interface. Into the system 278, the communication network access system 278 transmits to the switch switch 291 through the wireless carrier system 262 and the ground communication network 264, the switch 291 transmits to the remote control center 298, and the remote control center 298 transmits to the second processor 215, the second processing The receiver 215 is connected to the receiving module 263, the second processor 215 transmits the received image to the receiving module 263, the receiving module 263 transmits the received image to the compressed data scanning unit 449, and the compressed data scanning unit 449 transmits to the compression logic acquiring unit 450, the compression logic acquisition unit 450 transmits to the decompression reading unit 451, the decompression reading unit 451 transmits to the second judgment unit 452, and the second judgment unit 452 transmits to the original byte data recovery unit 453, and the original byte data is restored The unit 453 transmits the image to the visual display 255.
图17-图23是数据压缩、解压缩的方法和系统，图17是数据压缩、解压缩接口信息结构图，接口信息可包括压缩类型和原始字节数据的大小，压缩类型的定义可用0表示无压缩，用1表示已压缩。进一步对原始字节数据中存在连续递增的字节数据，以及原始字节数据中存在不连续相同且不连续递增的字节数据进行数据压缩，进一步降低了数据的冗余度，提高了数据传输效率；通过对压缩数据添加接口信息，可保证接收模块正确地完成解压缩处理过程。Figures 17-23 are data compression and decompression methods and systems. Figure 17 is a data compression and decompression interface information structure diagram. The interface information can include the compression type and the size of the original byte data. The definition of the compression type can be represented by 0 No compression, use 1 to indicate compressed. Further data compression is performed on the continuously increasing byte data in the original byte data and the discontinuously identical and discontinuously increasing byte data in the original byte data, which further reduces the redundancy of the data and improves the data transmission Efficiency: By adding interface information to the compressed data, it can ensure that the receiving module correctly completes the decompression process.
在图18中，传输数据经数据压缩后通过无线信道在无线节点411和无线节点414之间进行传输，其中，无线节点411包括发送模块412；无线节点414包括接收模块413；数据压缩功能部署在发送模块413上，数据解压缩功能部署在接收模块413上。数据压缩的方法包括以下步骤：415、扫描原始字节数据。扫描从原始字节数据的第一个字节开始，依次扫描，根据扫描结果，确定出原始字节数据所存在的冗余成分，再针对冗余数据的特点进行下一步的压缩处理步骤。416对原始字节数据进行压缩和存储。若原始字节数据中存在连续相同的字节数据，则与连续相同的字节数据的个数进行第一逻辑运算，得到第一逻辑运算值，将第一逻辑运算值存储为一个字节数据、连续相同的字节数据的任一个字节数据存储为另一个字节数据。例如，当原始字节数据中存在连续相同的字节数据有3个，且分别是0x05，0x05，0x05时，则第一逻辑运算是，使用0x80与连续相同的字节数据的个数0x03进行逻辑“或”运算得到第一逻辑运算值0x83，然后所得到的第一逻辑运算值0x83存储为一个字节数据，将连续相同的字节数据0x05存储为另一个字节数据。417判断原始字节数据是否扫描完成。若扫描完成，则转向步骤418；若未扫描完成，则返回步骤415继续扫描。418根据存储的字节数据，生成压缩数据。步骤416中针对原始字节数据中存在连续相同的字节数据进行压缩处理，使得原来占用3个字节数据的字节数据经数据压缩处理后，只占用2个字节数据。0x83，0x05即为压缩数据。由上述技术方案可知，通过步骤a：扫描原始字节数据b：若原始字节数据中存在连续相同的字节数据，则与连续相同的字节数据的个数进行第一逻辑运算，得到第一逻辑运算值，将第一逻辑运算值存储为一个字节数据、连续相同的字节数据的任一个字节数据存储为另一个字节数据c：判断原始字节数据是否扫描完成，若是，转向步骤d，若否，返回步骤a；d：根据存储的字节数据，生成压缩数据。In FIG. 18, the transmitted data is transmitted between the wireless node 411 and the wireless node 414 through the wireless channel after data compression. The wireless node 411 includes a sending module 412; the wireless node 414 includes a receiving module 413; the data compression function is deployed in On the sending module 413, the data decompression function is deployed on the receiving module 413. The data compression method includes the following steps: 415. Scan the original byte data. The scanning starts from the first byte of the original byte data, and scans sequentially. According to the scanning results, the redundant components of the original byte data are determined, and then the next step of compression processing is carried out according to the characteristics of the redundant data. 416 compresses and stores the original byte data. If there are consecutive identical byte data in the original byte data, perform the first logical operation with the number of consecutive identical byte data to obtain the first logical operation value, and store the first logical operation value as one byte data , Any one byte data of consecutive identical byte data is stored as another byte data. For example, when there are 3 consecutive identical byte data in the original byte data, and they are 0x05, 0x05, and 0x05, the first logical operation is to use 0x80 and the number of consecutive identical byte data 0x03. The logical OR operation obtains the first logical operation value 0x83, and then the obtained first logical operation value 0x83 is stored as one byte of data, and the continuous identical byte data 0x05 is stored as another byte of data. 417 Determine whether the original byte data has been scanned. If the scan is completed, go to step 418; if the scan is not completed, go back to step 415 to continue scanning. 418 generates compressed data based on the stored byte data. In step 416, compression processing is performed for the consecutive identical byte data in the original byte data, so that the byte data that originally occupied 3 bytes of data only occupies 2 bytes of data after data compression processing. 0x83 and 0x05 are compressed data. It can be seen from the above technical solution that through step a: scan the original byte data b: if there are consecutive identical byte data in the original byte data, perform the first logical operation with the number of consecutive identical byte data to obtain the first A logical operation value, the first logical operation value is stored as one byte of data, and any byte data of the same continuous byte data is stored as another byte data c: Determine whether the original byte data has been scanned, if so, Go to step d, if not, return to step a; d: generate compressed data according to the stored byte data.
在图19中，首先根据步骤419确定原始字节数据中的数据特点；若由步骤420，即当确定原始字节数据中存在连续递增的字节数据；则可执行步骤421，即与连续递增的字节数据的个数进行第二逻辑运算，得到第二逻辑运算值；接着执行步骤422，即将第二逻辑运算值存储为一个字节数据、连续递增的字节数据的第一个字节数据存储为另一个字节数据。其中，第二逻辑运算，是使用0xC0与连续递增的字节数据的个数进行“或”运算。若由步骤423，即当确定原始字节数据中存在不连续相同且不连续递增的字节数据；则可执行步骤424，即与不连续相同且不连续递增的字节数据的个数进行第三逻辑运算，得到第三逻辑运算值；接着执行步骤425，即将第三逻辑运算值存储为一个字节数据、不连续相同且不连续递增的字节数据的每一个字节数据依次各存储为另一个字节数据。其中，第三逻辑运算，是使用0x00与不连续相同且不连续递增的字节数据的个数进行“或”运算。最后根据步骤426获得存储的字节数据，根据存储的字节数据，生成压缩数据。In Figure 19, first determine the data characteristics in the original byte data according to step 419; if step 420, that is, when it is determined that there is continuously increasing byte data in the original byte data, step 421 can be performed, which is different from the continuous increase The second logical operation is performed on the number of byte data to obtain the second logical operation value; then step 422 is executed, that is, the second logical operation value is stored as one byte of data, the first byte of continuously increasing byte data The data is stored as another byte of data. Among them, the second logic operation is to use 0xC0 and the number of continuously increasing byte data to perform an "OR" operation. If step 423, that is, when it is determined that there are discontinuously identical and discontinuously increasing byte data in the original byte data, step 424 may be performed, that is, the number of discontinuously identical and discontinuously increasing byte data is carried out. Three logic operations are performed to obtain the third logic operation value; then step 425 is performed, that is, the third logic operation value is stored as one byte of data, and each byte of data that is discontinuously identical and discontinuously increasing is sequentially stored as Another byte of data. Among them, the third logical operation is to perform an "OR" operation using 0x00 and the number of discontinuously identical and discontinuously increasing byte data. Finally, the stored byte data is obtained according to step 426, and compressed data is generated according to the stored byte data.
在图20中，在步骤418之后还可增加以下步骤：419对压缩数据添加接口信息，生成压缩数据包。In FIG. 20, the following step may be added after step 418: 419 add interface information to the compressed data to generate a compressed data packet.
图23是数据解压缩的方法，当无线节点414的接收模块413接收到对方无线节点发送过来的压缩数据后，可使用数据解压缩的方法进行解压缩处理，然后把解压缩后的数据交由本无线节点做后续处理。数据解压缩的方法可包括以下步骤：427扫描压缩数据，压缩数据是原始字节数据通过实施例一中的数据压缩方法进行压缩后得到的数据。428对压缩数据的第n个字节数据进行压缩逻辑判断运算，获取压缩逻辑判断值和压缩逻辑，其中，n为大于等于1的自然数。429、根据压缩逻辑判断值和压缩逻辑，对压缩数据进行解压缩和读取。若压缩逻辑判断值等于第一预设值，则判断压缩逻辑对应原始字节数据中存在连续相同的字节数据；对第n个字节数据进行第一逻辑个数运算，得到数据个数i，其中，i为大于等于2的自然数，重复读取i个压缩数据的第n+1个字节数据。430判断压缩数据是否扫描完成。若扫描完成，转向步骤431；若未扫描完成，则返回步骤427继续扫描。431根据读取的字节数据，恢复原始字节数据。如步骤429，原始字节数据是读取的字节数据0x05，0x05，0x05。Figure 23 is a method of data decompression. After the receiving module 413 of the wireless node 414 receives the compressed data sent by the other wireless node, it can use the data decompression method to decompress, and then hand over the decompressed data to this The wireless node does subsequent processing. The method of data decompression may include the following steps: 427 scanning compressed data, the compressed data is the data obtained after the original byte data is compressed by the data compression method in the first embodiment. 428 performs a compression logic judgment operation on the n-th byte data of the compressed data to obtain a compression logic judgment value and a compression logic, where n is a natural number greater than or equal to 1. 429. Decompress and read the compressed data according to the compression logic judgment value and the compression logic. If the compression logic judgment value is equal to the first preset value, it is judged that there are consecutive identical byte data in the original byte data corresponding to the compression logic; the first logical number operation is performed on the nth byte data to obtain the number of data i , Where i is a natural number greater than or equal to 2, and the n+1th byte data of i compressed data is repeatedly read. 430 It is judged whether the scanning of the compressed data is completed. If the scan is completed, go to step 431; if the scan is not completed, go back to step 427 to continue scanning. 431 restores the original byte data according to the read byte data. In step 429, the original byte data is the read byte data 0x05, 0x05, 0x05.
在图22中，在步骤427的基础上，即在根据压缩逻辑判断值和压缩逻辑，对压缩数据进行解压缩和读取的基础上，进一步提供了压缩逻辑判断值等于第二预设值，即压缩逻辑对应原始字节数据中存在连续递增的字节数据；以及压缩逻辑判断值不等于第一预设值且不等于第二预设值，即压缩逻辑对应原始字节数据中存在不连续相同且不连续递增的字节数据的情况下，对压缩数据进行解压缩和读取。In FIG. 22, on the basis of step 427, that is, on the basis of decompressing and reading the compressed data according to the compression logic judgment value and the compression logic, it is further provided that the compression logic judgment value is equal to the second preset value, That is, there is continuously increasing byte data in the original byte data corresponding to the compression logic; and the judgment value of the compression logic is not equal to the first preset value and not equal to the second preset value, that is, there is discontinuity in the original byte data corresponding to the compression logic In the case of the same and discontinuously increasing byte data, the compressed data is decompressed and read.
在图23中，首先根据步骤432，确定压缩逻辑判断值，若由步骤433，确定压缩逻辑判断值等于第二预设值，则由步骤434，可确定压缩逻辑对应原始字节数据中存在连续递增的字节数据，进一步执行步骤435，对压缩数据的第n个字节数据进行第二逻辑个数运算，得到数据个数j，其中，j为大于等于2的自然数；最后根据步骤436，从压缩数据的第n+1个字节数据开始，依次读取j个字节数据。其中，第二预设值是0xC0；第二逻辑个数运算，是使用0x38与第n个字节数据进行“或”运算。In FIG. 23, first, determine the compression logic judgment value according to step 432. If it is determined in step 433 that the compression logic judgment value is equal to the second preset value, then step 434 can determine that the compression logic corresponds to the original byte data. For the incremented byte data, step 435 is further performed to perform the second logical number operation on the n-th byte data of the compressed data to obtain the data number j, where j is a natural number greater than or equal to 2; finally, according to step 436, Starting from the n+1th byte of compressed data, read j bytes of data in sequence. Among them, the second preset value is 0xC0; the second logical number operation is to use 0x38 and the nth byte of data to perform an "OR" operation.
若由步骤437，确定压缩逻辑判断值不等于第一预设值且不等于第二预设值，则由步骤438，可确定压缩逻辑对应原始字节数据中存在不连续相同且不连续递增的字节数据，进一步执行步骤439，对压缩数据的第n个字节数据进行第三逻辑个数运算，得到数据个数k，其中，k为大于等于2的自然数；最后，根据步骤440，即从压缩数据的第n+1个字节数据开始，依次读取k个字节数据。其中，第三逻辑个数运算，是使用0x00与第n个字节数据进行“或”运算。最后根据步骤441、获取读取的字节数据，恢复原字节数据。If it is determined in step 437 that the compression logic judgment value is not equal to the first preset value and not equal to the second preset value, then in step 438, it can be determined that there are discontinuously identical and discontinuously increasing original byte data corresponding to the compression logic For byte data, step 439 is further performed to perform a third logical number operation on the n-th byte data of the compressed data to obtain the number of data k, where k is a natural number greater than or equal to 2; finally, according to step 440, Starting from the n+1th byte of compressed data, read k bytes of data in sequence. Among them, the third logical number operation is an OR operation using 0x00 and the nth byte data. Finally, according to step 441, the read byte data is obtained, and the original byte data is restored.
图15是数据压缩的系统442包括：雷达110，视频采集设备120，雷达视频信息融合系统130，雷达110由激光雷达和毫米波雷达组成，原始字节数据扫描单元443，压缩存储单元444，第一判断单元445和压缩数据生成单元446，发送模块447，原始字节数据扫描单元443，用于扫描原始字节数据；压缩存储单元444，用于对原始字节数据进行压缩和存储；第一判断单元445，用于判断原始字节数据是否扫描完成；压缩数据生成单元446，用于根据存储的字节数据，生成压缩数据，压缩好的数据传输给发送模块447。Figure 15 is a data compression system 442 including: radar 110, video acquisition device 120, radar video information fusion system 130, radar 110 is composed of lidar and millimeter wave radar, raw byte data scanning unit 443, compression storage unit 444, A judging unit 445 and compressed data generating unit 446, sending module 447, original byte data scanning unit 443, used to scan the original byte data; compression storage unit 444, used to compress and store the original byte data; The judging unit 445 is used to judge whether the original byte data has been scanned; the compressed data generating unit 446 is used to generate compressed data according to the stored byte data, and the compressed data is transmitted to the sending module 447.
图16是数据解压缩的系统448包括接收模块263，压缩数据扫描单元449，压缩逻辑判断值和压缩逻辑获取单元450，解压缩读取单元451，第二判断单元452和原始字节数据恢复单元453。其中压缩数据扫描单元449，用于扫描压缩数据；压缩逻辑判断值和压缩逻辑获取单元450，用于对压缩数据的第n个字节数据进行压缩逻辑判断运算，获取压缩逻辑判断值和压缩逻辑，其中n为大于等于1的自然数；解压缩读取单元451，用于根据压缩逻辑判断值和压缩逻辑，对压缩数据进行解压缩和读取；第二判断单元452，用于判断压缩数据是否扫描完成；原始字节数据恢复单元453，用于根据读取的字节数据，恢复原始字节数据。显示器255显示原始字节数据恢复单元453恢复的原始字节数据16 is a data decompression system 448 including a receiving module 263, a compressed data scanning unit 449, a compression logic judgment value and compression logic acquisition unit 450, a decompression reading unit 451, a second judgment unit 452, and an original byte data recovery unit 453. The compressed data scanning unit 449 is used to scan compressed data; the compression logic judgment value and compression logic acquisition unit 450 is used to perform compression logic judgment operations on the nth byte of compressed data to obtain the compression logic judgment value and compression logic , Where n is a natural number greater than or equal to 1; the decompression reading unit 451 is used to decompress and read the compressed data according to the compression logic judgment value and the compression logic; the second judgment unit 452 is used to judge whether the compressed data is Scanning is complete; the original byte data recovery unit 453 is used to recover the original byte data according to the read byte data. The display 255 displays the original byte data recovered by the original byte data recovery unit 453
通过以上连接建立的各个子系统，远程操作员171用以下的系统远程驾驶车辆260行走：Through the various subsystems established by the above connection, the remote operator 171 uses the following system to remotely drive the vehicle 260 to walk:
图33一图46是远程驾驶系统，在图28中，第一连杆139的远端在提供水平枢转轴线138的关节处被连接到第二连杆137的近端。第三连杆124的近端在滚动关节处被连接到第二连杆137的远端，使得第三连杆通常围绕沿着第二连杆和第三连杆两者的轴线延伸的轴线在关节123处旋转或滚动，在枢轴关节125之后向远侧进行，第四连杆136的远端通过一对枢轴关节135、134连接到器械保持器136，枢轴关节135、134一起限定器械保持器121，机器人170操纵器臂组件133的平移或棱柱型关节132便于器械126轴向移动，能够将器械保持器131附接到插管，穿过该插管器械126可滑动地被插入，在器械保持器131的远侧，第二器械126包括额外的自由度，第二器械126的自由度的致动由机器人操纵器臂组件133的马达驱动，第二器械126和机器人操纵器臂组件133之间的接口可沿着操纵器臂组件133的运动链更近侧或更远侧地设置，第二器械126包括枢轴点PP的近侧的旋转关节130，其设置在需要的部位处，第二器械126的远侧允许末端执行器128围绕器械腕关节轴线129、127进行枢转运动。能够独立于末端执行器128的位置和取向来控制末端执行器钳口231之间的角度θ。Figures 33 to 46 are remote driving systems. In Figure 28, the distal end of the first link 139 is connected to the proximal end of the second link 137 at a joint that provides a horizontal pivot axis 138. The proximal end of the third link 124 is connected to the distal end of the second link 137 at the rolling joint, so that the third link generally lies around an axis extending along the axes of both the second link and the third link. The rotation or rolling at the joint 123 is performed distally after the pivot joint 125. The distal end of the fourth link 136 is connected to the instrument holder 136 through a pair of pivot joints 135, 134, and the pivot joints 135, 134 together define The instrument holder 121, the translation of the robot 170 manipulator arm assembly 133 or the prismatic joint 132 facilitates the axial movement of the instrument 126, enabling the instrument holder 131 to be attached to the cannula through which the instrument 126 is slidably inserted , On the distal side of the instrument holder 131, the second instrument 126 includes additional degrees of freedom, the actuation of the second instrument 126 degrees of freedom is driven by the motor of the robot manipulator arm assembly 133, the second instrument 126 and the robot manipulator arm The interface between the components 133 can be arranged closer or farther along the kinematic chain of the manipulator arm assembly 133. The second instrument 126 includes a rotary joint 130 on the proximal side of the pivot point PP, which is arranged at a desired location. At this point, the distal side of the second instrument 126 allows the end effector 128 to pivot about the instrument wrist axis 129, 127. The angle θ between the end effector jaws 231 can be controlled independently of the position and orientation of the end effector 128.
在图30中，左手持输入装置177和右主输入装置178通过无线通信与控制台169连接和分离，左手持输入装置177与第二处理器215连接，右手持输入装置178与第二处理器215连接，远程操作员171在远程控制台169激活第二处理器215后开始执行远程驾驶工作，远程操作员171的左手控制左手持输入装置177，左手持输入装置177通过第二处理器215控制手臂端197的移动，远程操作员171的右手控制右手持输入装置178，右手持输入装置178通过第二处理器215控制手臂端197的移动，手臂端197使用末端执行器193中的第一接触端194和第二接触端196与方向盘235接触并握紧，通过左手持输入装置177和右手持输入装置178向相反的方向运动能够使方向盘235转动，远程操作员171使用远程控制台169的第二处理器215软件控制机器人170的第一机械手182和第二机械手183，远程操作员171通过测量、模型估计、测量和建模来确定施加在机器人170第一机械手182和第二机械手183上用在方向盘235上的力，第一机械手182和第二机械手183通过远程控制台169向远程操作员171提供触觉反馈，这种触觉反馈能够为远程操作员171模拟手动操纵手臂端197控制方向盘235，能够为远程操作员171模拟由机器人170第一机械手182经历的对应于方向盘235的反作用力。In FIG. 30, the left hand-held input device 177 and the right main input device 178 are connected to and separated from the console 169 through wireless communication, the left hand-held input device 177 is connected to the second processor 215, and the right hand-held input device 178 is connected to the second processor. 215 connection, the remote operator 171 starts to perform remote driving work after the remote console 169 activates the second processor 215, the left hand of the remote operator 171 controls the left hand input device 177, and the left hand input device 177 is controlled by the second processor 215 The movement of the arm end 197, the right hand of the remote operator 171 controls the right hand input device 178, the right hand input device 178 controls the movement of the arm end 197 through the second processor 215, and the arm end 197 uses the first contact in the end effector 193 The end 194 and the second contact end 196 are in contact with the steering wheel 235 and held tightly. The steering wheel 235 can be rotated by moving the left hand input device 177 and the right hand input device 178 in opposite directions. The remote operator 171 uses the The second processor 215 software controls the first manipulator 182 and the second manipulator 183 of the robot 170. The remote operator 171 determines the application on the first manipulator 182 and the second manipulator 183 of the robot 170 through measurement, model estimation, measurement and modeling. With the force on the steering wheel 235, the first manipulator 182 and the second manipulator 183 provide tactile feedback to the remote operator 171 through the remote console 169. This tactile feedback can simulate the manual manipulation of the arm end 197 for the remote operator 171 to control the steering wheel 235. The reaction force corresponding to the steering wheel 235 experienced by the first manipulator 182 of the robot 170 can be simulated for the remote operator 171.
在图33中，末端执行器193中的第一接触端194和第二接触端196，它们相对于彼此枢转，以便限定一对末端执行器钳口231，对于具有末端执行器钳口231的器械，通过挤压输入装置177、178的抓手构件来致动钳口231，机器人170操纵第一机械手182和第二机械手183将使传动组件195在方向盘235上部移动，使得轴187延伸和收回，提供末端执行器193的期望移动。机器人170操纵第一机械手182和第二机械手183、第三机械手184和第四机械手185在远程驾驶过程中能够在方向盘235、档位400处、刹车401踏板和油门踏板402处移动。In FIG. 33, the first contact end 194 and the second contact end 196 of the end effector 193 pivot relative to each other so as to define a pair of end effector jaws 231. For those with end effector jaws 231 For the instrument, the jaws 231 are actuated by squeezing the grip members of the input devices 177 and 178, and the robot 170 manipulates the first manipulator 182 and the second manipulator 183 to move the transmission assembly 195 on the upper part of the steering wheel 235, so that the shaft 187 is extended and retracted , Provide the desired movement of the end effector 193. The robot 170 manipulates the first manipulator 182, the second manipulator 183, the third manipulator 184, and the fourth manipulator 185 to move at the steering wheel 235, the gear position 400, the brake 401 pedal, and the accelerator pedal 402 during remote driving.
在图33和图36中，第一机械手182和第二机械手183握紧方向盘235执行汽车方向的变化，第一机械手182上连接有器械保持器180，器械保持器180与器械186和手臂端197连接，器械保持器180与第一机械手182依靠机动化的关节连接，器械保持器180包括器械保持器框架188、夹具189和器械保持器托架190.夹具189被固定到器械保持器框架188的远端，夹具189能够与手臂端197连接和分离，器械保持器托架190与器械保持器框架188连接，器械保持器托架190沿器械保持器框架188的线性平移是由第二处理器215控制的机动化的平移移动。器械186包括传动组件195、细长轴187和末端执行器193，传动组件195与器械保持器托架190连接。轴187从传动组件195向远侧延伸。末端执行器193设置在轴187的远端处。轴187限定纵向轴线192，纵向轴线192与手臂端197的纵向轴线重合并且与由手臂端197限定的纵向轴线重合。当器械保持器托架190沿着器械保持器框架188平移时，器械186的细长轴187沿着纵向轴线192移动。末端执行器193能够从工作空间延长和缩回。In FIGS. 33 and 36, the first manipulator 182 and the second manipulator 183 grip the steering wheel 235 to change the direction of the car. The first manipulator 182 is connected with an instrument holder 180, the instrument holder 180 and the instrument 186 and the arm end 197 The instrument holder 180 is connected to the first manipulator 182 by a motorized joint. The instrument holder 180 includes an instrument holder frame 188, a clamp 189 and an instrument holder bracket 190. The clamp 189 is fixed to the instrument holder frame 188 At the distal end, the clamp 189 can be connected to and separated from the arm end 197, the instrument holder bracket 190 is connected with the instrument holder frame 188, and the linear translation of the instrument holder bracket 190 along the instrument holder frame 188 is determined by the second processor 215 Controlled motorized translational movement. The instrument 186 includes a transmission assembly 195, an elongated shaft 187 and an end effector 193, and the transmission assembly 195 is connected to the instrument holder bracket 190. The shaft 187 extends distally from the transmission assembly 195. The end effector 193 is provided at the distal end of the shaft 187. The shaft 187 defines a longitudinal axis 192 that coincides with the longitudinal axis of the arm end 197 and coincides with the longitudinal axis defined by the arm end 197. As the instrument holder bracket 190 is translated along the instrument holder frame 188, the elongate shaft 187 of the instrument 186 moves along the longitudinal axis 192. The end effector 193 can be extended and retracted from the working space.
在图34、图35、图36和图1中，远程操作员171通过第二处理器215发送指示，三态开关202接收到激活信号，远程操作员171使用第二处理器215和车辆远程驾驶体系258连接机器人170操纵第一机械手182手臂端197做握紧和远离方向盘235的移动，远程操作员171使用第二处理器215和车辆远程驾驶体系258连接机器人170操纵第二机械手183手臂端197做握紧和远离方向盘235的移动。末端执行器193中的第一接触端194和第二接触端196向方向盘235上施加力，使方向盘235转动，释放三态开关202将手臂端197停止移动，需要手臂端197与方向盘235连接时，远程操作员171发送激活第二方向信号，第一方向与第二方向相反，三态开关202接收到激活第二方向信号，手臂端197向方向盘235移动。In Figure 34, Figure 35, Figure 36, and Figure 1, the remote operator 171 sends instructions through the second processor 215, the three-state switch 202 receives the activation signal, and the remote operator 171 uses the second processor 215 to drive the vehicle remotely. The system 258 connects the robot 170 to manipulate the arm end 197 of the first manipulator 182 to grip and move away from the steering wheel 235. The remote operator 171 uses the second processor 215 and the vehicle remote driving system 258 to connect the robot 170 to manipulate the arm end 197 of the second manipulator 183 Make a grip and move away from the steering wheel 235. The first contact end 194 and the second contact end 196 in the end effector 193 apply force to the steering wheel 235 to rotate the steering wheel 235, release the three-state switch 202 to stop the movement of the arm end 197, and when the arm end 197 is connected to the steering wheel 235 , The remote operator 171 sends an activation second direction signal, the first direction is opposite to the second direction, the tri-state switch 202 receives the activation second direction signal, and the arm end 197 moves to the steering wheel 235.
在图37中，151是将主输入装置152连接到从操纵器154的主/从控制器153的简化控制器示意图。使用矢量数学符 号来描述控制器输入、输出和计算，其中矢量X将参考笛卡尔坐标中的方位矢量，并且其中矢量q将参考相关联的联动装置的关节铰接配置矢量，有时称为关节空间中的联动装置方位。当存在歧义时，能够将下标附加到这些矢量以识别特定结构，使得
Figure PCTCN2020000015-appb-000012
是主输入装置在相关联的主工作空间或坐标系中的方位，而x s表示该工作空间中的从动件的方位。与方位矢量相关联的速度矢量由矢量上方的点或矢量和下标之间的单词“dot”表示，例如主速度矢量的xdot m其中速度矢量在数学上被定义为方位矢量随时间变化的变化，控制器153包含逆雅可比速度控制器，在
Figure PCTCN2020000015-appb-000013
是主输入装置的方位并是主输入装置的速度的情况下，控制器153计算用于传输到从操纵器154的动力命令，以实现从主速度对应于输入装置的从末端执行器运动。控制器153能够计算从从方位x s和从速度施加到主输入装置和从那里到远程操作员171的手的力反射信号。 In FIG. 37, 151 is a simplified controller schematic diagram of the master/slave controller 153 connecting the master input device 152 to the slave manipulator 154. Use vector mathematical notation to describe controller inputs, outputs and calculations, where the vector X will refer to the orientation vector in Cartesian coordinates, and where the vector q will refer to the joint articulation configuration vector of the associated linkage, sometimes called the joint space The orientation of the linkage. When there is ambiguity, subscripts can be appended to these vectors to identify specific structures so that
Figure PCTCN2020000015-appb-000012
Is the position of the main input device in the associated main working space or coordinate system, and x s represents the position of the follower in the working space. The velocity vector associated with the bearing vector is represented by the dot above the vector or the word "dot" between the vector and the subscript, such as xdot m of the main velocity vector, where the velocity vector is mathematically defined as the change of the bearing vector over time , The controller 153 contains the inverse Jacobian speed controller, in
Figure PCTCN2020000015-appb-000013
In the case of the orientation of the master input device and the speed of the master input device, the controller 153 calculates the power command for transmission to the slave manipulator 154 to realize the movement of the slave end effector corresponding to the input device from the master speed. The controller 153 can calculate the force reflection signal applied to the main input device and from there to the hand of the remote operator 171 from the slave position x s and the slave speed.
在图38和图40中，第一模块159包含逆雅可比速度控制器，其具有来自使用根据虚拟从路径163修改的逆雅可比矩阵进行的计算的输出。首先描述虚拟从路径，与虚拟从动件相关联的矢量通常用v下标表示，使得关节空间qdot v中的虚拟从速度被积分以提供q v，使用逆运动模块162处理q v以生成虚拟从关节方位信号x v。虚拟从方位和主输入命令x m被组合并使用正向运动161进行处理。虚拟从动件的使用有助于在接近系统的硬限制时、在超越系统的软限制时等等进行平滑控制和力反射，由第一控制模块159和第二控制模块160指示的以及控制示意图165的其他部件和其他控制器的结构，该结构包含数据处理硬件、软件和固件，这样的结构包括可重新编程的软件和数据，其体现在机器可读代码中并存储在有形介质中，以供远程控制台169的第二处理器215使用机器可读代码以各种不同的配置存储，包括随机存取存储器、非易失性存储器、一次写入存储器、磁记录介质和光学记录介质。体现代码和与其相关联的数据的信号通过各种通信链路传输，该通信链路包括因特网、内联网、以太网、无线通信网络和链路、电信号和导体和光纤和网络。第二处理器215包含远程控制台169的一个和多个数据处理器，包括操纵器、器械、单独和远程处理结构和位置中的一个和多个的局部数据处理电路，模块包含单个公共处理器板、多个单独的板，模块中的一个和多个分散在多个板上，其中一些板还运行另一个模块的一些和全部计算。模块的软件代码被编写为单个集成软件代码，每个模块分成单独的子程序，或者一个模块的部分代码与另一个模块的一些或全部代码组合。数据和处理结构包括各种集中式或分布式数据处理和编程体系结构中的任何一种。 In FIGS. 38 and 40, the first module 159 contains an inverse Jacobian speed controller, which has an output from a calculation performed using an inverse Jacobian matrix modified according to the virtual slave path 163. First describe the virtual slave path. The vector associated with the virtual follower is usually represented by the v subscript, so that the virtual follower velocity in the joint space qdot v is integrated to provide q v , and the inverse motion module 162 is used to process q v to generate the virtual From the joint position signal x v . The virtual slave position and the master input command x m are combined and processed using the forward motion 161. The use of the virtual follower is helpful for smooth control and force reflection when approaching the hard limit of the system, when surpassing the soft limit of the system, etc., indicated by the first control module 159 and the second control module 160 and the control diagram The structure of other components of 165 and other controllers includes data processing hardware, software, and firmware. Such a structure includes reprogrammable software and data, which is embodied in machine-readable code and stored in a tangible medium to The second processor 215 for the remote console 169 uses machine-readable codes to store in a variety of different configurations, including random access memory, non-volatile memory, write-once memory, magnetic recording media, and optical recording media. Signals embodying codes and data associated with them are transmitted through various communication links, including the Internet, Intranet, Ethernet, wireless communication networks and links, electrical signals and conductors, and optical fibers and networks. The second processor 215 includes one or more data processors of the remote console 169, including one or more local data processing circuits of manipulators, instruments, separate and remote processing structures and locations, and the module includes a single common processor Board, multiple individual boards, one or more of the modules are scattered on multiple boards, some of which also run some and all calculations of another module. The software code of the module is written as a single integrated software code, and each module is divided into separate subroutines, or part of the code of one module is combined with some or all of the code of another module. The data and processing structure includes any of a variety of centralized or distributed data processing and programming architectures.
在图40中，控制器的输出，该控制器将经常试图求解一个特定的操纵器关节配置矢量q，以用于为这些高度可配置的从操纵器机构生成命令。操纵器联动装置通常具有足够的自由度，以便占据给定末端执行器状态的一系列关节状态。其中一个关节的致动直接被沿着运动链的不同关节的类似致动替换的结构。这些结构有时被称为具有过剩、额外或冗余的自由度，同时这些术语通常涵盖运动链，其中中间连杆能够移动而不改变末端执行器的方位。使用图40的速度控制器引导高度可配置的操纵器的移动时，第一模块的主要关节控制器经常试图确定或求解虚拟关节速度矢量qdot v，其能够被用于以使得末端执行器将准确地遵循主命令x m的方式驱动从操纵器164的关节。对于具有冗余自由度的从机构，逆雅可比矩阵通常不完全定义关节矢量解。在能够占据给定末端执行器状态的一系列关节状态的系统中，从笛卡尔命令xdot到关节运动qdot的映射是一对多的映射，因为机构是冗余的，所以存在数学上无数个解，其由逆生存的子空间表示。控制器使用列比行更多的雅可比矩阵来体现这种关系，将多个关节速度映射到相对较少的笛卡尔速度。通过由软件约束的远程运动中心298的概念被确定。通过具有计算软件枢轴点的能力，能够选择性地实现以系统的顺应性或刚度为特征的不同模式。在计算估计的枢轴点之后，实现在一定范围的枢轴点/中心上的不同系统模式。在固定枢轴实施方式中，能够将估计的枢轴点与期望的枢轴点进行比较以生成误差输出，该误差输出能够被用于将器械的枢轴驱动到期望的位置。相反地，在被动枢轴实施方式中，虽然期望的枢轴位置可能不是最重要的目标，但是估计的枢轴点能够被用于误差检测并因此用于安全性，因为估计的枢轴点位置的变化指示与方向盘分离或者传感器出现故障，从而使系统有机会采取纠正措施。 In Figure 40, the output of the controller, which will often try to solve for a specific manipulator joint configuration vector q, is used to generate commands for these highly configurable slave manipulator mechanisms. Manipulator linkages usually have enough degrees of freedom to occupy a series of joint states for a given end effector state. A structure in which the actuation of one joint is directly replaced by similar actuation of different joints along the kinematic chain. These structures are sometimes referred to as having excess, extra or redundant degrees of freedom, and these terms usually cover kinematic chains in which the middle link can move without changing the orientation of the end effector. When using the speed controller of FIG. 40 to guide the movement of a highly configurable manipulator, the main joint controller of the first module often tries to determine or solve the virtual joint speed vector qdot v , which can be used to make the end effector accurate The joint of the slave manipulator 164 is driven in a manner of following the master command x m . For slave mechanisms with redundant degrees of freedom, the inverse Jacobian matrix usually does not completely define the joint vector solution. In a system that can occupy a series of joint states for a given end effector state, the mapping from the Cartesian command xdot to the joint movement qdot is a one-to-many mapping. Because the mechanism is redundant, there are countless solutions in mathematics. , Which is represented by the subspace of inverse survival. The controller uses Jacobian matrices with more columns than rows to reflect this relationship, and maps multiple joint velocities to relatively few Cartesian velocities. The concept of the remote motion center 298 constrained by software is determined. By having the ability to calculate software pivot points, different modes characterized by system compliance or stiffness can be selectively implemented. After calculating the estimated pivot point, different system modes on a certain range of pivot points/centers are realized. In the fixed pivot embodiment, the estimated pivot point can be compared with the desired pivot point to generate an error output that can be used to drive the pivot of the instrument to a desired position. Conversely, in passive pivot implementations, although the desired pivot position may not be the most important target, the estimated pivot point can be used for error detection and therefore for safety, because the estimated pivot point position The change indicates that it is separated from the steering wheel or the sensor is malfunctioning, giving the system an opportunity to take corrective actions.
在图39中，处理器157包括第一控制器模块157和第二控制器模块160，第一模块157能够包含主要关节控制器，逆雅可比主-从控制器。第一模块157的主要关节控制器能够被配置用于响应于来自主输入装置156的输入产生期望的操纵器组件运动。操纵器联动装置具有一系列用于在空间中的给定末端执行器方位的替代配置。用于使末端执行器呈现给定方位的命令能够引起各种不同的关节运动和配置，第二模块160能够被配置为帮助将操纵器组件驱动到期望的配置，在主-从运动期间将操纵器朝向优选配置驱动，第二模块160将包含配置相关过滤器。第一模块157的主要关节控制器和第二模块160的配置相关过滤器都能够包含由处理器157使用来将关节的线性组合的控制权限传送到一个和多个目标或任务的实现的过滤器。假设X是关节运动的空间，则F(X)能够是对关节进行控制的过滤器，以i)提供期望的末端执行器运动，以及ii)在孔部位处提供器械轴的枢转运动。第一模块157的主要关节控制器可包含过滤器F(X)。从概念上讲，(1-F-1F)(X)能够描述配置相关的子空间过滤器，其给予与实现主要关节控制器的目标正交的关节速度的线性组合的控制致动权限。这种配置相关过滤器能够由控制器157的第二模块160使用以实现第二目标。两个过滤器能够进一步细分为对应于实现更具体任务的更多过滤器。过滤器F(X)能够分成F1(X)和F2(X)，分别用于控制末端执行器和控制枢转轴运动，其中任何一个都能够被选择作为处理器的最高优先级任务。In FIG. 39, the processor 157 includes a first controller module 157 and a second controller module 160. The first module 157 can include a main joint controller and an inverse Jacobian master-slave controller. The main joint controller of the first module 157 can be configured to generate the desired manipulator assembly motion in response to input from the main input device 156. The manipulator linkage has a series of alternative configurations for a given end effector orientation in space. Commands for the end effector to assume a given orientation can cause a variety of different joint motions and configurations. The second module 160 can be configured to help drive the manipulator assembly to the desired configuration, and manipulate the manipulator during the master-slave movement. The filter is driven toward the preferred configuration, and the second module 160 will contain configuration-related filters. Both the main joint controller of the first module 157 and the configuration-related filters of the second module 160 can include filters used by the processor 157 to transfer the control authority of the linear combination of joints to the realization of one or more goals or tasks. . Assuming that X is the space of joint movement, F(X) can be a filter that controls the joint to i) provide the desired end effector movement, and ii) provide the pivoting movement of the instrument shaft at the hole site. The main joint controller of the first module 157 may include the filter F(X). Conceptually, (1-F-1F)(X) can describe configuration-related subspace filters that give control actuation authority for linear combinations of joint speeds orthogonal to the goal of achieving the main joint controller. This configuration-related filter can be used by the second module 160 of the controller 157 to achieve the second goal. The two filters can be further subdivided into more filters corresponding to more specific tasks. The filter F(X) can be divided into F1(X) and F2(X), which are used to control the end effector and control the movement of the pivot axis respectively, any of which can be selected as the highest priority task of the processor.
机器人处理器和控制技术将经常利用经配置用于第一控制器任务的主要关节控制器，以及配置相关过滤器，该配置相关过滤器利用由主要关节控制器生成的欠约束解以用于第二任务。参考第一模块描述主要关节控制器，而将参考第二模块描述配置相关过滤器，还能够包括附加功能和各种优先级的附加模块。第一模块和第二模块功能的硬件和编程代码，是完全集成、部分集成能够完全分离的。控制器157能够同时使用两个模块的功能，能够具有多种不同模式，其中一个或两个模块分开使用或以不同的方式使用。在主-从操纵期间，第一模块157能够在很少或没有来自第二模块160的影响的情况下使用，并且当末端执行器未被机器人驱动时，第二模块160在系统装配期间具有更大的作用，两个模块均能够在启用机器人运动的大部分或全部时间是活动的，通过将第一模块的增益设置为零，通过将X s设置为x s.actual和通过降低逆雅可比控制器中的矩阵秩使其不能控制太多并使配置相关过滤器具有更多控制权限，能够减少或消除第一模块对操纵器组件的状态的影响，从而将处理器157的模式变为抓持模式。 The robot processor and control technology will often utilize the primary joint controller configured for the first controller task, as well as configuration related filters that utilize the underconstrained solution generated by the primary joint controller for the first controller task. Two tasks. The main joint controller will be described with reference to the first module, and the configuration related filters will be described with reference to the second module, which can also include additional functions and additional modules of various priorities. The hardware and programming codes of the first module and the second module are fully integrated, and partly integrated and can be completely separated. The controller 157 can use the functions of two modules at the same time, and can have a variety of different modes, in which one or two modules are used separately or in different ways. During master-slave manipulation, the first module 157 can be used with little or no influence from the second module 160, and when the end effector is not driven by the robot, the second module 160 has more features during system assembly. The big effect is that both modules can be active most or all of the time when the robot movement is enabled, by setting the gain of the first module to zero, by setting X s to x s.actual and by reducing the inverse Jacobian The matrix rank in the controller makes it impossible to control too much and allows the configuration-related filters to have more control authority, which can reduce or eliminate the influence of the first module on the state of the manipulator component, thereby changing the mode of the processor 157 to capture Support mode.
在图39中，第一模块157能够包含具有雅可比相关矩阵的某种形式的雅可比控制器。在端口抓持模式下，第二模块160能够从从操纵器158接收信号，该信号指示至少部分地由从操纵器联动装置的手动铰接产生的从动件的方位或速度。 响应于该输入，第二模块160能够产生适于驱动从动件的关节的动力命令，以便允许从联动装置的手动铰接，同时配置处于期望的关节配置的从动件。在主-从末端执行器操纵期间，控制器能够使用第二模块160来帮助基于不同的信号bqdot o导出动力命令。这种到控制器157的第二模块160的替代输入信号可用于驱动操纵器联动装置，以便沿着操纵器结构维持或移动微创孔枢轴位置，从而避免多个操纵器之间的碰撞，从而增加操纵器结构的运动范围和避免奇点，以便产生操纵器的期望姿势等。 In Figure 39, the first module 157 can contain some form of Jacobian controller with a Jacobian correlation matrix. In the port gripping mode, the second module 160 can receive a signal from the slave manipulator 158 that indicates the orientation or speed of the follower at least in part resulting from the manual articulation of the slave manipulator linkage. In response to this input, the second module 160 can generate a power command suitable for driving the joint of the follower in order to allow manual articulation of the slave linkage while configuring the follower in a desired joint configuration. During the master-slave end effector manipulation, the controller can use the second module 160 to help derive power commands based on different signals bqdot o . This alternative input signal to the second module 160 of the controller 157 can be used to drive the manipulator linkage to maintain or move the minimally invasive hole pivot position along the manipulator structure, thereby avoiding collisions between multiple manipulators, Thereby increasing the range of motion of the manipulator structure and avoiding singularities in order to produce the desired posture of the manipulator.
在图41中，使用来自MTM控制器的输入主动控制远程运动中心(RC)、手臂端197(C)和器械末端执行器(E)参照系的框图231。In Fig. 41, the input from the MTM controller is used to actively control a block diagram 231 of the remote motion center (RC), arm end 197 (C), and instrument end effector (E) reference frame.
在图42中，使用来自主操纵器控制器的输入主动控制器械末端执行器(E)系，同时使用次要输入装置控制远程中心(RC)和手臂端197(C)系的框图232。次要输入装置使用任意参照，而不一定是目系(EYE系)。参照系变换EYETREF能够被直接测量或由间接测量值计算。信号调节单元将这些输入组合在适当的公共系中，以供从操纵器控制器使用。In Figure 42, the instrument end effector (E) system is actively controlled using input from the primary manipulator controller, while the secondary input device is used to control the block diagram 232 of the remote center (RC) and arm end 197 (C) system. The secondary input device uses arbitrary references, not necessarily the target system (EYE system). The reference frame transformation EYETREF can be directly measured or calculated from indirect measurements. The signal conditioning unit combines these inputs in an appropriate common system for use by the slave manipulator controller.
在图42、图43和图44中，有三个待由系统的控制器控制的参照系，参照系之一(C)是手臂端197的参照系，假设EYETE由主工具操纵器(MTM)控制器命令。远程中心参照系和手臂端197参照系的姿势规范来自以下来源中的一个或组合：(i)MTM控制器指定这些系/参照系在EYE系中，即 EYET RC和 EYET C，(ii)次要装置命令这些系姿势在方便的参照系中，即 REFT RC和 REFT C(其中能够确定 EYET REF)，以及(iii)从侧控制器指定这些姿势在从臂的基系(base frame)中，即 WT RC和 WT C。 In Figure 42, Figure 43 and Figure 44, there are three reference frames to be controlled by the controller of the system. One of the reference frames (C) is the reference frame of the arm end 197. It is assumed that the EYETE is controlled by the master tool manipulator (MTM)器command. The posture specifications of the remote center frame of reference and the arm end 197 frame of reference come from one or a combination of the following sources: (i) The MTM controller designates these frames/frames in the EYE system, namely EYE T RC and EYE T C , (ii ) The secondary device commands these postures in a convenient reference frame, namely REF T RC and REF T C (where EYE T REF can be determined), and (iii) the slave side controller assigns these postures to the base system of the slave arm ( base frame), namely W T RC and W T C.
在图45和图46是系统212和213的示意框图，系统212和213用于使用计算机辅助的车辆远程驾驶体系258的第二处理器215来控制器械末端执行器193参照系和远程控制中心298参照系之间的关系。假设手臂端197参照系和远程控制中心298参照系重合。手臂端197参照系和远程控制中心298在物理上被约束为相对于器械末端执行器193仅沿着手臂端197和器械的纵向轴线移动。采用两种不同的策略，以控制器械末端执行器193的参照系(E系)和远程控制中心298的参照系(RC系)之间的关系。用于主动控制两个参照系无论E系是固定的还是移动的之间的相对距离(d)的一种策略使用来自力/扭矩传感器或三态开关的输入。使用框图来实现用于该模式的控制子系统，该控制子系统能够被描述为‘相对姿势控制器’。45 and 46 are schematic block diagrams of the systems 212 and 213. The systems 212 and 213 are used to use the second processor 215 of the computer-aided vehicle remote driving system 258 to control the instrument end effector 193 reference frame and the remote control center 298 The relationship between frames of reference. It is assumed that the reference frame of the arm end 197 and the reference frame of the remote control center 298 coincide. The arm end 197 frame of reference and the remote control center 298 are physically constrained to move relative to the instrument end effector 193 only along the arm end 197 and the longitudinal axis of the instrument. Two different strategies are adopted to control the relationship between the frame of reference (E system) of the instrument end effector 193 and the frame of reference (RC system) of the remote control center 298. One strategy for actively controlling the relative distance (d) between the two frames of reference whether the E frame is fixed or moving uses input from a force/torque sensor or a three-state switch. The block diagram is used to implement the control subsystem for this mode, which can be described as a'relative posture controller'.
在图45中，使用三态开关相对控制距尖端的距离的一般框图。对从操纵器的增量笛卡尔命令“slv_cart_delta”表示如下：slv_cart_delta＝S*slv_cart_vel*Ts(其中Ts是控制器的采样时间)。S取值[1，-1，0]，这取决于它命令控制方向盘235、档位400、刹车踏板401和油门踏板402的运动。In Figure 45, a general block diagram of the relative control of the distance from the tip using a three-state switch. The incremental Cartesian command "slv_cart_delta" for the slave manipulator is expressed as follows: slv_cart_delta=S*slv_cart_vel*Ts (where Ts is the sampling time of the controller). S takes the value [1, -1, 0], which depends on its commands to control the movement of the steering wheel 235, the gear 400, the brake pedal 401, and the accelerator pedal 402.
图46是力/扭矩或压力传感器相对控制手臂端197距尖端的距离的一般框图，对从操纵器的增量笛卡尔命令“slv_cart_delta”能够表示如下：slv_cart_delta＝f(F，p)“f”是使用感测到的力或压力F和一些用户定义的参数p作为输入的可编程函数。这是一个导纳控制器。这能够通过借助于关节扭矩传感器和臂运动学知识估计沿手臂端197轴线的笛卡尔力来实现。在这种估计之后，计算的估计值能够用作输入F以命令增量运动，信号F能够是基于用户与操纵器的交互的任何其他测量或计算的力的量。能够独立地控制RC系和E系的轨迹，其中管理这些轨迹的控制输入可能都来自主操纵器，该附加策略的控制子系统框图能够被称为‘独立姿势控制器’，能够概述插入(I/O)运动以允许远程控制中心298或手臂端197相对于器械尖端E的横向运动。远程控制中心298或手臂端197将需要围绕尖端枢转，同时驱动器械以补偿E的运动。这将允许RC和手臂端197在驾驶室内的运动。Figure 46 is a general block diagram of the distance between the force/torque or pressure sensor and the tip of the control arm 197. The incremental Cartesian command "slv_cart_delta" of the slave manipulator can be expressed as follows: slv_cart_delta=f(F,p)"f" It is a programmable function that uses the sensed force or pressure F and some user-defined parameters p as input. This is an admittance controller. This can be achieved by estimating the Cartesian force along the axis of the arm end 197 with the aid of joint torque sensors and arm kinematics knowledge. After this estimation, the calculated estimation value can be used as input F to command incremental movement, and the signal F can be any other measured or calculated amount of force based on the user's interaction with the manipulator. It can independently control the trajectories of RC system and E system, where the control input to manage these trajectories may all come from the main manipulator. The block diagram of the control subsystem of this additional strategy can be called an'independent posture controller', which can be summarized as inserted (I /O) Movement to allow lateral movement of the remote control center 298 or arm end 197 relative to the instrument tip E. The remote control center 298 or arm end 197 will need to pivot about the tip while driving the instrument to compensate for the movement of E. This will allow movement of the RC and arm end 197 in the cab.
图47-图52是提供远程系统的故障反应、故障隔离和故障弱化的方法，机器人170的组件协同交互以执行在机器人170中的故障反应、故障隔离和故障弱化的各种方面。第一机械手182、第二机械手183、第三机械手184和第四机械手185均包括多个节点。每个节点控制多个马达，该马达驱动在机械手臂中的接头和连杆机构以影响机械手臂的运动自由度，每个节点也控制用于停止马达旋转的多个制动器。第一机械手182具有马达307、309、311和313；多个制动器308、310、312和314以及多个节点315、316和317，每个节点315，316控制单个马达/制动器对；节点317控制两个马达/制动器对，传感器处理单元318被包括以提供马达位移传感器信息到节点317用于控制目的，第二机械手183、第三机械手184、第四机械手185与第一机械手182类似被配置为具有马达、制动器和节点。Figures 47-52 are methods for providing fault response, fault isolation, and fault weakening of the remote system. The components of the robot 170 cooperate and interact to perform various aspects of fault reaction, fault isolation and fault weakening in the robot 170. The first robot 182, the second robot 183, the third robot 184, and the fourth robot 185 each include a plurality of nodes. Each node controls multiple motors, which drive the joint and linkage mechanism in the robotic arm to affect the freedom of movement of the robotic arm, and each node also controls multiple brakes for stopping the rotation of the motor. The first manipulator 182 has motors 307, 309, 311, and 313; multiple brakes 308, 310, 312, and 314, and multiple nodes 315, 316, and 317, each node 315, 316 controls a single motor/brake pair; node 317 controls Two motor/brake pairs, the sensor processing unit 318 is included to provide motor displacement sensor information to the node 317 for control purposes. The second manipulator 183, the third manipulator 184, and the fourth manipulator 185 are similarly configured to the first manipulator 182. With motors, brakes and nodes.
每个机械手臂操作地耦连到手臂处理器。手臂处理器328操作地耦连到第一机械手182的节点，手臂处理器325操作地耦连到第二机械手183的节点，手臂处理器323操作地耦连到第三机械手184的节点，以及手臂处理器321操作地耦连到第四机械手185的节点。每个手臂处理器还包括接头位置控制器，用于将其操作地耦连的机械手臂的期望接头位置转变为用于驱动其操作地耦连的机械手臂中的马达的电流命令以将其相应的接头驱动到期望的接头位置。系统管理处理器320操作地耦连到手臂处理器328，325，323，321；系统管理处理器320还将与机械手臂相关联的用户操纵输入装置平移到期望的接头位置尽管示为分离单元，手臂处理器328，325，323，321也通过程序代码被实施为系统管理处理器320的一部分。手臂管理处理器319操作地耦连到系统管理处理器320和手臂处理器328、325、323、321。该手臂管理处理器319开始、控制和监控手臂的某些协调活动以便于使系统管理处理器320免于必须这样做。手臂管理器319也通过程序代码被实施为系统管理处理器320的一部分。处理器和节点中的每个被配置成通过硬件、固件和软件编程的任一组合来执行本文的各种任务。它们的功能通过一个单元来被执行或者在许多子单元之中被分配，每个子单元又通过硬件、固件和软件编程的任一组合来实施。该系统管理处理器320被分配为遍及机器人170的子单元，比如远程控制台169、以及机器人170的基座173。系统管理处理器320、手臂管理处理器319以及每个手臂处理器328，325，323，321包括多个处理器以执行各种处理器和控制器任务和功能。Each robotic arm is operatively coupled to the arm processor. The arm processor 328 is operatively coupled to the node of the first manipulator 182, the arm processor 325 is operatively coupled to the node of the second manipulator 183, the arm processor 323 is operatively coupled to the node of the third manipulator 184, and the arm The processor 321 is operatively coupled to the node of the fourth manipulator 185. Each arm processor also includes a joint position controller for converting the desired joint position of the operatively coupled robot arm into a current command for driving the motor in the operatively coupled robot arm to adjust it accordingly The joint is driven to the desired joint position. The system management processor 320 is operatively coupled to the arm processors 328, 325, 323, 321; the system management processor 320 also translates the user manipulation input device associated with the robotic arm to the desired joint position although shown as a separate unit, The arm processors 328, 325, 323, and 321 are also implemented as part of the system management processor 320 through program codes. The arm management processor 319 is operatively coupled to the system management processor 320 and the arm processors 328, 325, 323, 321. The arm management processor 319 initiates, controls, and monitors certain coordinated activities of the arm in order to save the system management processor 320 from having to do so. The arm manager 319 is also implemented as a part of the system management processor 320 through program code. Each of the processor and the node is configured to perform various tasks herein through any combination of hardware, firmware, and software programming. Their functions are executed by one unit or distributed among many subunits, and each subunit is implemented by any combination of hardware, firmware, and software programming. The system management processor 320 is allocated as sub-units throughout the robot 170, such as the remote console 169 and the base 173 of the robot 170. The system management processor 320, the arm management processor 319, and each arm processor 328, 325, 323, 321 include multiple processors to perform various processor and controller tasks and functions.
每个节点和传感器处理单元包括发射器/接收器(TX/RX)对，以便于与其机械手臂的其它节点和操作地耦连到其机械手臂的手臂处理器进行通信。TX/RX以菊花链连入网络。在这种菊花链布置中，当每个节点的RX接收来自邻近节点的TX的信息包时，它核实在该包中的目的地字段以确定是否该包用于其节点。该包用于其节点，则该节点处理该包。该包用于另一节点，则节点的TX在与它所来之处的相反方向上传递所接收包到邻近节点的RX。通过使用包交换协议(packet-switching protocol)，信息以包的形式在菊花链网络上传递。故障反应逻辑(FRL)线路被提供在每个机械手臂中，故障通知通过手快速传送。第一机械手182包括与手臂处理器328和机械手臂315的节点315、316和317中的每个耦连的FRL线路。当手臂处理器328和节点315，316和317中的一个检测到影响它的故障时，该手臂处理器或节点拉高FRL线路329以快速传送故障通知到与该线路329耦连的其它组件。相反地，当该手臂处理器328将要传输恢复通知到 第一机械手182的节点时，它拉低FRL线路329以快速传送该恢复通知到与该线路329耦连的其它组件。虚拟FRL线路329通过指定包中的一个或更多个字段包括这样的故障通知和恢复通知而被代替地使用。Each node and sensor processing unit includes a transmitter/receiver (TX/RX) pair to facilitate communication with other nodes of its robotic arm and an arm processor operatively coupled to its robotic arm. TX/RX are connected to the network in a daisy chain. In this daisy chain arrangement, when the RX of each node receives an information packet from the TX of a neighboring node, it verifies the destination field in the packet to determine whether the packet is for its node. If the packet is used for its node, the node processes the packet. If the packet is used for another node, the node's TX transfers the received packet to the neighboring node's RX in the opposite direction to where it came from. By using the packet-switching protocol, information is transmitted in the form of packets on the daisy chain network. The fault response logic (FRL) circuit is provided in each robot arm, and the fault notification is quickly transmitted by hand. The first manipulator 182 includes an FRL circuit coupled to each of the nodes 315, 316, and 317 of the arm processor 328 and the manipulator 315. When the arm processor 328 and one of the nodes 315, 316, and 317 detects a fault affecting it, the arm processor or node pulls up the FRL line 329 to quickly transmit a fault notification to other components coupled to the line 329. Conversely, when the arm processor 328 is about to transmit the recovery notification to the node of the first manipulator 182, it pulls down the FRL line 329 to quickly transmit the recovery notification to other components coupled to the line 329. The virtual FRL line 329 is used instead by specifying that one or more fields in the packet include such failure notification and recovery notification.
在图48中，在327中，该方法检测在多个机械手臂的失效手臂中的故障，其中由于所检测的故障该机械手臂变成“失效手臂”。在328中，该方法然后将该失效手臂置入安全状态，其中“安全状态”指的是通过阻止手臂的进一步运动来隔离所检测故障的失效手臂的状态。在329中，该方法确定该故障应被看作系统故障还是局部故障，其中“系统故障”指的是影响多个机械手臂中的至少一个其他的机械手臂的性能的故障，并且“局部故障”指的是影响仅失效手臂的性能的故障。因为局部故障导致仅失效手臂被保持在安全状态下直到该故障被清除，所以它不是导致非失效机械手臂的不安全操作的故障类型。该故障是导致非失效手臂的不安全操作的类型，则该方法应产生所检测故障是系统故障的确定，其中在系统中的所有机械手臂将被置入安全状态。在330中，该方法在仅当该故障将被看作系统故障时将多个手臂中的非失效手臂置入安全状态，其中“非失效手臂”指的是在其中没有故障已经被检测到的多个机械手臂中的机械手臂。在331中，该方法确定所检测故障被分类为可恢复系统故障还是不可恢复系统故障。在332中，该故障被分类为可恢复系统故障，则该方法提供给系统用户恢复选项。在333中，该故障被分类为不可恢复系统故障，则该方法等待系统关闭。在329中的确定是该故障将被看作局部故障，然后在334中，该方法确定该故障被分类为可恢复局部故障还是不可恢复局部故障。在335中，该故障被分类为可恢复局部故障，则该方法提供给系统用户恢复选项和弱化操作选项。在336中，该故障被分类为不可恢复局部故障，该方法仅提供弱化操作选项。In FIG. 48, in 327, the method detects a failure in a failed arm of a plurality of robot arms, where the robot arm becomes a "failed arm" due to the detected failure. In 328, the method then puts the failed arm into a safe state, where the "safe state" refers to the state of isolating the detected malfunctioning arm by preventing further movement of the arm. In 329, the method determines whether the failure should be regarded as a system failure or a partial failure, where "system failure" refers to a failure that affects the performance of at least one other of the multiple robot arms, and "partial failure" Refers to a malfunction that affects the performance of only the failed arm. Because a local fault causes only the failed arm to be kept in a safe state until the fault is cleared, it is not a type of fault that causes unsafe operation of a non-failed mechanical arm. The fault is the type that causes the unsafe operation of the non-failure arm, then the method should produce a determination that the detected fault is a system fault, in which all the robot arms in the system will be placed in a safe state. In 330, the method puts the non-failed arm of the multiple arms into a safe state only when the fault will be regarded as a system failure, where the "non-failed arm" refers to the one in which no failure has been detected The robotic arm among multiple robotic arms. In 331, the method determines whether the detected fault is classified as a recoverable system fault or an unrecoverable system fault. In 332, the fault is classified as a recoverable system fault, and the method provides the system user with recovery options. In 333, the fault is classified as an unrecoverable system fault, and the method waits for the system to shut down. The determination in 329 is that the fault will be treated as a partial fault, and then in 334, the method determines whether the fault is classified as a recoverable partial fault or an unrecoverable partial fault. In 335, the fault is classified as a recoverable partial fault, and the method provides the system user with recovery options and weakened operation options. In 336, the fault is classified as an unrecoverable partial fault, and the method only provides a weakened operation option.
图47是执行故障反应、故障隔离和故障弱化的方法的方面的流程图，该故障反应、故障隔离和故障弱化通过机器人170的多个机械手臂的每个节点315、316和317被执行。在337中，每个节点持续地监控在该节点中的信号和信息以使用常规故障检测方法来检测影响节点的故障。这种类型的所检测故障在本文被称为“局部故障”，因为它局限于节点。该节点也针对由其手臂处理器或其机械手臂内另一节点发出的故障通知监控FRL线路。这种类型的所检测故障在本文被称为“远程故障”，因为它不局限于该节点。所检测的故障是硬件、固件、软件、环境或相关的通信。其中故障已经被检测到的节点在本文被称为“失效节点”，它的机械手臂在本文被称为“失效手臂”。其中没有故障已经被检测到的节点在本文被称为“非失效节点”，其中没有故障已经被检测的机械手臂在本文被称为“非失效手臂”。FIG. 47 is a flowchart of aspects of a method of performing fault reaction, fault isolation, and fault weakening, which are performed by each node 315, 316, and 317 of the multiple robot arms of the robot 170. In 337, each node continuously monitors the signals and information in the node to use conventional fault detection methods to detect faults affecting the node. This type of detected fault is referred to herein as a "local fault" because it is limited to nodes. The node also monitors the FRL circuit for fault notifications sent by its arm processor or another node in its robotic arm. This type of detected fault is referred to herein as a "remote fault" because it is not limited to the node. The detected fault is hardware, firmware, software, environment, or related communications. The node where the fault has been detected is called "failed node" in this article, and its robotic arm is called "failed arm" in this article. A node in which no fault has been detected is referred to herein as a "non-failure node", and a mechanical arm in which no fault has been detected is referred to herein as a "non-failure arm".
在338中，在337中检测到故障，则节点将其自身置入安全状态。这通过停用该节点的一个或更多个受控马达来完成，这通过接合该节点的一个或多个受控制动器来完成。在339中，该节点确定所检测的故障是局部故障还是远程故障。如之前参照337，故障的来源确定它将被看作局部故障还是远程故障。该故障被确定为局部故障，则该节点是失效节点。在第一种情况下，该失效节点通过执行如下的343-346和341-342而持续。该故障被确定为远程故障，则该节点是非失效节点。在第二种情况下，非失效节点通过执行如下的340-342而持续。在343中，该失效节点在失效机械手臂中在上游方向和下游方向上传送故障通知到邻近节点。该“下游”方向指的是信息包远离该节点的手臂处理器行进的方向并且“上游”方向指的是信息包朝向该节点的手臂处理器行进的方向，节点完成该过程的一个方式是通过将FRL线路拉到高状态。In 338, a failure is detected in 337, and the node puts itself into a safe state. This is done by deactivating one or more controlled motors of the node, which is done by engaging one or more controlled brakes of the node. In 339, the node determines whether the detected fault is a local fault or a remote fault. As previously referred to 337, the source of the fault determines whether it will be regarded as a local fault or a remote fault. If the failure is determined to be a partial failure, the node is a failed node. In the first case, the failed node continues by executing the following 343-346 and 341-342. If the failure is determined to be a remote failure, the node is a non-failed node. In the second case, the non-failed node continues by performing the following 340-342. In 343, the failed node transmits a failure notification to neighboring nodes in the failed robot arm in the upstream and downstream directions. The "downstream" direction refers to the direction in which the packet travels away from the arm processor of the node and the "upstream" direction refers to the direction the packet travels toward the arm processor of the node. One way for a node to complete the process is through Pull the FRL line to the high state.
在344中，该失效节点然后诊断故障并且发送错误消息到系统管理处理器320。该错误消息优选包括故障信息，它的错误代码、错误类和起源(origin)。可能发生的会影响到节点的每种错误类型都被分配错误代码。该错误代码被分类为错误类。有至少四种错误类：可恢复手臂故障、不可恢复手臂故障、可恢复系统故障以及不可恢复系统故障。使用“可恢复”意味着用户被提供有尝试从故障中恢复的选项。使用“不可恢复”意味着用户不被提供有尝试从故障中恢复的选项。故障的起源包括节点的身份的信息以及节点内故障的来源的可选附加信息。在345中，失效节点确定所检测故障是否为可恢复局部故障。在345中的确定为否，则在346中，该失效节点保持在它的安全状态下并且忽视它可能在FRL线路上随后接收的任何恢复通知。在345中的确定为是则该失效节点行进到341。在339中的确定是所检测故障将被看作远程故障，则在340中，虚拟FRL线路被使用，则该非失效节点在与从故障通知所来之处的相反方向上传送所接收故障通知，在真实FRL线路的情况下，非失效节点不需要对故障通知的这种传送采取任何行动。在341中，失效节点和非失效节点两者等待将被接收的恢复通知。在342中，一旦恢复通知被接收，则节点将它自身从安全状态返回到它的正常操作状态。这通过颠倒在338中所采取的行动同时避免突然变化来完成。该节点返回以执行参照337的故障检测任务。In 344, the failed node then diagnoses the fault and sends an error message to the system management processor 320. The error message preferably includes failure information, its error code, error type, and origin. Each type of error that may occur that affects the node is assigned an error code. The error code is classified as an error class. There are at least four types of errors: recoverable arm failure, unrecoverable arm failure, recoverable system failure, and unrecoverable system failure. The use of "recoverable" means that the user is provided with the option to try to recover from the failure. Using "unrecoverable" means that the user is not provided with the option to try to recover from the failure. The origin of the fault includes information about the identity of the node and optional additional information about the source of the fault in the node. In 345, the failed node determines whether the detected fault is a recoverable partial fault. The determination in 345 is no, then in 346, the failed node remains in its safe state and ignores any recovery notifications it may subsequently receive on the FRL line. If the determination in 345 is yes, the failed node proceeds to 341. The determination in 339 is that the detected fault will be regarded as a remote fault, then in 340, the virtual FRL line is used, and the non-failed node transmits the received fault notification in the opposite direction from where the fault notification came from In the case of a real FRL line, the non-failure node does not need to take any action for this transmission of the failure notification. In 341, both the failed node and the non-failed node wait for the recovery notification to be received. In 342, once the recovery notification is received, the node returns itself from the safe state to its normal operating state. This is done by reversing the actions taken in 338 while avoiding sudden changes. The node returns to perform the fault detection task referred to in 337.
在图47-图52是执行故障反应、故障隔离和故障弱化的方法的各方面的流程图，该故障反应、故障隔离和故障弱化通过每个手臂处理器321、323、325和328被执行，该每个手臂处理器操作地耦连到机器人170的机械手臂。在347中，每个手臂处理器在执行它的正常操作任务时，也持续监控它自身操作并且注意在其操作地耦连的机械手臂中的失效节点所传送的故障通知。在监控它自身操作时故障被检测到，则该故障在本文被称为“局部故障”。故障通过接收到来自在其操作地耦连的机械手臂中的失效节点的故障通知而被检测，则该故障被称为“远程故障”。远程故障是通过手臂处理器操作地耦连到的机械手臂中的失效节点沿着FRL线路传送的故障通知。在347中故障已经被检测，在车辆视觉系统502中，手臂处理器通过将接头位置控制器的输出马达电流命令锁定到零来将它的接头位置控制器置入安全状态。这用于加强它们相应节点的安全状态。在349中，手臂处理器确定所检测故障是局部故障还是远程故障。在347中，故障的来源确定了故障将被看作局部故障还是远程故障。该故障被确定为局部故障，则手臂处理器被看作失效节点。手臂处理器通过执行如下353-356和351-352而持续.故障被确定为远程故障，则手臂处理器被看作非失效节点，手臂处理器通过执行350-352而持续。在353中，手臂处理器将故障通知向下游传送到其操作地连接的机械手臂的所有节点。该手臂处理器完成该过程的一个方式是通过将FRL线路拉到高状态。在354中，手臂处理器诊断故障并发送错误消息到系统管理处理器320。该错误消息包括故障信息，错误代码、错误类以及起源。发生的会影响到手臂处理器的每种错误类型都被分配错误代码。该错误代码被分类为错误类，有至少四种错误类：可恢复处理器故障、不可恢复处理器故障、可恢复系统故障以及不可恢复系统故障。故障的起源包括手臂处理器的身份的信息以及手臂处理器中故障的来源的可选附加信息。在355中，手臂处理器确定所检测故障是否为可恢复局部故障，通过故障的错误类来完成该确定，在355中的确定为否，则在356中，该失效手臂处理器的接头位置控制器保持在其安全状态并且手臂处理器忽视其可能在FRL线路上随后接收的任何恢复通知，在355中的确定为是，则该手臂处理器行进到350。在349中的确定是所检测故障将被看作远程故障，则在350中，手臂处理器等待从系统管理处理器320接收的恢复通知。在351中，一旦恢复通知被接收，则手臂处理器通过例 如将它的FRL线路拉低来传送该恢复通知到在其操作地耦连的机械手臂中的所有节点。在352中，手臂处理器然后将它的接头位置控制器从安全状态返回到它的正常操作状态。该过程通过释放接头位置控制器的输出马达电流命令来完成，以便它们可再一次反映其操作地耦连的机械手臂的期望接头位置同时避免突然的变化。该手臂处理器然后返回以执行参照347的它的故障检测任务。47-52 are flowcharts of various aspects of the methods for performing fault reaction, fault isolation and fault weakening. The fault reaction, fault isolation and fault weakening are executed by each arm processor 321, 323, 325 and 328, The each arm processor is operatively coupled to the robotic arm of the robot 170. In 347, each arm processor continuously monitors its own operation while performing its normal operation tasks and pays attention to the failure notifications transmitted by the failed node in the robot arm to which it is operatively coupled. When a fault is detected while monitoring its own operation, the fault is referred to herein as a "local fault". A fault is detected by receiving a fault notification from a failed node in the robotic arm to which it is operatively coupled, and the fault is called a "remote fault." The remote fault is a fault notification sent along the FRL line by the failed node in the robotic arm operatively coupled to the arm processor. In 347, the fault has been detected. In the vehicle vision system 502, the arm processor puts its joint position controller into a safe state by locking the joint position controller's output motor current command to zero. This is used to strengthen the security status of their corresponding nodes. In 349, the arm processor determines whether the detected fault is a local fault or a remote fault. In 347, the source of the fault determines whether the fault will be considered a local fault or a remote fault. The fault is determined to be a partial fault, and the arm processor is regarded as a failed node. The arm processor continues by executing the following 353-356 and 351-352. The fault is determined as a remote fault, the arm processor is regarded as a non-failure node, and the arm processor continues by executing 350-352. In 353, the arm processor transmits the failure notification downstream to all nodes of the robotic arm to which it is operatively connected. One way for the arm processor to complete this process is by pulling the FRL line to a high state. In 354, the arm processor diagnoses the fault and sends an error message to the system management processor 320. The error message includes fault information, error code, error type, and origin. Each type of error that occurs that affects the arm processor is assigned an error code. The error code is classified as an error type, and there are at least four error types: recoverable processor failure, unrecoverable processor failure, recoverable system failure, and unrecoverable system failure. The origin of the fault includes information on the identity of the arm processor and optional additional information on the source of the fault in the arm processor. In 355, the arm processor determines whether the detected fault is a recoverable partial fault, and the determination is completed by the error category of the fault. If the determination in 355 is no, then in 356, the joint position of the failed arm processor is controlled The arm processor remains in its safe state and the arm processor ignores any recovery notifications it may subsequently receive on the FRL line. The determination in 355 is yes, then the arm processor proceeds to 350. The determination in 349 is that the detected fault will be regarded as a remote fault, then in 350, the arm processor waits for the recovery notification received from the system management processor 320. In 351, once the recovery notification is received, the arm processor transmits the recovery notification to all nodes in the robotic arm to which it is operatively coupled by, for example, pulling its FRL line low. In 352, the arm processor then returns its joint position controller from the safe state to its normal operating state. This process is accomplished by releasing the output motor current command of the joint position controller so that they can once again reflect the desired joint position of the robotic arm to which they are operatively coupled while avoiding sudden changes. The arm processor then returns to perform its fault detection task with reference to 347.
在图47和图51是执行故障反应、故障隔离和故障弱化的方法的各方面的流程图，该故障反应、故障隔离和故障弱化通过机器人170的系统管理处理器320被执行。在357中，系统管理处理器在执行它的正常操作任务时也等待以接收从机器人170的另一组件所传输的错误消息。错误消息在357被接收，则在358中，系统管理处理器为了安全目的通过例如命令机器人系统中的所有手臂处理器328、325、323和321的接头位置控制器以它们的电流值锁定它们各自的输出来停止该系统。没有新的电流命令输入被提供到机械手臂，直到接头位置控制器的输出被解锁。接头位置控制器的输出的这种锁定在本文被称为“软锁”接头位置控制器。该方法然后行进到359。在359中，该系统管理处理器确定所检测故障应被看作系统故障还是手臂故障。该系统管理处理器通过检查在错误消息中所提供的错误类信息来完成该步骤。系统故障包括被分类为或者可恢复系统故障或者不可恢复系统故障的所有故障，因为这些故障可应用于不只是失效机械手臂。相反地，手臂故障包括被分类为或者可恢复局部故障或者不可恢复局部故障的所有故障，因为这些故障仅可应用于失效机械手臂。针对将被看作手臂故障的所有故障，执行360-366。在360中，该系统管理处理器给机器人170的远程操作员171提供接受机器人170的弱化操作的选项。该局部故障是可恢复局部故障，则该系统管理处理器还给用户提供从故障中恢复的选项。除了每个所提供的选项之外，所检测故障的信息也通过系统管理处理器提供，以辅助远程操作员171确定是否接受该选项。该选项和故障信息被提供在远程控制台169的视觉显示器255上。47 and 51 are flowcharts of various aspects of methods for performing fault reaction, fault isolation, and fault weakening, which are executed by the system management processor 320 of the robot 170. In 357, the system management processor also waits to receive an error message transmitted from another component of the robot 170 while performing its normal operation tasks. The error message is received in 357, then in 358, the system management processor for safety purposes, for example, by instructing the joint position controllers of all arm processors 328, 325, 323, and 321 in the robot system to lock their respective current values To stop the system. No new current command input is provided to the robotic arm until the output of the joint position controller is unlocked. This locking of the output of the joint position controller is referred to herein as a "soft lock" joint position controller. The method then proceeds to 359. In 359, the system management processor determines whether the detected fault should be regarded as a system fault or an arm fault. The system management processor completes this step by checking the error type information provided in the error message. System faults include all faults classified as either recoverable system faults or unrecoverable system faults, because these faults can be applied to more than just failed robots. Conversely, arm faults include all faults classified as either recoverable partial faults or unrecoverable partial faults, because these faults can only be applied to a failed robot arm. Perform 360-366 for all faults that will be regarded as arm faults. In 360, the system management processor provides the remote operator 171 of the robot 170 with the option of accepting the weakened operation of the robot 170. If the partial failure is a recoverable partial failure, the system management processor also provides the user with an option to recover from the failure. In addition to each provided option, information about the detected failure is also provided by the system management processor to assist the remote operator 171 in determining whether to accept the option. The option and fault information are provided on the visual display 255 of the remote console 169.
在361中，该系统管理处理器等待远程操作员171选取在360中所提供的选项。一旦选项被远程操作员171选取，则在362中，该系统管理处理器确定所选取的选项是弱化操作选项还是恢复选项。恢复选项被提供并且该远程操作员171选取恢复选项，则在381中，该系统管理处理器发送恢复通知到失效机械手臂的手臂处理器。该失效机械手臂的手臂处理器将处理该恢复通知，包括传送恢复通知到失效手臂的所有节点，失效手臂的节点然后处理该恢复通知。在382中，系统管理处理器然后通过解锁所有手臂处理器的接头控制器的输出而释放接头控制器的软锁，以便接头控制器再次发出反映它们的操作地耦连的机械手臂的期望接头位置的马达电流命令然后，该系统管理处理器返回执行参照357的它的任务。In 361, the system management processor waits for the remote operator 171 to select the option provided in 360. Once the option is selected by the remote operator 171, in 362, the system management processor determines whether the selected option is a weakened operation option or a recovery option. The recovery option is provided and the remote operator 171 selects the recovery option, then in 381, the system management processor sends a recovery notification to the arm processor of the failed robotic arm. The arm processor of the failed robotic arm will process the recovery notification, including sending the recovery notification to all nodes of the failed arm, and the node of the failed arm will then process the recovery notification. In 382, the system management processor then releases the soft locks of the joint controllers by unlocking the outputs of the joint controllers of all arm processors, so that the joint controllers again issue the desired joint position reflecting their operationally coupled robotic arms Then, the system management processor returns to perform its task with reference to 357.
远程操作员171选取弱化操作选项，则在363中，该系统管理处理器给远程操作员171提供从故障中恢复的选项。在这种情况下从故障中恢复不同于参照381-382的恢复，因为没有尝试恢复失效手臂。恢复仅应用于恢复非失效手臂的正常操作。在364中，该系统管理处理器等待用户选取在363中所提供的选项。一旦选项被远程操作员171选取，则在365中，该系统管理处理器发送消息到失效手臂的手臂处理器以加强该故障。故障的加强在这种情况下意味着附加步骤被采取以完全关闭失效机械手臂的操作。这种加强措施的一个示例是从主/从控制系统的其它部件操作地断开手臂处理器的接头位置控制器，该主/从控制系统产生其操作地耦连的机械手臂的期望接头位置另一加强措施是关闭到失效机械手臂的电源。在366中，该系统管理处理器随然后通过解锁所有非失效手臂的手臂处理器的接头控制器的输出来释放接头控制器的软锁，以便接头控制器再次发出反映其操作地耦连的机械手臂的期望接头位置的马达电流命令该系统管理处理器然后返回执行参照357的任务。The remote operator 171 selects the weakened operation option, then in 363, the system management processor provides the remote operator 171 with an option to recover from the failure. Recovery from the failure in this case is different from the recovery with reference 381-382 because no attempt is made to recover the failed arm. Recovery should only be used to restore normal operation of the non-failed arm. In 364, the system management processor waits for the user to select the option provided in 363. Once the option is selected by the remote operator 171, in 365, the system management processor sends a message to the arm processor of the disabled arm to reinforce the failure. The enhancement of the failure in this case means that additional steps are taken to completely shut down the operation of the failed robot arm. An example of such an enhanced measure is to operatively disconnect the joint position controller of the arm processor from other components of the master/slave control system that generates the desired joint position of the operatively coupled robot arm. One strengthening measure is to turn off the power to the failed robotic arm. In 366, the system management processor then releases the soft lock of the joint controller by unlocking the output of the joint controller of the arm processor of all non-failed arms, so that the joint controller again sends out the mechanical coupling reflecting its operation. The motor current at the desired joint position of the arm commands the system management processor and then returns to perform the task with reference to 357.
在367中，该系统管理处理器使系统FRL情况对在机器人170中的所有节点有效。通过引起FRL线路329、327、384和385被拉高从而使故障通知同时被提供给手臂处理器和第一机械手182、第二机械手183、第三机械手184和第四机械手185的节点来完成该步骤。在369中，该系统管理处理器然后确定该系统故障是否为可恢复系统故障。通过检查在所接收错误消息中的错误类来完成该步骤。在369中的确定为否，则在363中，该系统管理处理器没有采取进一步行动并且等待该系统被关闭。In 367, the system management processor validates the system FRL situation for all nodes in the robot 170. This is done by causing the FRL lines 329, 327, 384, and 385 to be pulled high so that the fault notification is provided to the arm processor and the nodes of the first robot 182, second robot 183, third robot 184, and fourth robot 185 at the same time. step. In 369, the system management processor then determines whether the system failure is a recoverable system failure. Complete this step by checking the error class in the received error message. The determination in 369 is no, then in 363, the system management processor takes no further action and waits for the system to be shut down.
在369中的确定为是，则在370中，该系统管理处理器提供给用户从故障中恢复的选项。在371中，该管理处理器等待远程操作员171选取恢复选项。该选项被选取则在372中，该系统管理处理器发送恢复通知到机器人170的所有机械手臂的手臂处理器。在373中，该系统管理处理器在接收来自171的请求或行动时释放每个接头控制器的软锁以在其正常操作状态下操作接头控制器的手臂，以便所释放的接头控制器再一次发出反映其操作地耦连的机械手臂的期望接头位置的马达电流命令该系统管理处理器随后返回执行参照357的任务。The determination in 369 is yes, then in 370, the system management processor provides the user with an option to recover from the failure. In 371, the management processor waits for the remote operator 171 to select a recovery option. If this option is selected, in 372, the system management processor sends a recovery notification to the arm processors of all the robot arms of the robot 170. In 373, the system management processor, upon receiving a request or action from 171, releases the soft lock of each joint controller to operate the joint controller arm in its normal operating state, so that the released joint controller is once again Sending a motor current reflecting the desired joint position of the robotic arm to which it is operatively coupled instructs the system management processor to then return to perform the task of reference 357.
在图47和图52是执行故障反应、故障隔离和故障弱化的方法的各方面的流程图，该故障反应、故障隔离和故障弱化通过操作地耦连到机器人170的系统管理处理器和手臂处理器328、325、323和321的手臂管理处理器被执行。手臂管理处理器319开始、控制和监控机器人170的第一机械手臂第一机械手182、第二机械手183、第三机械手184、第四机械手185的某些协调活动。该手臂管理器319开始和监控启动制动器测试，其中该手臂管理器319与手臂处理器328、325和323中的每个通信，以便具有不同扭矩值的具体制动顺序被应用到它们的各自机械手臂的制动器。该活动的协调在这种情况下通过手臂管理器319进行，因为将其编码到每个手臂处理器的开销是多余的。在每个顺序的结束时，由每个手臂处理器所计算的最大扭矩值被传递回手臂管理器319，超出范围的错误发生时，则手臂管理器319将传递故障通知到失效手臂。手臂管理器命令该手臂处理器执行手臂活动，监控结果，并且决定该活动结果是否表示手臂失效。在374中，手臂管理器根据机械手臂各自手臂处理器的报告监控机械手臂的协调活动以检测在一个手臂中的故障。当所报告的测量超过预期值达阈值量时，该手臂管理器确定故障已经发生。在这种情况下所检测的故障是机械手臂的节点中的一个节点或手臂处理器一般不会检测的故障。在故障已经在374中被检测到之后，然后在375中，手臂管理器抑制到失效手臂的任何进一步命令。没有进一步命令将从手臂管理器被传输到失效手臂的手臂处理器，直到或者从系统管理器接收恢复通知或者重新启动该系统。在376中，该手臂管理器通过将失效手臂的FRL线路拉到高状态来传送故障通知到失效手臂。在虚拟FRL线路的情况下，手臂管理器通过失效手臂的节点中的一个节点或手臂处理器传输在与被指定用于传输故障通知的包字段相同或不同的包字段中的故障通知。在377中，该手臂管理器发送错误消息到系统管理器，该错误消息具有故障的可用细节，通过手臂管理器检测的每种故障类型被分配错误代码，并且该错误代码被分类为错误类，故障的起源包括失效手臂的身份信息以及故障的来源的可选附加信息。In FIGS. 47 and 52 are flowcharts of various aspects of methods for performing fault reaction, fault isolation, and fault weakening, which are operatively coupled to the system management processor and arm processing of the robot 170 The arm management processors of the devices 328, 325, 323, and 321 are executed. The arm management processor 319 starts, controls, and monitors certain coordinated activities of the first robot 170, the first robot 182, the second robot 183, the third robot 184, and the fourth robot 185. The arm manager 319 initiates and monitors the start of the brake test, where the arm manager 319 communicates with each of the arm processors 328, 325, and 323 so that specific braking sequences with different torque values are applied to their respective machines The brake of the arm. The coordination of this activity is performed by the arm manager 319 in this case, because the overhead of coding it to each arm processor is redundant. At the end of each sequence, the maximum torque value calculated by each arm processor is transmitted back to the arm manager 319. When an out-of-range error occurs, the arm manager 319 will notify the failed arm of the transmission failure. The arm manager instructs the arm processor to perform arm activity, monitors the result, and determines whether the result of the activity indicates arm failure. In 374, the arm manager monitors the coordinated activities of the robotic arms based on the reports of the respective arm processors of the robotic arms to detect faults in one arm. When the reported measurement exceeds the expected value by a threshold amount, the arm manager determines that a fault has occurred. In this case, the detected fault is a fault that is generally not detected by one of the nodes of the robot arm or the arm processor. After the fault has been detected in 374, then in 375, the arm manager suppresses any further commands of the disabled arm. No further commands will be transmitted from the arm manager to the arm processor of the failed arm until either a recovery notification is received from the system manager or the system is restarted. In 376, the arm manager sends a fault notification to the failed arm by pulling the FRL line of the failed arm to a high state. In the case of a virtual FRL line, the arm manager transmits a failure notification in a packet field that is the same as or different from the packet field designated for transmission of the failure notification through one of the nodes of the failed arm or the arm processor. In 377, the arm manager sends an error message to the system manager, the error message has the available details of the failure, each type of failure detected by the arm manager is assigned an error code, and the error code is classified as an error class, The origin of the fault includes the identification information of the failed arm and optional additional information on the source of the fault.
在图51和图52中，在357中该系统管理器然后开始处理错误消息。在378中该手臂管理器确定所检测故障是否为可恢复故障。根据故障的错误类来进行该确定。在378中的确定为否，则在381中，手臂管理器持续抑制到失效手臂的任何进一步命令并且忽视随后从系统管理器接收的任何恢复通知。在378中的确定为是，则在379中，该手臂管理器等待从管理处理器所接收的恢复通知。在380中，恢复通知被接收，该手臂管理器停止抑制到失效手臂的命令并且返回它的正常操作模式并且执行参照374的故障检测任务。In Figure 51 and Figure 52, in 357 the system manager then starts processing error messages. In 378, the arm manager determines whether the detected fault is a recoverable fault. This determination is made according to the error class of the fault. The determination in 378 is no, then in 381, the arm manager continues to suppress any further commands to the disabled arm and ignores any recovery notifications subsequently received from the system manager. The determination in 378 is yes, then in 379, the arm manager waits for the recovery notification received from the management processor. In 380, the recovery notification is received, the arm manager stops suppressing the command to the failed arm and returns to its normal operation mode and performs the fault detection task with reference 374.
图24是本发明第二和第三实施例双模驾驶模式的系统连接图，采用模块化设计思路无人驾驶控制器525，无人驾驶控制器525与车辆视觉系统502连接、无人驾驶控制器525与定位导航模块274、规划系统模块528、无人驾驶控制器525与车辆总线276连接，控制系统模块529与机器人170连接，各模块之间采用以太网和CAN总线通讯。无人驾驶控制器525采用可重构计算AI芯片，基于Linux系统平台进行程序编写，综合车辆整车参数和运营特点参数定制控制策略，具体控制策略包括：1、无人驾驶控制器525作为控制模式的仲裁控制器，决定当前是否处于无人驾驶模式，是否满足无人驾驶模式的条件；2、当处于远程驾驶模式下，控制系统模块屏蔽无人驾驶控制器发出的控制命令，响应远程操作命令；3、当处于无人驾驶控制模式下，根据无人驾驶控制器525收到的环境感知模块接受到的激光雷达、毫米波雷达和摄像头等的信息，经各传感器融合的优化信息，以及接收到的陀螺仪感知的车身姿态信息和DGPS定位导航模块提供的经纬度信息，结合经深度学习获知的循迹路线，通过并行建图与定位技术(SLAM)自适应路线规划行驶。Fig. 24 is a system connection diagram of the dual-mode driving mode of the second and third embodiments of the present invention. The modular design concept is adopted for the unmanned driving controller 525. The unmanned driving controller 525 is connected to the vehicle vision system 502 for unmanned driving control The device 525 is connected with the positioning and navigation module 274, the planning system module 528, and the unmanned driving controller 525 with the vehicle bus 276, and the control system module 529 is connected with the robot 170. Ethernet and CAN bus communication are adopted between the modules. The unmanned driving controller 525 adopts a reconfigurable computing AI chip and is programmed based on the Linux system platform. The control strategy is customized by integrating vehicle parameters and operating characteristics. The specific control strategies include: 1. The unmanned driving controller 525 is used as control Mode arbitration controller determines whether it is currently in unmanned driving mode and whether it meets the conditions of unmanned driving mode; 2. When in remote driving mode, the control system module shields the control commands issued by the unmanned driving controller and responds to remote operations Command; 3. When in the unmanned driving control mode, according to the information of lidar, millimeter-wave radar and camera received by the environment sensing module received by the unmanned controller 525, the optimized information fused by each sensor, and The received body attitude information sensed by the gyroscope and the latitude and longitude information provided by the DGPS positioning and navigation module are combined with the tracking route learned through deep learning, and adaptive route planning through parallel mapping and positioning technology (SLAM).
图25是本发明第二实施例双模驾驶切换逻辑图，采用远程控制驾驶模式能够切换成智能无人驾驶模式。应急切换模式：“远程控制驾驶模式”信号中断自动发送请求517，此时无人驾驶控制器判断进入无人驾驶模式自动接管远程驾驶模式，执行518，不允许进入无人驾驶模式，反馈信号519，当需从“无人驾驶模式”进入“远程控制驾驶模式”时，通过复位外置无人驾驶控制开关或预先规划好的退出无人驾驶逻辑切换到正常驾驶模式，执行控制指令520。Fig. 25 is a logic diagram of dual-mode driving switching in the second embodiment of the present invention. The remote control driving mode can be switched to the intelligent unmanned driving mode. Emergency switching mode: the "remote control driving mode" signal interrupts the automatic sending request 517. At this time, the unmanned driving controller judges to enter the unmanned driving mode and automatically takes over the remote driving mode, and executes 518. It is not allowed to enter the unmanned driving mode, and the feedback signal 519 When it is necessary to enter the “remote control driving mode” from the “unmanned driving mode”, the external unmanned driving control switch is reset or the pre-planned exit unmanned driving logic is switched to the normal driving mode, and the control instruction 520 is executed.
图26是本发明第三实施例双模驾驶切换逻辑图，采用远程控制驾驶模式能够切换成智能无人驾驶模式，正常切换模式：当需从“远程控制驾驶模式”进入“智能无人驾驶模式”时，按下外置无人驾驶控制开关向无人驾驶控制器发送请求521，此时无人驾驶控制器判断是否进入无人驾驶模式，允许进入无人驾驶模式，执行522，不允许进入无人驾驶模式，反馈信号523；当需从“无人驾驶模式”进入“远程控制驾驶模式”时，通过复位外置无人驾驶控制开关或预先规划好的退出无人驾驶逻辑切换到正常驾驶模式，执行控制指令524。Figure 26 is a logic diagram of dual-mode driving switching logic diagram of the third embodiment of the present invention. The remote control driving mode can be used to switch to the intelligent unmanned driving mode. Normal switching mode: when it is necessary to enter the "intelligent unmanned driving mode" from the "remote control driving mode" ”, press the external driverless control switch to send a request 521 to the driverless controller. At this time, the driverless controller judges whether to enter the driverless mode, and allows the driver to enter the driverless mode, execute 522, and not allow Unmanned driving mode, feedback signal 523; when it is necessary to enter the “remote control driving mode” from “unmanned driving mode”, switch to normal driving by resetting the external unmanned driving control switch or pre-planned exit unmanned driving logic Mode, execute the control instruction 524.

Claims (9)

  1. 一种主次无线设备通过物联网连接成的车辆远程驾驶体系，其特征是：远程操作员171通过以下连接，控制机器人170驾驶车辆260行驶，远程控制台169与远程控制中心298连接，远程控制中心298与有线和无线局域网295连接，有线和无线局域网295与交换机291连接，交换机291与地面网络264连接，地面网络264与无线载波系统262连接，无线载波系统262与通信网络接入系统278连接，通信网络接入系统278与远程信息处理单元269连接，远程信息处理单元269与车辆总线276连接，车辆总线276与机器人170连接，机器人170与第一机械手182连接、机器人170与第二机械手183连接、机器人170与第三机械手184连接、机器人170与第四机械手185连接，第一机械手182与方向盘235连接，第二机械手183与方向盘235连接，第二机械手183还能够与档位400连接，第三机械手184与油门踏板402连接，第四机械手185与刹车踏板401连接，备用驾驶系统：远程控制台169与远程控制中心298连接，远程控制中心298与交换机291连接，交换机291与地面网络264连接，地面网络264与上行链路发射站290连接，上行链路发射站290与通信卫星289连接，通信卫星289与通信网络接入系统278连接，通信网络接入系统278与远程信息处理单元269连接，远程信息处理单元269与机器人170连接，机器人170与方向盘235连接、机器人170与档位400连接、机器人170与刹车踏板401连接、机器人170与油门踏板402连接，A vehicle remote driving system that connects primary and secondary wireless devices through the Internet of Things, which is characterized by: the remote operator 171 controls the robot 170 to drive the vehicle 260 through the following connections, the remote console 169 is connected to the remote control center 298, and the remote control Center 298 is connected to wired and wireless LAN 295, wired and wireless LAN 295 is connected to switch 291, switch 291 is connected to ground network 264, ground network 264 is connected to wireless carrier system 262, and wireless carrier system 262 is connected to communication network access system 278 , The communication network access system 278 is connected to the telematics unit 269, the telematics unit 269 is connected to the vehicle bus 276, the vehicle bus 276 is connected to the robot 170, the robot 170 is connected to the first manipulator 182, and the robot 170 and the second manipulator 183 Connection: The robot 170 is connected to the third manipulator 184, the robot 170 is connected to the fourth manipulator 185, the first manipulator 182 is connected to the steering wheel 235, the second manipulator 183 is connected to the steering wheel 235, and the second manipulator 183 can also be connected to the gear 400, The third manipulator 184 is connected to the accelerator pedal 402, the fourth manipulator 185 is connected to the brake pedal 401, and the backup driving system: the remote console 169 is connected to the remote control center 298, the remote control center 298 is connected to the switch 291, and the switch 291 is connected to the ground network 264 Connected, the ground network 264 is connected to the uplink transmitting station 290, the uplink transmitting station 290 is connected to the communication satellite 289, the communication satellite 289 is connected to the communication network access system 278, and the communication network access system 278 is connected to the telematics unit 269 The telematics unit 269 is connected to the robot 170, the robot 170 is connected to the steering wheel 235, the robot 170 is connected to the gear 400, the robot 170 is connected to the brake pedal 401, and the robot 170 is connected to the accelerator pedal 402.
    主雷达视频图像传输线路：车辆视觉系统502与处理器280连接，处理器280与通信网络接入系统278连接，通信网络接入系统278与无线载波系统262连接，无线载波系统262与地面网络264连接，地面网络264与交换机291连接，交换机291与远程控制中心298连接，远程控制中心298与第二处理器215连接，第二处理器215与视觉显示器255连接，备用雷达视频图像传输线路：车辆视觉系统502与处理器280连接，处理器280与通信网络接入系统278连接，通信网络接入系统278与通信卫星289连接，通信卫星289与上行链路发射站290连接，上行链路发射站290与地面网络264连接，地面网络264与交换机291连接，交换机291与远程控制中心298连接，远程控制中心298与第二处理器215连接，第二处理器215与视觉显示器255连接，车辆视觉系统502的雷达110和视频采集设备120由雷达视频信息融合系统130融合，443扫描雷达视频信息融合系统130图像传输给压缩存储单元444、压缩存储单元444把图像传输给第一判断单元445，第一判断单元445把图像传输给压缩数据生成单元446，压缩数据生成单元446把压缩好的图像传输给发送模块447，由发送模块447发送给通信网络接入系统278，通信网络接入系统278通过无线载波系统262和地面通信网络264传输给交换机交换机291，交换机291传输给远程控制中心298，远程控制中心298传输给第二处理器215，第二处理器215连接接收模块263，第二处理器215把接收到的图像传输给接收模块263，接收模块263把接收到的图像传输给压缩数据扫描单元449，压缩数据扫描单元449传输给压缩逻辑获取单元450，压缩逻辑获取单元450传输给解压缩读取单元451，解压缩读取单元451传输给第二判断单元452，第二判断单元452传输给原始字节数据恢复单元453，原始字节数据恢复单元453把图像传输给视觉显示器255。Main radar video image transmission line: the vehicle vision system 502 is connected to the processor 280, the processor 280 is connected to the communication network access system 278, the communication network access system 278 is connected to the wireless carrier system 262, and the wireless carrier system 262 is connected to the ground network 264 Connected, the ground network 264 is connected to the switch 291, the switch 291 is connected to the remote control center 298, the remote control center 298 is connected to the second processor 215, and the second processor 215 is connected to the visual display 255. Backup radar video image transmission line: vehicle The vision system 502 is connected to the processor 280, the processor 280 is connected to the communication network access system 278, the communication network access system 278 is connected to the communication satellite 289, the communication satellite 289 is connected to the uplink transmitting station 290, and the uplink transmitting station 290 is connected to the ground network 264, the ground network 264 is connected to the switch 291, the switch 291 is connected to the remote control center 298, the remote control center 298 is connected to the second processor 215, the second processor 215 is connected to the visual display 255, and the vehicle vision system The radar 110 of 502 and the video acquisition device 120 are fused by the radar video information fusion system 130. The 443 scanning radar video information fusion system 130 transmits the image to the compression storage unit 444, and the compression storage unit 444 transmits the image to the first judgment unit 445. The judging unit 445 transmits the image to the compressed data generating unit 446, and the compressed data generating unit 446 transmits the compressed image to the sending module 447. The sending module 447 sends the compressed image to the communication network access system 278. The communication network access system 278 uses wireless The carrier system 262 and the ground communication network 264 are transmitted to the switch switch 291, the switch 291 transmits to the remote control center 298, and the remote control center 298 transmits to the second processor 215. The second processor 215 is connected to the receiving module 263, and the second processor 215 The received image is transmitted to the receiving module 263, the receiving module 263 transmits the received image to the compressed data scanning unit 449, the compressed data scanning unit 449 transmits to the compression logic acquisition unit 450, and the compression logic acquisition unit 450 transmits to the decompression reading unit. The fetching unit 451 and the decompression reading unit 451 transmit to the second judgment unit 452, the second judgment unit 452 transmits to the original byte data recovery unit 453, and the original byte data recovery unit 453 transmits the image to the visual display 255.
  2. 一种主次无线设备通过物联网连接成的车辆远程驾驶体系，其特征是：远程操作员171通过以下连接，控制机器人170驾驶车辆260行驶，远程控制台169与远程控制中心298连接，远程控制中心298与有线和无线局域网295连接，有线和无线局域网295与交换机291连接，交换机291与地面网络264连接，地面网络264与无线载波系统262连接，无线载波系统262与通信网络接入系统278连接，通信网络接入系统278与远程信息处理单元269连接，远程信息处理单元269与车辆总线276连接，车辆总线276与机器人170连接，机器人170与第一机械手182连接、机器人170与第二机械手183连接、机器人170与第三机械手184连接、机器人170与第四机械手185连接，第一机械手182与方向盘235连接，第二机械手183与方向盘235连接，第二机械手183还能够与档位400连接，第三机械手184与油门踏板402连接，第四机械手185与刹车踏板401连接，备用驾驶系统：远程控制台169与远程控制中心298连接，远程控制中心298与交换机291连接，交换机291与地面网络264连接，地面网络264与上行链路发射站290连接，上行链路发射站290与通信卫星289连接，通信卫星289与通信网络接入系统278连接，通信网络接入系统278与远程信息处理单元269连接，远程信息处理单元269与机器人170连接，机器人170与方向盘235连接、机器人170与档位400连接、机器人170与刹车踏板401连接、机器人170与油门踏板402连接。A vehicle remote driving system that connects primary and secondary wireless devices through the Internet of Things, which is characterized by: the remote operator 171 controls the robot 170 to drive the vehicle 260 through the following connections, the remote console 169 is connected to the remote control center 298, and the remote control Center 298 is connected to wired and wireless LAN 295, wired and wireless LAN 295 is connected to switch 291, switch 291 is connected to ground network 264, ground network 264 is connected to wireless carrier system 262, and wireless carrier system 262 is connected to communication network access system 278 , The communication network access system 278 is connected to the telematics unit 269, the telematics unit 269 is connected to the vehicle bus 276, the vehicle bus 276 is connected to the robot 170, the robot 170 is connected to the first manipulator 182, and the robot 170 and the second manipulator 183 Connection: The robot 170 is connected to the third manipulator 184, the robot 170 is connected to the fourth manipulator 185, the first manipulator 182 is connected to the steering wheel 235, the second manipulator 183 is connected to the steering wheel 235, and the second manipulator 183 can also be connected to the gear 400, The third manipulator 184 is connected to the accelerator pedal 402, the fourth manipulator 185 is connected to the brake pedal 401, and the backup driving system: the remote console 169 is connected to the remote control center 298, the remote control center 298 is connected to the switch 291, and the switch 291 is connected to the ground network 264 Connected, the ground network 264 is connected to the uplink transmitting station 290, the uplink transmitting station 290 is connected to the communication satellite 289, the communication satellite 289 is connected to the communication network access system 278, and the communication network access system 278 is connected to the telematics unit 269 In connection, the telematics unit 269 is connected to the robot 170, the robot 170 is connected to the steering wheel 235, the robot 170 is connected to the gear 400, the robot 170 is connected to the brake pedal 401, and the robot 170 is connected to the accelerator pedal 402.
  3. 一种主次无线设备通过物联网连接成的车辆远程驾驶体系，其特征是：主雷达视频图像传输线路：车辆视觉系统502与处理器280连接，处理器280与通信网络接入系统278连接，通信网络接入系统278与无线载波系统262连接，无线载波系统262与地面网络264连接，地面网络264与交换机291连接，交换机291与远程控制中心298连接，远程控制中心298与第二处理器215连接，第二处理器215与视觉显示器255连接，备用雷达视频图像传输线路：车辆视觉系统502与处理器280连接，处理器280与通信网络接入系统278连接，通信网络接入系统278与通信卫星289连接，通信卫星289与上行链路发射站290连接，上行链路发射站290与地面网络264连接，地面网络264与交换机291连接，交换机291与远程控制中心298连接，远程控制中心298与第二处理器215连接，第二处理器215与视觉显示器255连接。A vehicle remote driving system connected by primary and secondary wireless devices through the Internet of Things is characterized by: primary radar video image transmission line: vehicle vision system 502 is connected to processor 280, and processor 280 is connected to communication network access system 278, The communication network access system 278 is connected to the wireless carrier system 262, the wireless carrier system 262 is connected to the ground network 264, the ground network 264 is connected to the switch 291, the switch 291 is connected to the remote control center 298, and the remote control center 298 is connected to the second processor 215 Connected, the second processor 215 is connected to the visual display 255, and a spare radar video image transmission line: the vehicle vision system 502 is connected to the processor 280, the processor 280 is connected to the communication network access system 278, and the communication network access system 278 is connected to communication The satellite 289 is connected, the communication satellite 289 is connected to the uplink transmitting station 290, the uplink transmitting station 290 is connected to the ground network 264, the ground network 264 is connected to the switch 291, the switch 291 is connected to the remote control center 298, and the remote control center 298 is connected to The second processor 215 is connected, and the second processor 215 is connected with the visual display 255.
  4. 根据权利要求1和2所述的一种主次无线设备通过物联网连接成的车辆远程驾驶体系，其特征是：主雷达视频图像传输线路：车辆视觉系统502与处理器280连接，处理器280与通信网络接入系统278连接，通信网络接入系统278与无线载波系统262连接，无线载波系统262与地面网络264连接，地面网络264与交换机291连接，交换机291与远程控制中心298连接，远程控制中心298与第二处理器215连接，第二处理器215与视觉显示器255连接。According to claim 1 and 2, a vehicle remote driving system formed by connecting primary and secondary wireless devices through the Internet of Things, characterized in that: the main radar video image transmission line: the vehicle vision system 502 is connected to the processor 280, and the processor 280 It is connected to the communication network access system 278, the communication network access system 278 is connected to the wireless carrier system 262, the wireless carrier system 262 is connected to the ground network 264, the ground network 264 is connected to the switch 291, and the switch 291 is connected to the remote control center 298. The control center 298 is connected to the second processor 215, and the second processor 215 is connected to the visual display 255.
  5. 根据权利要求1和2所述的一种主次无线设备通过物联网连接成的车辆远程驾驶体系，其特征是：远程操作员 171通过以下连接，控制机器人170驾驶车辆260行驶，远程控制台169与远程控制中心298连接，远程控制中心298与有线和无线局域网295连接，有线和无线局域网295与交换机291连接，交换机291与地面网络264连接，地面网络264与无线载波系统262连接，无线载波系统262与通信网络接入系统278连接，通信网络接入系统278与远程信息处理单元269连接，远程信息处理单元269与车辆总线276连接，车辆总线276与机器人170连接，机器人170与第一机械手182连接、机器人170与第二机械手183连接、机器人170与第三机械手184连接、机器人170与第四机械手185连接，第一机械手182与方向盘235连接，第二机械手183与方向盘235连接，第二机械手183还能够与档位400连接，第三机械手184与油门踏板402连接，第四机械手185与刹车踏板401连接。According to claim 1 and 2, a vehicle remote driving system formed by connecting primary and secondary wireless devices through the Internet of Things, characterized in that: the remote operator 171 controls the robot 170 to drive the vehicle 260 through the following connection, and the remote console 169 Connected with remote control center 298, remote control center 298 connected with wired and wireless LAN 295, wired and wireless LAN 295 connected with switch 291, switch 291 connected with ground network 264, ground network 264 connected with wireless carrier system 262, wireless carrier system 262 is connected to the communication network access system 278, the communication network access system 278 is connected to the telematics unit 269, the telematics unit 269 is connected to the vehicle bus 276, the vehicle bus 276 is connected to the robot 170, and the robot 170 is connected to the first manipulator 182 Connection, the robot 170 is connected to the second manipulator 183, the robot 170 is connected to the third manipulator 184, the robot 170 is connected to the fourth manipulator 185, the first manipulator 182 is connected to the steering wheel 235, the second manipulator 183 is connected to the steering wheel 235, and the second manipulator 183 can also be connected to the gear 400, the third manipulator 184 is connected to the accelerator pedal 402, and the fourth manipulator 185 is connected to the brake pedal 401.
  6. 根据权利要求1和2所述的一种主次无线设备通过物联网连接成的车辆远程驾驶体系，其特征是：备用驾驶系统：远程控制台169与远程控制中心298连接，远程控制中心298与交换机291连接，交换机291与地面网络264连接，地面网络264与上行链路发射站290连接，上行链路发射站290与通信卫星289连接，通信卫星289与通信网络接入系统278连接，通信网络接入系统278与远程信息处理单元269连接，远程信息处理单元269与机器人170连接，机器人170与方向盘235连接、机器人170与档位400连接、机器人170与刹车踏板401连接、机器人170与油门踏板402连接。According to claims 1 and 2, a vehicle remote driving system formed by connecting primary and secondary wireless devices through the Internet of Things is characterized in that: standby driving system: remote console 169 is connected to remote control center 298, and remote control center 298 is connected to The switch 291 is connected, the switch 291 is connected to the ground network 264, the ground network 264 is connected to the uplink transmitting station 290, the uplink transmitting station 290 is connected to the communication satellite 289, and the communication satellite 289 is connected to the communication network access system 278. The communication network The access system 278 is connected to the telematics unit 269, the telematics unit 269 is connected to the robot 170, the robot 170 is connected to the steering wheel 235, the robot 170 is connected to the gear 400, the robot 170 is connected to the brake pedal 401, and the robot 170 is connected to the accelerator pedal 402 connection.
  7. 根据权利要求1、2和3所述的一种主次无线设备通过物联网连接成的车辆远程驾驶体系，其特征是：备用雷达视频图像传输线路：车辆视觉系统502与处理器280连接，处理器280与通信网络接入系统278连接，通信网络接入系统278与通信卫星289连接，通信卫星289与上行链路发射站290连接，上行链路发射站290与地面网络264连接，地面网络264与交换机291连接，交换机291与远程控制中心298连接，远程控制中心298与第二处理器215连接，第二处理器215与视觉显示器255连接。According to claim 1, 2 and 3, a vehicle remote driving system formed by connecting primary and secondary wireless devices through the Internet of Things is characterized in that: a backup radar video image transmission line: the vehicle vision system 502 is connected to the processor 280 for processing The device 280 is connected to the communication network access system 278, the communication network access system 278 is connected to the communication satellite 289, the communication satellite 289 is connected to the uplink transmitting station 290, the uplink transmitting station 290 is connected to the ground network 264, and the ground network 264 It is connected to the switch 291, the switch 291 is connected to the remote control center 298, the remote control center 298 is connected to the second processor 215, and the second processor 215 is connected to the visual display 255.
  8. 根据权利要求1和2所述的一种主次无线设备通过物联网连接成的车辆远程驾驶体系，其特征是：车辆视觉系统502的雷达110和视频采集设备120由雷达视频信息融合系统130融合，443扫描雷达视频信息融合系统130图像传输给压缩存储单元444、压缩存储单元444把图像传输给第一判断单元445，第一判断单元445把图像传输给压缩数据生成单元446，压缩数据生成单元446把压缩好的图像传输给发送模块447，由发送模块447发送给通信网络接入系统278，通信网络接入系统278通过无线载波系统262和地面通信网络264传输给交换机交换机291，交换机291传输给远程控制中心298，远程控制中心298传输给第二处理器215，第二处理器215连接接收模块263，第二处理器215把接收到的图像传输给接收模块263，接收模块263把接收到的图像传输给压缩数据扫描单元449，压缩数据扫描单元449传输给压缩逻辑获取单元450，压缩逻辑获取单元450传输给解压缩读取单元451，解压缩读取单元451传输给第二判断单元452，第二判断单元452传输给原始字节数据恢复单元453，原始字节数据恢复单元453把图像传输给视觉显示器255。A vehicle remote driving system formed by connecting primary and secondary wireless devices through the Internet of Things according to claims 1 and 2, characterized in that: the radar 110 of the vehicle vision system 502 and the video acquisition device 120 are fused by the radar video information fusion system 130 , 443 scanning radar video information fusion system 130 image is transmitted to the compression storage unit 444, the compression storage unit 444 transmits the image to the first judgment unit 445, the first judgment unit 445 transmits the image to the compressed data generating unit 446, the compressed data generating unit 446 transmits the compressed image to the sending module 447, which is sent by the sending module 447 to the communication network access system 278. The communication network access system 278 transmits to the switch switch 291 through the wireless carrier system 262 and the ground communication network 264. The switch 291 transmits To the remote control center 298, the remote control center 298 transmits to the second processor 215, the second processor 215 is connected to the receiving module 263, the second processor 215 transmits the received image to the receiving module 263, and the receiving module 263 receives The compressed data scanning unit 449 is transmitted to the compressed data scanning unit 449, the compressed data scanning unit 449 is transmitted to the compression logic acquisition unit 450, the compression logic acquisition unit 450 is transmitted to the decompression reading unit 451, and the decompression reading unit 451 is transmitted to the second judgment unit 452 , The second judgment unit 452 transmits to the original byte data recovery unit 453, and the original byte data recovery unit 453 transmits the image to the visual display 255.
  9. 根据权利要求1、2和3所述的一种主次无线设备通过物联网连接成的车辆远程驾驶体系，其特征是：远程操作员171通过远程控制机器人170驾驶车辆260行驶：远程控制台169与远程控制中心298连接，远程控制中心298与有线和无线局域网295连接，有线和无线局域网295与交换机291连接，交换机291与地面网络264连接，地面网络264与无线载波系统262连接，无线载波系统262与通信网络接入系统278连接，通信网络接入系统278与远程信息处理单元269连接，远程信息处理单元269与车辆总线276连接，端组中的各个席位按照该席位相应的权限和工作要求、制定计划发布控制指令，该控制指令数据传输网络交换机，网络交换机将控制指令数据传输至主服务器和二级服务器，经过主服务器和二级服务器的逻辑处理，然后将处理后的数据传输至图形拼接控制器上，图形拼接控制器智能化地实现各种数据的拼接、组合等操作，最后在大屏液晶显示屏中显示出来，图形拼接控制器智能化地实现各种数据的拼接、组合等操作，最后在大屏液晶显示屏中显示出来，PDA控制器发布控制指令，该控制指令数据传输网络交换机，网络交换机将控制指令数据传输至主服务器和二级服务器，经过主服务器和二级服务器的逻辑处理，然后将处理后的数据传输至图形拼接控制器上，图形拼接控制器智能化地实现各种数据的拼接、组合等操作，最后在大屏液晶显示屏中显示出来，在大屏液晶显示屏能够及时地显示出各种终端以及摄像头等部件的数据信息，便于终端各个席位的人员观看，以便于人们得到车辆260的信息数据，然后进行合适地协调操作，远程控制中心298有数据终端的数量、语音终端的数量、图形工作站、PDA控制器，大屏显示控制主机的显示控制模块具有16种显示控制模式，通过图形拼接控制器实现大屏液晶显示屏的16种显示模式的选取和切换；大屏液晶显示屏为大屏幕显示器，远程控制中心298包括：大屏液晶显示屏、大屏显示控制主机、网络交换机、图形拼接控制器、图形工作站、图形工作站组控制主机、服务器组和终端组；网络交换机分别与图形工作站、图形拼接控制器、图形工作站组控制主机、服务器、终端一一对应电通信连接；大屏液晶显示屏用于显示图形拼接控制器拼接后的图形、视频、音频资料；图形拼接控制器用于从图形工作站中调取图形或视频或音频并完成组合、拼接工作；图形工作站组控制用于控制图形工作站中图形或视频或音频的存储、移动、显示、删除操作；网络交换机实现与图形工作站、图形拼接控制器、图形工作站组控制主机、服务器、终端之间对应的数据通讯；服务器由主服务器和二级服务器构成，终端由数据终端和语音终端构成，主服务器用于接收和控制数据终端的数据信息，二级服务器用于接收和控制语音终端的语音信息；大屏显示控制主机还电通信连接有无线接收器，无线接收器通过无线通讯方式通信连接有PDA控制器，数据终端发出的数据指令信息通过网络交换机传输给主服务器，通过主服务器进行逻辑运算处理，将数据信息和处理结果通过大屏液晶显示屏和数据终端的液晶显示屏显示出来；语音终端发出的语音指令信息通过网络交换机传输给二级服务器，通过二级服务器进行逻辑运算处理，将语音信息和处理结果通过大屏液晶显示屏和语音终端的液晶显示屏显示出来；PDA控制器发出的数据、语音指令信息通过无线通讯方式传输给无线接收器，无线接收 器将数据、语音信息通过大屏显示控制主机传输给图形拼接控制器，通过主服务器、二级服务器的逻辑运算处理，将数据、语音信息和处理结果通过大屏液晶显示屏和PDA控制器的液晶显示屏显示出来；图形工作站组控制主机通过网络交换机将数据信息传输给主服务器，通过主服务器进行逻辑运算处理，将数据信息和处理结果通过大屏液晶显示屏显示出来，远程控制台169的第二处理器215由硬件、软件和固件组成，由一个单元执行或者分给若干子单元，每个子单元能够进而通过硬件、软件和固件的任何组合来实现，第二处理器215能够交叉连接控制逻辑和控制器，第二处理器215也能够作为子单元分布在整个车辆远程驾驶体系258中，第二处理器215能够执行来自非暂时性机器可读介质的机器可读指令，其激活第二处理器215以执行与指令相对应的动作，第二处理器215执行远程操作员171输入的各种指令，第二处理器215执行远程操作员171用左输入装置177和右输入装置178输入的指令，以致动第一机械手182和第二机械手183各自的关节，远程控制台169的第二处理器215与视觉显示器255、左输入装置177和右输入装置178、第一脚踏板214和第二脚底板233连接，视觉显示器255由第一显示屏174、第二显示屏175、第三显示屏176和第四显示屏179组成，第九成像装置410采集的驾驶室内部图像，通过压缩被传输到远程控制台169解压缩后显示在视觉显示器255的显示屏上，远程操作员171用双眼查看视觉显示器255上的图像，机器人170在汽车的驾驶室内直接控制汽车的方向盘235、油门踏板402、刹车踏板401、档位400，车门403、空调404和音响405，汽车座椅173包括座椅靠背216、防潜梁217、防潜连杆机构219、第五连杆218、第六连杆230、第七连杆231和立柱256，机器人170被固定到汽车座椅173上，在立柱256上安装第一机械手182、第二机械手183、第三机械手184和第四机械手185，机器人170上的机械手多于1个，第一机械手182、第二机械手183、第三机械手184和第四机械手185能够上下移动，并能够操纵附连的器械300、301、302、303，远程操作员171用左手抓握左输入装置177，左输入装置177能够引起第一机械手182的移动并与方向盘235连接；用右手抓握右输入装置178，右输入装置178能够引起第二机械手183的移动并与方向盘235连接，右输入装置178还能够引起第二机械手183的移动并与档位400连接，远程操作员171用左脚连接第二脚底板233，远程操作员171用右脚连接第一脚踏板214，第一脚踏板214能够引起机器人第三机械手184的移动，第三机械手184能够引起油门踏板402移动，第二脚底板233能够引起第四机械手185的移动，第四机械手185的移动能够与刹车踏板401连接，远程驾驶系统258包括车辆260、无线载波系统262、地面通信网络264、计算机266和呼叫中心265，车辆260为摩托车、卡车、客车、运动型多功能车(SUV)、休闲车(RV)、船只、航空器和超高速管道列车，车辆电子设备263包括远程信息处理单元269、麦克风270、按钮和控制输入271、音频系统272、可视显示器273、GPS和BDS卫星导航模块274、多个车辆系统模块(VSM虚拟交换矩阵)275，车辆总线276与娱乐总线277连接，控制器区域网络(CAN)、媒体导向系统传输(MOST)、本地互连网络(LIN)、局域网(LAN)和其它连接，以太网或符合ISO、SAE和IEEE标准和规范，远程信息处理单元269安装在车辆260上，无线载波系统262通过无线联网能够使车辆260与呼叫中心265进行无线语音和数据通信通信，远程信息处理单元269使用无线电传输建立与无线载波系统262的通信信道，通信信道包括语音信道和数据信道，通信信道能够发送和接收语音和数据传输，通信网络接入系统278包括：处理器、微波通信单元、卫星通信单元和移动通信单元；其中处理器模块适于接收微波通信单元、卫星通信单元和移动通信单元的数据信息，远程信息处理单元269使用根据GSM或CDMA标准的蜂窝通信，包括用于语音通信的标准蜂窝芯片组279、用于数据传输的无线调制解调器、电子处理设备280、多个数字存储设备281和通信网络接入系统278包括：处理器、微波通信单元、卫星通信单元和移动通信单元；其中处理器模块适于接收微波通信单元、卫星通信单元和移动通信单元的数据信息，能够通过存储在远程信息处理单元269中且通过处理器280执行的软件实现调制解调器，或者调制解调器能够是位于远程信息处理单元269内部或外部的分立硬件组件，调制解调器能够使用任何数量的不同标准或协议诸如EVDO、CDMA、GPRS和EDGE来运行，能够使用远程信息处理单元269实施车辆与其它联网的设备之间的无线联网，远程信息处理单元269能够被配置为根据一个或多个无线协议诸如IEEE 422.11协议、WiMAX或蓝牙中的任一个进行无线通信，当用于TCP/IP的包交换数据通信时，远程信息处理单元能够配置有静态IP地址或者能够设置为自动接收来自网络上的另一设备诸如路由器或者来自网络地址服务器的所分配的IP地址，According to claims 1, 2 and 3, a vehicle remote driving system formed by connecting primary and secondary wireless devices through the Internet of Things, characterized in that: the remote operator 171 drives the vehicle 260 through the remote control robot 170: remote console 169 Connected with remote control center 298, remote control center 298 connected with wired and wireless LAN 295, wired and wireless LAN 295 connected with switch 291, switch 291 connected with ground network 264, ground network 264 connected with wireless carrier system 262, wireless carrier system 262 is connected to the communication network access system 278, the communication network access system 278 is connected to the telematics unit 269, and the telematics unit 269 is connected to the vehicle bus 276. Each seat in the end group is in accordance with the corresponding authority and work requirements of the seat , Make a plan to issue control instructions. The control instruction data is transmitted to the network switch. The network switch transmits the control instruction data to the primary server and the secondary server, and then transmits the processed data to the graphics after the logic processing of the primary server and the secondary server On the splicing controller, the graphic splicing controller intelligently realizes the splicing and combination of various data, and finally displays it on the large-screen LCD display. The graphic splicing controller intelligently realizes the splicing and combination of various data, etc. The operation is finally displayed on the large-screen LCD display. The PDA controller issues a control command. The control command data is transmitted to the network switch. The network switch transmits the control command data to the main server and the secondary server, and passes through the main server and the secondary server. Then the processed data is transmitted to the graphics splicing controller. The graphics splicing controller intelligently realizes the splicing and combination of various data, and finally displays it on the large LCD screen. The LCD screen can display the data information of various terminals and cameras in a timely manner, which is convenient for the personnel in each seat of the terminal to view, so that people can obtain the information data of the vehicle 260, and then perform appropriate coordination operations. The remote control center 298 has data The number of terminals, the number of voice terminals, graphics workstations, PDA controllers, the display control module of the large-screen display control host has 16 display control modes, and the selection of 16 display modes of the large-screen LCD screen is realized through the graphics splicing controller And switching; the large-screen LCD display is a large-screen display, and the remote control center 298 includes: large-screen LCD, large-screen display control host, network switch, graphics splicing controller, graphics workstation, graphics workstation group control host, server group And the terminal group; the network switch is respectively connected to the graphics workstation, graphics splicing controller, graphics workstation group control host, server, and terminal in one-to-one correspondence; the large-screen LCD screen is used to display the graphics and video spliced by the graphics splicing controller , Audio data; the graphics splicing controller is used to retrieve graphics or video or audio from the graphics workstation and complete the combination and splicing work; the graphics workstation group control is used to control the storage, movement, and movement of graphics or video or audio in the graphics workstation Display and delete operations; the network switch realizes the corresponding data communication with the graphics workstation, graphics splicing controller, graphics workstation group control host, server, and terminal; the server is composed of a primary server and a secondary server, and the terminal is a data terminal and a voice terminal The main server is used to receive and control the data information of the data terminal, and the secondary server is used to receive and control the voice information of the voice terminal; the large-screen display control host is also connected to a wireless receiver for electronic communication, and the wireless receiver is through wireless communication The communication is connected with the PDA controller, the data command information sent by the data terminal is transmitted to the main server through the network switch, and the logical operation is processed by the main server, and the data information and processing results are displayed on the large-screen LCD display and the LCD display of the data terminal The voice command information issued by the voice terminal is transmitted to the secondary server through the network switch, and the secondary server performs logical operation processing, and the voice information and processing results are displayed on the large-screen LCD display and the LCD display of the voice terminal; PDA The data and voice command information sent by the controller are transmitted to the wireless receiver through wireless communication, and the wireless receiver transmits the data and voice information to the graphics splicing controller through the large-screen display control host, through the logic operation of the main server and the secondary server Processing, the data, voice information and processing results are displayed through the large-screen LCD display and the LCD display of the PDA controller; the graphics workstation group control host transmits the data information to the main server through the network switch, and performs logical operation processing through the main server , The data information and processing results are displayed on the large-screen LCD display. The second processor 215 of the remote console 169 is composed of hardware, software and firmware. It is executed by one unit or distributed to several sub-units. Each sub-unit can further Realized by any combination of hardware, software, and firmware, the second processor 215 can cross-connect the control logic and the controller, and the second processor 215 can also be distributed as a subunit in the entire vehicle remote driving system 258. The second processor 215 can execute machine-readable instructions from a non-transitory machine-readable medium, which activates the second processor 215 to perform actions corresponding to the instructions, and the second processor 215 executes various instructions input by the remote operator 171. The second processor 215 executes the instructions input by the remote operator 171 using the left input device 177 and the right input device 178 to activate the respective joints of the first manipulator 182 and the second manipulator 183. The second processor 215 of the remote console 169 and the visual The display 255, the left input device 177 and the right input device 178, the first foot pedal 214 and the second foot sole plate 233 are connected, and the visual display 255 consists of a first display screen 174, a second display screen 175, a third display screen 176 and a second It consists of four display screens 179. The internal images of the cab collected by the ninth imaging device 410 are compressed and transmitted to the remote console 169 and then displayed on the display screen of the visual display 255. The remote operator 171 uses both eyes to view the visual The image on the display 255, the robot 170 directly controls the steering wheel 235, the accelerator pedal 402, the brake pedal 401, the gear 400, the door 403, the air conditioner 404 and the audio 405 in the cab of the car. The car seat 173 includes the seat back 216 , Anti-submersible beam 217, anti-submersible link mechanism 219, fifth link 218, sixth link 230, seventh link 231 and column 256. The robot 170 is fixed to the car seat 173 and installed on the column 256 The first manipulator 182, the second manipulator 183, the third manipulator 184 and the fourth manipulator 185, the robot 170 has more than one manipulator, the first manipulator 182, the second manipulator 183, the third manipulator 184 and the fourth manipulator 185 can Move up and down and be able to manipulate the attached instruments 300, 301, 302, 303. The remote operator 171 grasps the left input device 177 with his left hand. The left input device 177 can cause the first manipulator 182 to move and connect with the steering wheel 235; Grasp the right input device 178 with the right hand. The right input device 178 can cause the movement of the second manipulator 183 and connect with the steering wheel 235. The right input device 178 can also cause the movement of the second manipulator 183 and connect with the gear 400. The remote operator 171 Connect the second foot plate 233 with the left foot, and the remote operator 171 connects the first foot pedal 214 with the right foot. The first foot pedal 214 can cause the movement of the third manipulator 184 of the robot, and the third manipulator 184 can cause the accelerator pedal 402 Movement, the second soleplate 233 can cause the movement of the fourth manipulator 185, and the movement of the fourth manipulator 185 can be connected to the brake pedal 401. The remote driving system 258 includes a vehicle 260, a wireless carrier system 262, a ground communication network 264, a computer 266 and Call center 265, vehicles 260 are motorcycles, trucks, buses, sport utility vehicles (SUVs), recreational vehicles (RVs), boats, aircrafts, and ultra-high-speed pipeline trains. Vehicle electronic equipment 263 includes telematics units 269, microphones 270, button and control input 271, audio system 272, visual display 273, GPS and BDS satellite navigation module 274, multiple vehicle system modules (VSM virtual switch matrix) 275, vehicle bus 276 is connected to entertainment bus 277, controller area Network (CAN), Media Oriented System Transmission (MOST), Local Interconnection Network (LIN), Local Area Network (LAN) and other connections, Ethernet or conform to ISO, SAE and IEEE standards and specifications, telematics unit 269 is installed in the vehicle On 260, the wireless carrier system 262 enables the vehicle 260 to perform wireless voice and data communication with the call center 265 through wireless networking. The telematics unit 269 uses radio transmission to establish a communication channel with the wireless carrier system 262. The communication channels include voice channels and Data channel, communication channel can send and receive voice and The data transmission and communication network access system 278 includes: a processor, a microwave communication unit, a satellite communication unit and a mobile communication unit; the processor module is suitable for receiving data information and remote information from the microwave communication unit, satellite communication unit and mobile communication unit The processing unit 269 uses cellular communication according to the GSM or CDMA standard, and includes a standard cellular chipset 279 for voice communication, a wireless modem for data transmission, an electronic processing device 280, multiple digital storage devices 281, and a communication network access system 278 includes: a processor, a microwave communication unit, a satellite communication unit, and a mobile communication unit; the processor module is adapted to receive data information of the microwave communication unit, satellite communication unit, and mobile communication unit, and can be stored in the telematics unit 269 And the modem is implemented by software executed by the processor 280, or the modem can be a discrete hardware component located inside or outside the telematics unit 269. The modem can use any number of different standards or protocols such as EVDO, CDMA, GPRS and EDGE to operate, The telematics unit 269 can be used to implement wireless networking between the vehicle and other networked devices. The telematics unit 269 can be configured to perform wirelessly according to any one of one or more wireless protocols such as the IEEE 422.11 protocol, WiMAX or Bluetooth. Communication, when used for TCP/IP packet exchange data communication, the telematics unit can be configured with a static IP address or can be set to automatically receive the assigned IP from another device on the network such as a router or from a network address server address,
    智能电话284是与远程信息处理单元269通信的联网设备中的无线设备，智能电话284包括计算机处理能力、能够使用短程无线协议通信的收发机、以及可视智能电话显示器286，智能电话显示器286包括触屏图形用户接口，智能电话284构造成使用传统Wi-Fi协议通信，智能电话284包括使用无线载波系统262经由蜂窝通信进行通信的能力，包括处理能力、显示器286、在短程无线通信链接上通信的能力，智能电话282可使用WiFi直接协议建立短程无线链接，智能电话282包括不具有蜂窝通信能力的设备，处理器280能够处理电子指令的设备，包括微处理器、微控制器、主处理器、控制器、车辆通信处理器和专用集成电路(ASIC)，是用于远程信息处理单元269的专用处理器，能够与其它车辆系统共享，处理器280执行各种类型的数字存储指令，诸如存储器281中存储的软件或固件程序，使远程信息处理单元能够提供多种类型的服务，处理器280能够执行程序或处理数据，The smart phone 284 is a wireless device among the networked devices that communicate with the telematics unit 269. The smart phone 284 includes computer processing capabilities, a transceiver capable of communicating using a short-range wireless protocol, and a visual smart phone display 286. The smart phone display 286 includes Touch screen graphical user interface, smart phone 284 is configured to communicate using traditional Wi-Fi protocol, smart phone 284 includes the ability to communicate via cellular communication using wireless carrier system 262, including processing power, display 286, communication over short-range wireless communication links The smart phone 282 can use the WiFi direct protocol to establish a short-range wireless link. The smart phone 282 includes devices that do not have cellular communication capabilities, and the processor 280 can process electronic instructions, including microprocessors, microcontrollers, and main processors. , Controller, vehicle communication processor and application specific integrated circuit (ASIC), which are dedicated processors for the telematics unit 269, which can be shared with other vehicle systems. The processor 280 executes various types of digital storage instructions, such as memory The software or firmware program stored in 281 enables the telematics unit to provide multiple types of services, and the processor 280 can execute programs or process data,
    远程信息处理单元269提供来自车辆的无线通信服务，这些服务包括：卫星导航模块274提供的服务；结合碰撞传感器接口模块和车身控制模块提供的安全气囊展开通知服务；使用诊断模块的诊断报告；以及信息、音乐、网页、电影、电视节目、视频游戏，在将模块实施为位于远程信息处理单元269外部的VSM 275的情况下，它们能够使用车辆总线276以与远程信息处理单元269交换数据和命令，The telematics unit 269 provides wireless communication services from the vehicle. These services include: the service provided by the satellite navigation module 274; the airbag deployment notification service provided by the combination of the collision sensor interface module and the body control module; the diagnosis report using the diagnosis module; and Information, music, webpages, movies, TV shows, video games, when the modules are implemented as VSM 275 located outside the telematics unit 269, they can use the vehicle bus 276 to exchange data and commands with the telematics unit 269 ,
    卫星导航模块274根据从卫星285接收的无线电信号，卫星导航模块274能够确定车辆位置，向车辆驾驶员提供导 航和其它位置相关服务，导航信息能够在显示器273上呈现和语音提示，位置信息能够提供给呼叫中心265或其它远程计算机系统，诸如计算机266，通过远程信息处理单元269将新的或更新的地图数据从呼叫中心265下载到卫星导航模块274，卫星导航包括GPS卫星导航系统和北斗(BDS)卫星导航系统，使用通信卫星289和上行链路发射站290来实施双向通信，节目内容由发射站290接收、被打包用于上载、然后发送到卫星289，卫星289向用户广播节目，双向通信使用卫星289的卫星电话服务，以在车辆260与站290之间中继电话通信，车辆260包括车辆系统模块(VSM)275，其位于车辆内且从传感器接收输入且使用感测的输入执行诊断、监控、控制、报告，每个VSM 275通过车辆总线276连接到其它VSM以及连接到远程信息处理单元269，并且能够被编程以运行车辆系统和子系统诊断测试，一个VSM 275能够是发动机控制模块(ECM)，其控制发动机操作的各个方面，包括燃料点火和点火正时，另一VSM 275能够是动力系统控制模块，其调整车辆动力系的组件的操作，另一VSM 275能够是车身控制模块，管理车辆内的电子组件包括车辆的电动门锁和前灯，发动机控制模块配备有车载诊断(OBD)特征件，其提供诸如从包括车辆排放传感器的各种传感器接收到的各种实时数据，并且提供标准化的一系列诊断故障码(DTC)，The satellite navigation module 274 can determine the location of the vehicle based on the radio signal received from the satellite 285, and provide navigation and other location-related services to the vehicle driver. The navigation information can be presented and voiced on the display 273, and the location information can be provided. To the call center 265 or other remote computer systems, such as the computer 266, the new or updated map data is downloaded from the call center 265 to the satellite navigation module 274 through the telematics unit 269. The satellite navigation includes GPS satellite navigation system and Beidou (BDS) ) A satellite navigation system that uses a communication satellite 289 and an uplink transmitter station 290 to implement two-way communication. The program content is received by the transmitter station 290, packaged for upload, and then sent to the satellite 289. The satellite 289 broadcasts programs to users, and two-way communication Satellite phone service using satellite 289 to relay telephone communication between vehicle 260 and station 290. Vehicle 260 includes vehicle system module (VSM) 275, which is located in the vehicle and receives input from sensors and uses the sensed input to perform diagnosis , Monitoring, control, and reporting. Each VSM 275 is connected to other VSMs through the vehicle bus 276 and to the telematics unit 269, and can be programmed to run vehicle system and subsystem diagnostic tests. A VSM 275 can be an engine control module ( ECM), which controls all aspects of engine operation, including fuel ignition and ignition timing. Another VSM 275 can be a power system control module, which adjusts the operation of components of the vehicle’s powertrain, and the other VSM 275 can be a body control module. The electronic components in the management vehicle include the electric door locks and headlights of the vehicle, the engine control module is equipped with on-board diagnostics (OBD) features that provide various real-time data such as received from various sensors including vehicle emission sensors, and Provide a standardized series of diagnostic trouble codes (DTC),
    车辆电子设备263包括多个车辆用户接口，提供和接收信息的装置，包括麦克风270、按钮271、音频系统272和可视显示器273，麦克风270向远程信息处理单元269提供音频输入，使车辆占用者能够通过无线载波系统262提供语音命令和实施免提呼叫，能够利用人机接口(HMI)技术连接到车载自动语音处理单元，按钮271允许远程信息处理单元269的手动用户输入，以启动无线电话呼叫和提供其它数据、响应或控制输入，分立的按钮能够使用以便向呼叫中心265发起紧急呼叫和常规服务援助呼叫，音频系统272向车辆占用者提供音频输出，音频系统272连接到车辆总线276和娱乐总线277，能够提供AM、FM、卫星无线电、CD、DVD和其它多媒体功能，可视显示器273是图形显示器，无线载波系统262是蜂窝电话系统，包括蜂窝塔287，移动交换中心(MSC)288以及将无线载波系统262与地面网络264连接所需的任何其它联网组件，蜂窝塔287包括发送和接收天线以及基站，来自不同蜂窝塔的基站直接地连接到MSC 288或者通过基站控制器的中间设备连接到MSC288，蜂窝系统262能够实施通信技术，包括AMPS的模拟技术和CDMA和GSM/GPRS数字技术，The vehicle electronic equipment 263 includes multiple vehicle user interfaces, devices for providing and receiving information, including a microphone 270, buttons 271, an audio system 272, and a visual display 273. The microphone 270 provides audio input to the telematics unit 269 to enable the vehicle occupant It can provide voice commands and implement hands-free calls through the wireless carrier system 262, and can connect to the on-board automatic voice processing unit using human machine interface (HMI) technology. The button 271 allows manual user input of the telematics unit 269 to initiate a wireless phone call In addition to providing other data, response or control inputs, separate buttons can be used to initiate emergency calls and regular service assistance calls to the call center 265, the audio system 272 provides audio output to the vehicle occupants, and the audio system 272 is connected to the vehicle bus 276 and entertainment The bus 277 can provide AM, FM, satellite radio, CD, DVD and other multimedia functions. The visual display 273 is a graphic display, and the wireless carrier system 262 is a cellular telephone system, including a cellular tower 287, a mobile switching center (MSC) 288, and Any other networking components needed to connect the wireless carrier system 262 to the ground network 264. The cell tower 287 includes transmitting and receiving antennas and base stations. The base stations from different cell towers are directly connected to the MSC 288 or through the intermediate equipment of the base station controller. By MSC288, cellular system 262 can implement communication technologies, including AMPS analog technology and CDMA and GSM/GPRS digital technology.
    地面网络264是地面的电信网络，其连接有线电话并且将无线载波系统262连接到呼叫中心265，地面网络264包括公共交换电话网(PSTN)，用于提供硬线电话、包交换数据通信和因特网基础设施，能够通过使用标准有线网络、光纤和其它光学网络、电缆网络、电源线、诸如无线局域网(WLAN)的其它无线网络或者提供宽带无线接入(BWA)的网络或者其任意组合来实施一段或多段地面网络264，呼叫中心265与无线载波系统262直接连接，The terrestrial network 264 is a terrestrial telecommunications network that connects wired telephones and connects the wireless carrier system 262 to the call center 265. The terrestrial network 264 includes the Public Switched Telephone Network (PSTN) to provide hard-line telephones, packet-switched data communications and the Internet Infrastructure, which can be implemented by using standard wired networks, fiber and other optical networks, cable networks, power lines, other wireless networks such as wireless local area networks (WLAN), or networks that provide broadband wireless access (BWA), or any combination thereof Or multi-segment ground network 264, call center 265 and wireless carrier system 262 are directly connected,
    计算机266通过远程信息处理单元269和无线载波262是由车辆访问的web服务器，计算机266通过远程信息处理单元269从车辆20上载诊断信息；计算机266提供因特网连接，提供DNS服务和作为网络地址服务器，其使用DHCP或其它适当协议向车辆260分配IP地址，呼叫中心265向车辆电子设备263提供系统后端功能，这些后端功能包括交换机291、服务器283、数据库292、远程控制中心298和自动语音响应系统(VRS)294通过有线和无线局域网295连接在一起，交换机291是专用交换(PBX)交换机，路由进入信号，使得语音传输通常通过常规电话发送到远程控制中心298，使用VoIP发送到自动语音响应系统294，远程控制中心298的电话也能够使用VoIP，通过交换机291的VoIP和其它数据通信通过在交换机291与网络295之间连接的调制解调器来实施，数据传输经由调制解调器传到服务器283和数据库292，数据库292能够存储账户信息、用户认证信息、车辆标识，还能够通过无线系统422.11x、GPRS进行数据传输，它通过远程控制中心298连接人工呼叫中心265而使用，呼叫中心265使用VRS 294作为自动指导者，呼叫中心265使用VRS 294与远程控制中心298连接，The computer 266 is a web server accessed by the vehicle through the telematics unit 269 and the wireless carrier 262, and the computer 266 uploads diagnostic information from the vehicle 20 through the telematics unit 269; the computer 266 provides Internet connection, provides DNS services and acts as a network address server, It uses DHCP or other appropriate protocols to assign an IP address to the vehicle 260. The call center 265 provides system back-end functions to the vehicle electronic equipment 263. These back-end functions include switch 291, server 283, database 292, remote control center 298, and automatic voice response. The system (VRS) 294 is connected by wired and wireless LAN 295. The switch 291 is a private exchange (PBX) switch that routes incoming signals so that the voice transmission is usually sent to the remote control center 298 via regular telephone, and sent to the automatic voice response using VoIP The system 294, the telephone of the remote control center 298 can also use VoIP. VoIP and other data communication through the switch 291 are implemented through the modem connected between the switch 291 and the network 295, and the data transmission is transmitted to the server 283 and the database 292 via the modem. The database 292 can store account information, user authentication information, and vehicle identification. It can also transmit data through the wireless system 422.11x and GPRS. It connects to the manual call center 265 through the remote control center 298, and the call center 265 uses VRS 294 as automatic guidance Or, the call center 265 uses VRS 294 to connect to the remote control center 298,
    车辆总线276与机器人170连接，机器人170与第一机械手182连接、机器人170与第二机械手183连接、机器人170与第三机械手184连接、机器人170与第四机械手185连接，第一机械手182与方向盘235连接，第二机械手183与方向盘235连接，第二机械手183还能够与档位400连接，184与油门踏板402连接，第四机械手185与刹车踏板401连接，The vehicle bus 276 is connected to the robot 170, the robot 170 is connected to the first robot 182, the robot 170 is connected to the second robot 183, the robot 170 is connected to the third robot 184, the robot 170 is connected to the fourth robot 185, and the first robot 182 is connected to the steering wheel. 235 is connected, the second manipulator 183 is connected to the steering wheel 235, the second manipulator 183 can also be connected to the gear 400, 184 is connected to the accelerator pedal 402, and the fourth manipulator 185 is connected to the brake pedal 401,
    备用驾驶系统：远程控制台169与远程控制中心298，远程控制中心298与交换机291，交换机291与地面网络264，地面网络264与上行链路发射站290连接，上行链路发射站290与通信卫星289连接，通信卫星289与通信网络接入系统278，通信网络接入系统278与远程信息处理单元269，远程信息处理单元269与机器人170连接，机器人170与方向盘235连接、机器人170与档位400连接、机器人170与刹车踏板401连接和机器人170与油门踏板402连接后远程操作员171驾驶车辆260行驶；机器人170的电源线与车辆260的电源线连接在一起，Backup driving system: remote control station 169 and remote control center 298, remote control center 298 and switch 291, switch 291 and ground network 264, ground network 264 and uplink transmission station 290 connected, uplink transmission station 290 and communication satellite 289 connection, communication satellite 289 and communication network access system 278, communication network access system 278 and telematics unit 269, telematics unit 269 and robot 170, robot 170 and steering wheel 235, robot 170 and gear 400 After the robot 170 is connected to the brake pedal 401 and the robot 170 is connected to the accelerator pedal 402, the remote operator 171 drives the vehicle 260; the power cord of the robot 170 is connected to the power cord of the vehicle 260,
    主雷达视频图像传输线路：车辆视觉系统502与处理器280连接，The main radar video image transmission line: the vehicle vision system 502 is connected to the processor 280,
    车辆260包括车辆视觉系统502，车辆视觉系统502被配置成捕获车辆周围360°区域内的图像，车辆视觉系统502的第一成像装置500安装在前挡风玻璃后面、车辆格栅、前仪表板和更接近车辆前边缘的位置，用于捕获车辆260向前视场(FOV)506的图像的前视摄像机，车辆视觉系统502的第二成像装置508安装在车辆的后部用于捕获车辆的向后视场(FOV)510的后视摄像机，车辆视觉系统502的第三成像装置512安装在车辆的左侧用于捕获车辆的侧面视场(FOV)514的侧视图像摄像机，车辆视觉系统502的第四成像装置504安装在车辆的右侧用于捕获车辆的侧面视场(FOV)519的侧视摄像机，The vehicle 260 includes a vehicle vision system 502 that is configured to capture images in a 360° area around the vehicle. The first imaging device 500 of the vehicle vision system 502 is installed behind the front windshield, the vehicle grille, and the front dashboard And a position closer to the front edge of the vehicle, the front-view camera used to capture the image of the forward field of view (FOV) 506 of the vehicle 260, the second imaging device 508 of the vehicle vision system 502 is installed at the rear of the vehicle to capture the The rear view camera of the field of view (FOV) 510, the third imaging device 512 of the vehicle vision system 502 is installed on the left side of the vehicle to capture the side view of the vehicle’s field of view (FOV) 514. The side view camera, vehicle vision system The fourth imaging device 504 of 502 is installed on the right side of the vehicle to capture the side-view camera of the vehicle's lateral field of view (FOV) 519,
    第一机械手182上安装第五成像装置406、第二机械手183上安装第六成像装置407、第三机械手184上安装第七 成像装置408和第四机械手185上安装第八成像装置409，在立柱256上安装第九成像装置410，第一成像装置到第九成像装置的成像系统都由视频采集设备120和雷达110组成，雷达110由激光雷达或毫米波雷达组成，The fifth imaging device 406 is installed on the first manipulator 182, the sixth imaging device 407 is installed on the second manipulator 183, the seventh imaging device 408 is installed on the third manipulator 184, and the eighth imaging device 409 is installed on the fourth manipulator 185. The ninth imaging device 410 is installed on the 256. The imaging systems from the first imaging device to the ninth imaging device are all composed of a video acquisition device 120 and a radar 110. The radar 110 is composed of a lidar or millimeter wave radar.
    雷达110用于探测目标采集目标的目标数据和环境坐标，雷达110采用一发双收的FMCW体制，2D-FFT数据处理技术，探测的目标数据包含目标的径向距离、径向速度及角度信息，通过数据特征变换，根据几何关系将径向距离与角度信息转换为目标的横向距离及纵向距离信息，横向距离及纵向距离信息组成目标相对视频采集设备的环境坐标，对于运动目标的检测，雷达每次探测到的目标数据都会不同，为了获得更加准确的目标信息，尽可能的剔除虚假目标，需要采用数据关联及目标跟踪技术，将雷达多次探测到的目标信息进行数据关联并进行自适应滤波预测，当雷达获得准确的目标信息，对探测到的目标建立稳定跟踪时输出视频触发信号，触发摄像机进行图像获取及目标提取，并将雷达探测到的目标转换为相对相机的环境坐标数据传输给雷达视频信息融合系统130进行信息融合，The radar 110 is used to detect the target and collect the target data and environmental coordinates of the target. The radar 110 adopts the FMCW system with one transmitter and two receivers and 2D-FFT data processing technology. The detected target data includes the target's radial distance, radial velocity and angle information , Through data feature transformation, the radial distance and angle information is converted into the target's horizontal distance and vertical distance information according to the geometric relationship. The horizontal distance and the vertical distance information constitute the target's environmental coordinates relative to the video acquisition device. For the detection of moving targets, radar The target data detected each time will be different. In order to obtain more accurate target information and eliminate false targets as much as possible, it is necessary to adopt data association and target tracking technology to associate the target information detected multiple times by the radar and perform adaptation. Filtering prediction. When the radar obtains accurate target information and establishes stable tracking of the detected target, it outputs a video trigger signal, triggers the camera to perform image acquisition and target extraction, and converts the target detected by the radar into the environment coordinate data transmission relative to the camera Perform information fusion for the radar video information fusion system 130,
    视频采集设备120用于在雷达对目标实现跟踪后，采集目标的图像信息和像素坐标，视频采集设备120由摄像机组成，采集图形信息后通过对图像的处理获得目标特征数据，将目标的像素坐标数据等传输给雷达视频信息融合系统130，雷达视频信息融合系统130的输入端通信连接的雷达与视频采集设备，用于对目标的目标数据与图像信息进行信息融合，具体包含将获得的雷达110采集的目标数据进行坐标转换，从环境坐标转换为图像对应的像素坐标，雷达110探测目标位置与视频采集设备120采集的图像信息或视频数据进行时间配准、第一数据关联及决策判决，并将目标融合结果在显示屏上进行显示，The video acquisition device 120 is used to collect the image information and pixel coordinates of the target after the radar is tracking the target. The video acquisition device 120 is composed of a camera. After the image information is collected, the target feature data is obtained by processing the image, and the pixel coordinates of the target Data etc. are transmitted to the radar video information fusion system 130. The radar and video acquisition equipment connected to the input end of the radar video information fusion system 130 are used for information fusion of the target data and image information of the target, including the radar 110 to be obtained. The collected target data undergoes coordinate conversion, from the environment coordinates to the corresponding pixel coordinates of the image, the radar 110 detects the target position and the image information or video data collected by the video acquisition device 120 for time registration, first data association and decision making, and Display the target fusion result on the screen,
    雷达视频复合数据探测与处理系统的探测与处理方法包含以下步骤：The detection and processing method of the radar video composite data detection and processing system includes the following steps:
    S1、雷达探测目标采集目标的目标数据和环境坐标，S1. The radar detects the target and collects the target data and environmental coordinates of the target,
    S1.1、雷达探测目标，对回波数据进行处理获得目标数据，目标数据包含目标的径向距离、径向速度及角度信息，S1.1. The radar detects the target and processes the echo data to obtain target data. The target data includes the radial distance, radial velocity and angle information of the target.
    S1.2、雷达通过数据特征变换，根据几何关系将径向距离与角度信息转换为目标的横向距离及纵向距离，该目标的横向距离及纵向距离组成目标相对视频采集设备的环境坐标，S1.2. The radar converts the radial distance and angle information into the horizontal distance and the vertical distance of the target according to the geometric relationship through data feature transformation. The horizontal and vertical distance of the target constitute the environmental coordinates of the target relative to the video acquisition device.
    S1.3、雷达获取目标的目标数据后，对雷达信息进行第二数据关联，雷达对当前时刻获取的目标数据进行第二数据关联的方法包含：航迹分叉法、最近邻方法、联合概率数据关联算法(JPDA)，雷达判断雷达探测的目标数，若雷达探测的目标数小于预设的数目阈值，目标数少或稀疏，则采用航迹分叉法或最近邻方法进行数据关联，计算简单实时性好，若雷达探测的目标数大于预设的数目阈值，目标数多且密集，则采用联合概率数据关联算法(JPDA)进行数据关联，该算法在杂波环境下有很好的跟踪性能，假设杂波环境下有多个目标存在，并且每个目标的航迹已经形成，如果回波有多个，则认为在跟踪门所有的回波都可能源于目标，只是每个回波源于目标的概率有所不同，S1.3. After the radar obtains the target data of the target, it performs the second data association on the radar information. The methods for the radar to perform the second data association on the target data obtained at the current moment include: track bifurcation method, nearest neighbor method, and joint probability Data association algorithm (JPDA), the radar judges the number of targets detected by the radar. If the number of targets detected by the radar is less than the preset number threshold, and the number of targets is small or sparse, the track bifurcation method or the nearest neighbor method is used for data association and calculation Simple and real-time. If the number of targets detected by the radar is greater than the preset number threshold, and the number of targets is large and dense, the joint probabilistic data association algorithm (JPDA) is used for data association. This algorithm has good tracking in clutter environment Performance, assuming that there are multiple targets in the clutter environment, and the track of each target has been formed, if there are multiple echoes, it is considered that all the echoes at the tracking gate may originate from the target, but each echo source The probability of the target is different,
    S1.4、雷达对当前时刻获取的目标数据进行自适应滤波预测，自适应滤波预测可采用卡尔曼(kalman)滤波跟踪进行目标跟踪预测，将目标，S1.4. The radar performs adaptive filter prediction on the target data acquired at the current moment. The adaptive filter prediction can use Kalman filter tracking to perform target tracking prediction.
    S2、雷达实现目标跟踪后，视频采集设备采集目标的图像信息和像素坐标，S2. After the radar achieves target tracking, the video acquisition device collects the image information and pixel coordinates of the target,
    S2.1、视频采集设备采集目标的图像信息，S2.1. The video capture device captures the image information of the target,
    S2.2、视频采集设备对图像信息进行图像处理，获得目标特征数据，将目标特征数和像素坐标数据等传输给雷达视频信息融合系统，S2.2. The video acquisition device performs image processing on the image information to obtain target feature data, and transmits the target feature number and pixel coordinate data to the radar video information fusion system,
    S3雷达视频信息融合系统将目标的目标数据与图像信息进行信息融合；信息融合包含：坐标变换、时间配准、数据决策和第一数据关联，The S3 radar video information fusion system integrates the target data and image information of the target; information fusion includes: coordinate transformation, time registration, data decision-making and first data association,
    S3.1、雷达视频信息融合系统将雷达获取的目标数据由环境坐标向视频信息对应的像素坐标进行坐标转换，具体包含；环境坐标系Ow-XwYwZw，其原点以视频采集设备垂直于地面的交点为原点Ow(也可设置在任意位置，一般是参照实际情况进行设置)，Yw轴指向视频采集设备采集视频的水平正前方，Zw轴指向垂直于水平面向上，Xw轴位于水平面且垂直于Yw轴，像素坐标系Oo-UV，U轴和Y轴组成成像平面，成像平面垂直于环境坐标系Yw轴，以成像平面左上角为坐标原点Oo，像素坐标系的单位是像素，设视频采集设备离地高度H米时，环境坐标与像素坐标的关系如式(1)：
    Figure PCTCN2020000015-appb-100001
    S3.1. The radar video information fusion system converts the target data obtained by the radar from the environment coordinates to the pixel coordinates corresponding to the video information, which specifically includes; the environment coordinate system Ow-XwYwZw, whose origin is the intersection point of the video acquisition device perpendicular to the ground It is the origin Ow (can also be set at any position, generally set according to the actual situation), the Yw axis points to the horizontal front of the video captured by the video capture device, the Zw axis points upwards perpendicular to the horizontal plane, and the Xw axis lies on the horizontal plane and perpendicular to the Yw axis , The pixel coordinate system Oo-UV, the U axis and the Y axis constitute the imaging plane, the imaging plane is perpendicular to the Yw axis of the environment coordinate system, and the upper left corner of the imaging plane is the coordinate origin Oo. The unit of the pixel coordinate system is pixels. When the ground height is H meters, the relationship between environment coordinates and pixel coordinates is as formula (1):
    Figure PCTCN2020000015-appb-100001
    式(1)中，u为目标在像素坐标系的U轴坐标，v为目标在像素坐标系的V轴坐标，ax、az为视频采集设备Xw轴和Zw轴方向的等效
    Figure PCTCN2020000015-appb-100002
    u0，v0为图像信息的像素中心的坐标，xw，yw，zw分别为相机照射物理范围内的点的环境坐标值，     In formula (1), u is the U-axis coordinate of the target in the pixel coordinate system, v is the V-axis coordinate of the target in the pixel coordinate system, and ax and az are the equivalent of the Xw and Zw axis directions of the video capture device
    Figure PCTCN2020000015-appb-100002
    u0, v0 are the coordinates of the pixel center of the image information, xw, yw, and zw are the environmental coordinates of the points within the physical range of the camera's illumination,
    S3.2、雷达视频信息融合系统对雷达的目标数据和视频采集设备的图像信息进行时间配准，雷达与视频相机数据刷新频率不同，需要将雷达探测目标信息与视频目标提取信息进行时间上的融合，确保配对数据的同步性，发挥好雷达与视频优势互补的作用，一般雷达的数据刷新频率要比摄像机快，可采用基于最小二乘准则的时间配准算法，具体包含：不同种类的传感器C和R，传感器C的采样周期为τ，传感器R的采样周期为T，采样周期的比例系数为整数n，如果距离传感器C的最近一次目标状态估计时刻记为(k-1)τ，则当前时刻表示为kτ＝[(k-1)τ+nT]，意味着在传感器C的一个周期之内，传感器R对目标状态估计的次数为n，最小二乘法时间配准的思路是将传感器R采集到的n次测量值融合为一虚拟测量测，并作为当前时刻传感器R的测量值，利用该测量值与传感器C的测量值进行融合，消除时间偏差引起的目标状态测量值不同步的目的，消除时间不匹配对多传感器信息融合精确度造成的影响，S3.2. The radar video information fusion system performs time registration on the radar target data and the image information of the video acquisition device. The refresh frequency of radar and video camera data is different, and the radar detection target information and the video target extraction information need to be timed. Fusion, to ensure the synchronization of paired data, and play the complementary role of radar and video. Generally, the data refresh rate of radar is faster than that of cameras. The time registration algorithm based on the least square criterion can be used, which specifically includes: different types of sensors C and R, the sampling period of sensor C is τ, the sampling period of sensor R is T, the proportional coefficient of the sampling period is an integer n, if the most recent target state estimation time from sensor C is recorded as (k-1)τ, then The current moment is expressed as kτ=[(k-1)τ+nT], which means that within a period of sensor C, the number of times that sensor R estimates the target state is n, and the idea of least squares time registration is to The n measurements collected by R are fused into a virtual measurement, and used as the measurement value of sensor R at the current moment. The measurement value is fused with the measurement value of sensor C to eliminate the unsynchronized target state measurement value caused by time deviation The purpose is to eliminate the impact of time mismatch on the accuracy of multi-sensor information fusion,
    设视频采集设备的采集周期为τ，雷达的采集周期为T，采集周期的比例系数为整数n；若视频采集设备最近一次 目标状态估计时刻记为(k-1)τ，则当前时刻表示为kτ＝[(k-1)τ+nT]，n为视频采集设备的一个周期之内雷达对目标探测的次数；Suppose the acquisition period of the video acquisition device is τ, the acquisition period of the radar is T, and the scale factor of the acquisition period is an integer n; if the latest target state estimation time of the video acquisition device is recorded as (k-1)τ, the current time is expressed as kτ=[(k-1)τ+nT], n is the number of times that the radar detects the target in one cycle of the video acquisition device;
    将雷达采集到的n次测量值融合为一虚拟测量测，并作为当前时刻雷达的测量值，假设S n＝[S1，S2，...，Sn] T为(k-1)τ到kτ时刻雷达探测到的某一目标位置数据的集合，sn与kτ时刻视频采集数据对应，若用表示S1，S2，...，Sn融合以后的量测值及其导数构成的列向量，则雷达探测数据的虚拟量测值si表示成： Fuse the n measurements collected by the radar into a virtual measurement and use it as the measurement value of the radar at the current moment. Suppose S n =[S1, S2,..., Sn] T is (k-1)τ to kτ A set of target position data detected by the radar at time, sn corresponds to the video collection data at time kτ. If it is used to represent the column vector composed of the measured values of S1, S2,..., Sn fusion and their derivatives, then the radar The virtual measurement value si of the detection data is expressed as:
    Figure PCTCN2020000015-appb-100003
    Figure PCTCN2020000015-appb-100003
    其中c1＝-2/n，c2＝6/[n(n+1)]Where c1=-2/n, c2=6/[n(n+1)]
    当前时刻雷达的测量值与视频采集设备的测量值采用最近邻数据关联方法进行融合，The measured value of the radar at the current moment and the measured value of the video acquisition device are fused using the nearest neighbor data association method.
    S3.3、雷达视频信息融合系统对雷达的目标数据和视频采集设备的图像信息进行数据决策，具体包含：雷达视频信息融合系统判断当前时刻视频采集设备采集的图像信息的图像质量是否大于预设的阈值，若是则采用图像信息提取的目标数目信息，若否则采用雷达采集的目标数据提取的目标数目信息，S3.3. The radar video information fusion system makes data decisions on the target data of the radar and the image information of the video acquisition device, which specifically includes: the radar video information fusion system determines whether the image quality of the image information collected by the video acquisition device at the current moment is greater than the preset If yes, use the target number information extracted from the image information, if otherwise, use the target number information extracted from the target data collected by radar,
    S3.4、雷达视频信息融合系统对雷达的目标数据和视频采集设备的图像信息进行第一数据关联，这里第一数据关联采用最近邻数据关联方法，具体包含：首先，设置跟踪门以限制潜在的决策数目，跟踪门是跟踪空间中的一块子空间，以视频处理或雷达探测目标位置为中心来设置跟踪门，其大小应保证具有一定的正确匹配的概率，因此，残差较大的将首先被剔除，若跟踪门内雷达探测目标数大于1，则残差最小者被看作目标，S3.4. The radar video information fusion system performs the first data association between the radar target data and the image information of the video acquisition device. Here, the first data association uses the nearest neighbor data association method, which specifically includes: First, set up tracking gates to limit potential The tracking gate is a subspace in the tracking space. The tracking gate is set with video processing or radar detection target position as the center. Its size should ensure a certain probability of correct matching. Therefore, the larger residual will be It is first eliminated. If the number of targets detected by the radar in the tracking gate is greater than 1, the one with the smallest residual is regarded as the target
    S3.5、雷达视频信息融合系统显示通过显示屏显示目标融合结果信息；处理器280与通信网络接入系统278连接，通信网络接入系统278包括：处理器、微波通信单元、卫星通信单元和移动通信单元；其中处理器模块适于接收微波通信单元、卫星通信单元和移动通信单元的数据信息，微波通信单元包括：定向天线、射频单元；其中定向天线适于将接收到射频信号发送至射频单元，射频单元适于将射频信号进行调制后，发送给处理器模块进行解调成数据信息；或将数据信息通过处理器模块进行调制后，经过射频单元、定向天线进行发送，卫星通信单元包括：收发器以及Ka频段调制解调器；其中收发器与一特高频UHF天线连接UHF频段信号，处理器用于将接收到的UHF频段信号转换为Ka频段信号，Ka频段调制解调器相连的Ka天线用于向卫星发送转换后的Ka频段信号；或Ka频段调制解调器通过相连的Ka天线接收卫星发送的Ka频段信号，处理器用于将接收到的Ka频段信号转换为UHF频段信号，收发器适于通过特高频UHF天线发送转换后的UHF频段信号，移动通信单元为4G和5G通信模块；处理器适于接收或发送4G和5G信号，有线通讯单元包括：串口通讯电路、CAN总线模块、以太网模块；其中处理器适于接收串口通讯电路和CAN总线模块和以太网模块发送的数据信息，再将上述数据信息转换为Ka频段信号和UHF频段信号；或从Ka频段信号和UHF频段信号中提取出数据信息，通过串口通讯电路、CAN总线模块、以太网模块向外发送，多协议通信网络接入系统将无线通讯方式和有线通讯方式进行结合，即将相应协议的通讯信号发送给处理器，由处理器进行相应协议转换后，经相应通讯方式进行发送，实现了多协议之间的转换，处理器采用STM32系列单片机，STM32F10XC型处理器的21、22、25、26、27、28号引脚分别与以太网模块的36、37、32、33、34、35号引脚相连以进行通信，能够从STM32F10XC型处理器的18、19、20、39、40、41、42、43、45、46号引脚中选择任意引脚用于连接射频单元、收发器和Ka频段调制解调器以及4G和5G通信模块，CAN总线模块采用SN65HVD230型芯片，CAN总线模块1、4号引脚与处理器46、45号引脚电性相连，通过CAN总线模块实现多块处理器进行级联，实现对处理器的拓展，以满足控制多个处理器之间进行通信的需求，串口通讯电路包括：通讯接口、与处理器电性相连的RS485信号通讯电路、RS232信号通讯芯片；其中通讯接口设置有RS485信号通讯电路的输入端和RS232信号通讯芯片的输入端，RS485信号通讯电路的输入端和RS232信号通讯芯片的输入端将数据信息发送至处理器；处理器适于将RS232信号转换为RS485信号，通讯接口的9、10号引脚与处理器30、31号引脚电性相连，通讯接口的3、4号引脚接入RS485信号通讯电路的6、7号引脚，通讯接口的5、6号引脚接入RS232信号通讯芯片的7、8号引脚，RS485信号通讯电路的1、2、4号引脚分别与处理器14、15、16号引脚相连，RS232信号通讯芯片的10、9号引脚分别与处理器12、13号引脚相连，处理器模块内设置有信息分类数据库；处理器模块适于提取数据信息中的关键内容，并在信息分类数据库中进行比对，且根据比对结果进行分类后，按照分类对应的传输方式进行传输，在分类传输时，会将相应待传输的通讯协议加载在数据信息中，以满足相应通讯要求，进而实现多协议之间自动配置，串口通讯电路还包括：与通讯接口电性相连的通信指示电路；通信指示电路设置有第一指示灯、第二指示灯，当与通讯接口相 连的RS485信号通讯电路正常工作时，第一指示灯指示为绿灯亮，以及当与通讯接口相连的RS232信号通讯芯片正常工作时，第二指示灯指示为绿灯亮，多协议通信网络接入系统还包括：DC-DC降压电路；DC-DC降压电路适于对设备供电、稳压，通信网络接入系统278与无线载波系统262连接，无线载波系统262与地面网络264连接，地面网络264与交换机291连接，交换机291与远程控制中心298连接，远程控制中心298与第二处理器215连接，第二处理器215与视觉显示器255连接，备用雷达视频图像传输线路：车辆视觉系统502与处理器280连接，处理器280与通信网络接入系统278连接，通信网络接入系统278与通信卫星289连接，289与上行链路发射站290连接，上行链路发射站290与地面网络264连接，地面网络264与交换机291连接，交换机291与远程控制中心298连接，远程控制中心298与第二处理器215连接，第二处理器215与视觉显示器255连接，车辆视觉系统502的雷达110和视频采集设备120由雷达视频信息融合系统130融合，443扫描雷达视频信息融合系统130的图像传输给压缩存储单元444、压缩存储单元444把图像传输给第一判断单元445，第一判断单元445把图像传输给压缩数据生成单元446，压缩数据生成单元446把压缩好的图像传输给发送模块447，由发送模块447发送给通信网络接入系统278，通信网络接入系统278通过无线载波系统262和地面通信网络264传输给交换机交换机291，交换机291传输给远程控制中心298，远程控制中心298传输给第二处理器215，第二处理器215连接接收模块263，第二处理器215把接收到的图像传输给接收模块263，接收模块263把接收到的图像传输给压缩数据扫描单元449，压缩数据扫描单元449传输给压缩逻辑获取单元450，压缩逻辑获取单元450传输给解压缩读取单元451，解压缩读取单元451传输给第二判断单元452，第二判断单元452传输给原始字节数据恢复单元453，原始字节数据恢复单元453把图像传输给视觉显示器255，数据压缩、解压缩的方法和系统，图17是数据压缩、解压缩接口信息结构图，接口信息可包括压缩类型和原始字节数据的大小，压缩类型的定义可用0表示无压缩，用1表示已压缩，进一步对原始字节数据中存在连续递增的字节数据，以及原始字节数据中存在不连续相同且不连续递增的字节数据进行数据压缩，进一步降低了数据的冗余度，提高了数据传输效率；通过对压缩数据添加接口信息，可保证接收模块正确地完成解压缩处理过程，传输数据经数据压缩后通过无线信道在无线节点411和无线节点414之间进行传输，其中，无线节点411包括发送模块412；无线节点414包括接收模块413；数据压缩功能部署在发送模块413上，数据解压缩功能部署在接收模块413上，数据压缩的方法包括以下步骤：415、扫描原始字节数据，扫描从原始字节数据的第一个字节开始，依次扫描，根据扫描结果，确定出原始字节数据所存在的冗余成分，再针对冗余数据的特点进行下一步的压缩处理步骤，416对原始字节数据进行压缩和存储，若原始字节数据中存在连续相同的字节数据，则与连续相同的字节数据的个数进行第一逻辑运算，得到第一逻辑运算值，将第一逻辑运算值存储为一个字节数据、连续相同的字节数据的任一个字节数据存储为另一个字节数据，例如，当原始字节数据中存在连续相同的字节数据有3个，且分别是0x05，0x05，0x05时，则第一逻辑运算是，使用0x80与连续相同的字节数据的个数0x03进行逻辑“或”运算得到第一逻辑运算值0x83，然后所得到的第一逻辑运算值0x83存储为一个字节数据，将连续相同的字节数据0x05存储为另一个字节数据，417判断原始字节数据是否扫描完成，若扫描完成，则转向步骤418；若未扫描完成，则返回步骤415继续扫描，418根据存储的字节数据，生成压缩数据，步骤416中针对原始字节数据中存在连续相同的字节数据进行压缩处理，使得原来占用3个字节数据的字节数据经数据压缩处理后，只占用2个字节数据，0x83，0x05即为压缩数据，由上述技术方案可知，通过步骤a：扫描原始字节数据b：若原始字节数据中存在连续相同的字节数据，则与连续相同的字节数据的个数进行第一逻辑运算，得到第一逻辑运算值，将第一逻辑运算值存储为一个字节数据、连续相同的字节数据的任一个字节数据存储为另一个字节数据c：判断原始字节数据是否扫描完成，若是，转向步骤d，若否，返回步骤a；d：根据存储的字节数据，生成压缩数据，首先根据步骤419确定原始字节数据中的数据特点；若由步骤420，即当确定原始字节数据中存在连续递增的字节数据；则可执行步骤421，即与连续递增的字节数据的个数进行第二逻辑运算，得到第二逻辑运算值；接着执行步骤422，即将第二逻辑运算值存储为一个字节数据、连续递增的字节数据的第一个字节数据存储为另一个字节数据，其中，第二逻辑运算，是使用0xC0与连续递增的字节数据的个数进行“或”运算，若由步骤423，即当确定原始字节数据中存在不连续相同且不连续递增的字节数据；则可执行步骤424，即与不连续相同且不连续递增的字节数据的个数进行第三逻辑运算，得到第三逻辑运算值；接着执行步骤425，即将第三逻辑运算值存储为一个字节数据、不连续相同且不连续递增的字节数据的每一个字节数据依次各存储为另一个字节数据，其中，第三逻辑运算，是使用0x00与不连续相同且不连续递增的字节数据的个数进行“或”运算，最后根据步骤426获得存储的字节数据，根据存储的字节数据，生成压缩数据，在步骤418之后还可增加以下步骤：419对压缩数据添加接口信息，生成压缩数据包，数据解压缩的方法，当无线节点414的接收模块413接收到对方无线节点发送过来的压缩数据后，可使用数据解压缩的方法进行解压缩处理，然后把解压缩后的数据交由本无线节点做后续处理，数据解压缩的方法可包括以下步骤：427扫描压缩数据，压缩数据是原始字节数据通过实施例一中的数据压缩方法进行压缩后得到的数据，428对压缩数据的第n个字节数据进行压缩逻辑判断运算，获取压缩逻辑判断值和压缩逻辑，其中，n为大于等于1的自然数，429、根据压缩逻辑判断值和压缩逻辑，对压缩数据进行解压缩和读取，若压缩逻辑判断值等于第一预设值，则判断压缩逻辑对应原始字节数据中存在连续相同的字节数据；对第n个字节数据进行第一逻辑个数运算，得到数据个数i，其中，i为大于等于2的自然数，重复读取i个压缩数据的第n+1个字节数据，430判断压缩数据是否扫描完成，若扫描完成，转向步骤431；若未扫描完成，则返回步骤427继续扫描，431根据读取的字节数据，恢复原始字节数据，如步骤429，原始字节数据是读取的字节数据0x05，0x05，0x05，在步骤427的基础上，即在根据压缩逻辑判断值和压 缩逻辑，对压缩数据进行解压缩和读取的基础上，进一步提供了压缩逻辑判断值等于第二预设值，即压缩逻辑对应原始字节数据中存在连续递增的字节数据；以及压缩逻辑判断值不等于第一预设值且不等于第二预设值，即压缩逻辑对应原始字节数据中存在不连续相同且不连续递增的字节数据的情况下，对压缩数据进行解压缩和读取，首先根据步骤432，确定压缩逻辑判断值，若由步骤433，确定压缩逻辑判断值等于第二预设值，则由步骤434，可确定压缩逻辑对应原始字节数据中存在连续递增的字节数据，进一步执行步骤435，对压缩数据的第n个字节数据进行第二逻辑个数运算，得到数据个数j，其中，j为大于等于2的自然数；最后根据步骤436，从压缩数据的第n+1个字节数据开始，依次读取j个字节数据，其中，第二预设值是0xC0；第二逻辑个数运算，是使用0x38与第n个字节数据进行“或”运算，若由步骤437，确定压缩逻辑判断值不等于第一预设值且不等于第二预设值，则由步骤438，可确定压缩逻辑对应原始字节数据中存在不连续相同且不连续递增的字节数据，进一步执行步骤439，对压缩数据的第n个字节数据进行第三逻辑个数运算，得到数据个数k，其中，k为大于等于2的自然数；最后，根据步骤440，即从压缩数据的第n+1个字节数据开始，依次读取k个字节数据，其中，第三逻辑个数运算，是使用0x00与第n个字节数据进行“或”运算，最后根据步骤441、获取读取的字节数据，恢复原字节数据，数据压缩的系统442包括：雷达110，视频采集设备120，雷达视频信息融合系统130，雷达110由激光雷达和毫米波雷达组成，原始字节数据扫描单元443，压缩存储单元444，第一判断单元445和压缩数据生成单元446，发送模块447，原始字节数据扫描单元443，用于扫描原始字节数据；压缩存储单元444，用于对原始字节数据进行压缩和存储；第一判断单元445，用于判断原始字节数据是否扫描完成；压缩数据生成单元446，用于根据存储的字节数据，生成压缩数据，压缩好的数据传输给发送模块447，S3.5. The radar video information fusion system displays the target fusion result information through the display screen; the processor 280 is connected to the communication network access system 278, and the communication network access system 278 includes: a processor, a microwave communication unit, a satellite communication unit and Mobile communication unit; wherein the processor module is suitable for receiving data information of microwave communication unit, satellite communication unit and mobile communication unit, microwave communication unit includes: directional antenna, radio frequency unit; wherein the directional antenna is suitable for sending the received radio frequency signal to the radio frequency The radio frequency unit is adapted to modulate the radio frequency signal and send it to the processor module for demodulation into data information; or after the data information is modulated by the processor module, it is sent through the radio frequency unit and the directional antenna. The satellite communication unit includes : Transceiver and Ka-band modem; among them, the transceiver is connected with a UHF UHF antenna to UHF-band signals, the processor is used to convert the received UHF-band signals into Ka-band signals, and the Ka antenna connected to the Ka-band modem is used to send signals to the satellite Transmit the converted Ka-band signal; or the Ka-band modem receives the Ka-band signal sent by the satellite through the connected Ka antenna. The processor is used to convert the received Ka-band signal into a UHF-band signal. The transceiver is suitable for passing UHF UHF The antenna sends the converted UHF frequency band signal, the mobile communication unit is 4G and 5G communication modules; the processor is suitable for receiving or sending 4G and 5G signals, the wired communication unit includes: serial communication circuit, CAN bus module, Ethernet module; among which processing The device is suitable for receiving the data information sent by the serial communication circuit and the CAN bus module and the Ethernet module, and then converting the above data information into Ka band signals and UHF band signals; or extracting data information from Ka band signals and UHF band signals, Send out through the serial communication circuit, CAN bus module, and Ethernet module. The multi-protocol communication network access system combines the wireless communication method and the wired communication method, that is, the communication signal of the corresponding protocol is sent to the processor, and the processor performs the corresponding After the protocol is converted, it is sent through the corresponding communication method to realize the conversion between multiple protocols. The processor adopts STM32 series single-chip microcomputer. The 21, 22, 25, 26, 27, 28 pins of the STM32F10XC processor are connected to the Ethernet The 36, 37, 32, 33, 34, 35 pins of the module are connected for communication, and it can be connected to the 18, 19, 20, 39, 40, 41, 42, 43, 45, 46 pins of the STM32F10XC processor Choose any pin among the RF units, transceivers and Ka-band modems, and 4G and 5G communication modules. The CAN bus module uses SN65HVD230 chip, CAN bus module 1, 4 pins and processor 46, 45 pins Electrically connected, cascading multiple processors through the CAN bus module to achieve the expansion of the processors to meet the needs of controlling communication between multiple processors. The serial communication circuit includes: Communication interface, RS485 signal communication circuit electrically connected to the processor, RS232 signal communication chip; the communication interface is provided with the input end of the RS485 signal communication circuit and the input end of the RS232 signal communication chip, the input end of the RS485 signal communication circuit and RS232 The input terminal of the signal communication chip sends data information to the processor; the processor is suitable for converting RS232 signals into RS485 signals, the 9th and 10th pins of the communication interface are electrically connected to the 30th and 31st pins of the processor, and the communication interface The 3 and 4 pins of the RS485 signal communication circuit are connected to the 6 and 7 pins, the 5 and 6 pins of the communication interface are connected to the 7 and 8 pins of the RS232 signal communication chip, and the RS485 signal communication circuit is 1 Pins 2, 4, and 4 are connected to pins 14, 15, and 16 of the processor, and pins 10 and 9 of the RS232 signal communication chip are connected to pins 12 and 13 of the processor. The processor module is equipped with Information classification database; the processor module is suitable for extracting the key content of the data information, and comparing it in the information classification database, and after classifying according to the comparison result, it transmits according to the transmission mode corresponding to the classification. When classifying and transmitting, The corresponding communication protocol to be transmitted will be loaded into the data information to meet the corresponding communication requirements, thereby realizing automatic configuration between multiple protocols. The serial communication circuit also includes: a communication indication circuit electrically connected to the communication interface; communication indication circuit settings There are a first indicator light and a second indicator light. When the RS485 signal communication circuit connected to the communication interface is working normally, the first indicator light indicates a green light, and when the RS232 signal communication chip connected to the communication interface is working normally, the first indicator light is on. The second indicator light is green. The multi-protocol communication network access system also includes: DC-DC step-down circuit; DC-DC step-down circuit is suitable for powering and stabilizing equipment, communication network access system 278 and wireless carrier system 262 connection, the wireless carrier system 262 is connected to the ground network 264, the ground network 264 is connected to the switch 291, the switch 291 is connected to the remote control center 298, the remote control center 298 is connected to the second processor 215, and the second processor 215 is connected to the visual display 255 connection, spare radar video image transmission line: vehicle vision system 502 is connected to processor 280, processor 280 is connected to communication network access system 278, communication network access system 278 is connected to communication satellite 289, and 289 is connected to uplink transmission The station 290 is connected, the uplink transmitting station 290 is connected to the ground network 264, the ground network 264 is connected to the switch 291, the switch 291 is connected to the remote control center 298, the remote control center 298 is connected to the second processor 215, and the second processor 215 Connected to the visual display 255, the radar 110 of the vehicle vision system 502 and the video acquisition device 120 are fused by the radar video information fusion system 130, and the image of the 443 scanning radar video information fusion system 130 is transmitted to the compression storage The storage unit 444 and the compression storage unit 444 transmit the image to the first judging unit 445, the first judging unit 445 transmits the image to the compressed data generating unit 446, and the compressed data generating unit 446 transmits the compressed image to the sending module 447. The sending module 447 sends to the communication network access system 278, the communication network access system 278 transmits to the switch switch 291 through the wireless carrier system 262 and the ground communication network 264, the switch 291 transmits to the remote control center 298, and the remote control center 298 transmits to the first The second processor 215, the second processor 215 is connected to the receiving module 263, the second processor 215 transmits the received image to the receiving module 263, the receiving module 263 transmits the received image to the compressed data scanning unit 449, and the compressed data scans The unit 449 is transmitted to the compression logic acquisition unit 450, the compression logic acquisition unit 450 is transmitted to the decompression reading unit 451, the decompression reading unit 451 is transmitted to the second judgment unit 452, and the second judgment unit 452 is transmitted to the original byte data recovery Unit 453, the original byte data recovery unit 453 transmits the image to the visual display 255, the method and system of data compression and decompression, Figure 17 is a data compression and decompression interface information structure diagram, the interface information can include compression type and original words The size of the section data and the definition of the compression type can be 0 for no compression, and 1 for compressed. Further, there are continuously increasing byte data in the original byte data, and the original byte data is discontinuous and discontinuous. Data compression of the byte data further reduces the redundancy of the data and improves the efficiency of data transmission; by adding interface information to the compressed data, it can ensure that the receiving module correctly completes the decompression process, and the transmitted data passes through the compressed data. The wireless channel is transmitted between the wireless node 411 and the wireless node 414. The wireless node 411 includes a sending module 412; the wireless node 414 includes a receiving module 413; the data compression function is deployed on the sending module 413, and the data decompression function is deployed on the receiving module. On the module 413, the method of data compression includes the following steps: 415. Scan the original byte data. The scan starts from the first byte of the original byte data and scans sequentially. According to the scanning result, determine the existence of the original byte data. For redundant components, the next step of compression processing is carried out according to the characteristics of redundant data. 416 compresses and stores the original byte data. If there are consecutive identical byte data in the original byte data, it will be the same as the consecutive identical words. Perform the first logical operation on the number of section data to obtain the first logical operation value, store the first logical operation value as one byte of data, and store any one byte of continuous identical byte data as another byte of data For example, when there are 3 consecutive identical byte data in the original byte data, and they are 0x05, 0x05, 0x05, then the first logical operation is to use 0x80 and the number of consecutive identical byte data 0x03 get on The logical "or" operation obtains the first logical operation value 0x83, and then the obtained first logical operation value 0x83 is stored as one byte of data, the continuous same byte data 0x05 is stored as another byte data, and 417 judges the original word Whether the scan of the data is completed, if the scan is completed, go to step 418; if the scan is not completed, return to step 415 to continue scanning, 418 generates compressed data according to the stored byte data, and in step 416, there is continuous The same byte data is compressed, so that the byte data that originally occupied 3 bytes of data after the data compression process only occupies 2 bytes of data. 0x83 and 0x05 are compressed data. According to the above technical solution, Step a: Scan the original byte data b: If there are consecutive identical byte data in the original byte data, perform the first logical operation with the number of consecutive identical byte data to obtain the first logical operation value, and set the A logical operation value is stored as one byte of data, and any byte data of the same continuous byte data is stored as another byte data c: Determine whether the original byte data has been scanned, if yes, go to step d, if not, Return to step a; d: generate compressed data according to the stored byte data, first determine the data characteristics in the original byte data according to step 419; if it is step 420, that is, when it is determined that there are continuously increasing bytes in the original byte data Data; Step 421 may be performed, that is, the second logical operation is performed with the number of continuously increasing byte data to obtain the second logical operation value; then step 422 is performed, that is, the second logical operation value is stored as a byte of data, The first byte data of the continuously increasing byte data is stored as another byte data. The second logic operation is to use 0xC0 and the number of continuously increasing byte data to perform an "OR" operation. 423, that is, when it is determined that there are discontinuously identical and discontinuously increasing byte data in the original byte data; then step 424 may be performed, that is, the third logical operation is performed with the number of discontinuously identical and discontinuously increasing byte data , Obtain the third logical operation value; then perform step 425, that is, store the third logical operation value as one byte of data, and each byte of data that is discontinuously identical and discontinuously increasing is sequentially stored as another word Section data, the third logical operation is to use 0x00 and the number of discontinuously identical and discontinuously increasing byte data to perform an "OR" operation, and finally obtain the stored byte data according to step 426, according to the stored byte Data, generate compressed data, after step 418, you can add the following steps: 419 add interface information to the compressed data, generate compressed data packets, data decompression method, when the receiving module 413 of the wireless node 414 receives the send from the other wireless node After the compressed data, you can use the data decompression method for decompression processing, and then hand over the decompressed data to the wireless node for subsequent processing. The data decompression method can include the following steps: 427 scan the compressed data, the compressed data is Raw bytes The data is compressed by the data compression method in the first embodiment, 428 performs a compression logic judgment operation on the nth byte of the compressed data to obtain the compression logic judgment value and the compression logic, where n is greater than or equal to 1. The natural number of 429. According to the compression logic judgment value and the compression logic, the compressed data is decompressed and read. If the compression logic judgment value is equal to the first preset value, it is judged that the compression logic corresponds to the same consecutive original byte data Byte data; perform the first logical number operation on the nth byte data to obtain the data number i, where i is a natural number greater than or equal to 2, and repeatedly read the n+1th byte of i compressed data Data, 430 judge whether the scanning of compressed data is completed, if the scanning is completed, go to step 431; if the scanning is not completed, return to step 427 to continue scanning, 431 restore the original byte data according to the read byte data, such as step 429, original The byte data is the read byte data 0x05, 0x05, 0x05. On the basis of step 427, that is, on the basis of decompressing and reading the compressed data according to the compression logic judgment value and the compression logic, it further provides The compression logic judgment value is equal to the second preset value, that is, there is continuously increasing byte data in the original byte data corresponding to the compression logic; and the compression logic judgment value is not equal to the first preset value and not equal to the second preset value, that is, The compression logic corresponds to the case where there are discontinuous identical and discontinuously increasing byte data in the original byte data, decompress and read the compressed data. First, determine the compression logic judgment value according to step 432. If step 433, If it is determined that the compression logic judgment value is equal to the second preset value, it can be determined in step 434 that there is continuously increasing byte data in the original byte data corresponding to the compression logic, and step 435 is further executed to perform step 435 on the nth byte data of the compressed data. Perform the second logical number operation to obtain the number of data j, where j is a natural number greater than or equal to 2; finally, according to step 436, starting from the n+1th byte of the compressed data, read j bytes in sequence Data, where the second preset value is 0xC0; the second logical number operation is an OR operation using 0x38 and the n-th byte data. If step 437, it is determined that the compression logic judgment value is not equal to the first preset If the value is set and is not equal to the second preset value, then in step 438, it can be determined that there are discontinuously identical and discontinuously increasing byte data in the original byte data corresponding to the compression logic, and step 439 is further performed to determine the nth of the compressed data. Perform the third logical number operation on each byte data to obtain the data number k, where k is a natural number greater than or equal to 2; finally, according to step 440, starting from the n+1th byte data of the compressed data, Read k bytes of data, where the third logical number operation is to use 0x00 and the nth byte of data to perform an "OR" operation. Finally, according to step 441, obtain the read byte data and restore the original byte The data compression system 442 includes: radar 110, video acquisition device 120, radar video Frequency information fusion system 130, radar 110 is composed of lidar and millimeter wave radar, original byte data scanning unit 443, compression storage unit 444, first judgment unit 445 and compressed data generation unit 446, sending module 447, original byte data The scanning unit 443 is used for scanning the original byte data; the compression storage unit 444 is used for compressing and storing the original byte data; the first judging unit 445 is used for judging whether the scanning of the original byte data is completed; the compressed data generating unit 446, used to generate compressed data according to the stored byte data, and transmit the compressed data to the sending module 447,
    数据解压缩的系统448包括接收模块263，压缩数据扫描单元449，压缩逻辑判断值和压缩逻辑获取单元450，解压缩读取单元451，第二判断单元452和原始字节数据恢复单元453，其中压缩数据扫描单元449，用于扫描压缩数据；压缩逻辑判断值和压缩逻辑获取单元450，用于对压缩数据的第n个字节数据进行压缩逻辑判断运算，获取压缩逻辑判断值和压缩逻辑，其中n为大于等于1的自然数；解压缩读取单元451，用于根据压缩逻辑判断值和压缩逻辑，对压缩数据进行解压缩和读取；第二判断单元452，用于判断压缩数据是否扫描完成；原始字节数据恢复单元453，用于根据读取的字节数据，恢复原始字节数据，显示器255显示原始字节数据恢复单元453恢复的原始字节数据通过以上连接建立的各个子系统，远程操作员171用以下的系统远程驾驶车辆260行走：第一连杆139的远端在提供水平枢转轴线138的关节处被连接到第二连杆137的近端，第三连杆124的近端在滚动关节处被连接到第二连杆137的远端，使得第三连杆通常围绕沿着第二连杆和第三连杆两者的轴线延伸的轴线在关节123处旋转或滚动，在枢轴关节125之后向远侧进行，第四连杆136的远端通过一对枢轴关节135、134连接到器械保持器136，枢轴关节135、134一起限定器械保持器121，机器人170操纵器臂组件133的平移或棱柱型关节132便于器械126轴向移动，能够将器械保持器131附接到插管，穿过该插管器械126可滑动地被插入，在器械保持器131的远侧，第二器械126包括额外的自由度，第二器械126的自由度的致动由机器人操纵器臂组件133的马达驱动，第二器械126和机器人操纵器臂组件133之间的接口可沿着操纵器臂组件133的运动链更近侧或更远侧地设置，第二器械126包括枢轴点PP的近侧的旋转关节130，其设置在需要的部位处，第二器械126的远侧允许末端执行器128围绕器械腕关节轴线129、127进行枢转运动，能够独立于末端执行器128的位置和取向来控制末端执行器钳口231之间的角度θ，左手持输入装置177和右主输入装置178通过无线通信与控制台169连接和分离，左手持输入装置177与第二处理器215连接，右手持输入装置178与第二处理器215连接，远程操作员171在远程控制台169激活第二处理器215后开始执行远程驾驶工作，远程操作员171的左手控制左手持输入装置177，左手持输入装置177通过第二处理器215控制手臂端197的移动，远程操作员171的右手控制右手持输入装置178，右手持输入装置178通过第二处理器215控制手臂端197的移动，手臂端197使用末端执行器193中的第一接触端194和第二接触端196与方向盘235接触并握紧，通过左手持输入装置177和右手持输入装置178向相反的方向运动能够使方向盘235转动，远程操作员171使用远程控制台169的第二处理器215软件控制机器人170的第一机械手182和第二机械手183，远程操作员171通过测量、模型估计、测量和建模来确定施加在机器人170第一机械手182和第二机械手183上用在方向盘235上的力，第一机械手182和第二机械手183通过远程控制台169向远程操作员171提供触觉反馈，这种触觉反馈能够为远程操作员171模拟手动操纵手臂端197控制方向盘235，能够为远程操作员171模拟由机器人170第一机械手182经历的对应于方向盘235的反作用力，末端执行器193中的第一接触端194和第二接触端196，它们相对于彼此枢转，以便限定一对末端执行器钳口231，对于具有末端执行器钳口231的器械，通过挤压输入装置177、178的抓手构件来致动钳口231，机器人170操纵第一机械手182和第二机械手183将使传动组件195在方向盘235上部移动，使得轴187延伸和收回，提供末端执行器193的期望移动，机器人170操纵第一机械手182和第二机械手183、第三机械手184和第四机械手185在远程驾驶过程中能够在方向盘235、档位400处、刹车401踏板和油门踏板402处移动，第一机械手182和第二机械手183握紧方向盘235执行汽车方向的变化，第一机械手182上连接有器械保持器180，器械保持器180与器械186和手臂端197连接，器械保持器180与第一机械手182依靠机动化的关节连接，器械保持器180包括器械保持器框架188、夹具189和器械保持器托架190，夹具189被固定到器械保持器框架188的远端，夹具189能够与手臂端197连接和分离，器械保持器托架190与器械保持器框架188连接，器械保持器托架190沿器械保持器框架188的线性平移是由第二处理器215控制的机动化的平移移动，器械186包括传动组件195、细长轴187和末端执行器193，传动组件195与器械保持器托架190连接，轴187从传动组件195向远侧延伸，末端执行器193设置 在轴187的远端处，轴187限定纵向轴线192，纵向轴线192与手臂端197的纵向轴线重合并且与由手臂端197限定的纵向轴线重合，当器械保持器托架190沿着器械保持器框架188平移时，器械186的细长轴187沿着纵向轴线192移动，末端执行器193能够从工作空间延长和缩回，远程操作员171通过第二处理器215发送指示，三态开关202接收到激活信号，远程操作员171使用第二处理器215和车辆远程驾驶体系258连接机器人170操纵第一机械手182手臂端197做握紧和远离方向盘235的移动，远程操作员171使用第二处理器215和车辆远程驾驶体系258连接机器人170操纵第二机械手183手臂端197做握紧和远离方向盘235的移动，末端执行器193中的第一接触端194和第二接触端196向方向盘235上施加力，使方向盘235转动，释放三态开关202将手臂端197停止移动，需要手臂端197与方向盘235连接时，远程操作员171发送激活第二方向信号，第一方向与第二方向相反，三态开关202接收到激活第二方向信号，手臂端197向方向盘235移动，151是将主输入装置152连接到从操纵器154的主/从控制器153的简化控制器示意图，使用矢量数学符号来描述控制器输入、输出和计算，其中矢量X将参考笛卡尔坐标中的方位矢量，并且其中矢量q将参考相关联的联动装置的关节铰接配置矢量，有时称为关节空间中的联动装置方位，当存在歧义时，能够将下标附加到这些矢量以识别特定结构，使得
    Figure PCTCN2020000015-appb-100004
    是主输入装置在相关联的主工作空间或坐标系中的方位，而x s表示该工作空间中的从动件的方位，与方位矢量相关联的速度矢量由矢量上方的点或矢量和下标之间的单词“dot”表示，例如主速度矢量的xdot m其中速度矢量在数学上被定义为方位矢量随时间变化的变化，控制器153包含逆雅可比速度控制器，在
    Figure PCTCN2020000015-appb-100005
    是主输入装置的方位并是主输入装置的速度的情况下，控制器153计算用于传输到从操纵器154的动力命令，以实现从主速度对应于输入装置的从末端执行器运动，控制器153能够计算从从方位x s和从速度施加到主输入装置和从那里到远程操作员171的手的力反射信号，第一模块159包含逆雅可比速度控制器，其具有来自使用根据虚拟从路径163修改的逆雅可比矩阵进行的计算的输出，首先描述虚拟从路径，与虚拟从动件相关联的矢量通常用v下标表示，使得关节空间qdot v中的虚拟从速度被积分以提供q v，使用逆运动模块162处理q v以生成虚拟从关节方位信号x v，虚拟从方位和主输入命令x m被组合并使用正向运动161进行处理，虚拟从动件的使用有助于在接近系统的硬限制时、在超越系统的软限制时等等进行平滑控制和力反射，由第一控制模块159和第二控制模块160指示的以及控制示意图165的其他部件和其他控制器的结构，该结构包含数据处理硬件、软件和固件，这样的结构包括可重新编程的软件和数据，其体现在机器可读代码中并存储在有形介质中，以供远程控制台169的第二处理器215使用机器可读代码以各种不同的配置存储，包括随机存取存储器、非易失性存储器、一次写入存储器、磁记录介质和光学记录介质，体现代码和与其相关联的数据的信号通过各种通信链路传输，该通信链路包括因特网、内联网、以太网、无线通信网络和链路、电信号和导体和光纤和网络，第二处理器215包含远程控制台169的一个和多个数据处理器，包括操纵器、器械、单独和远程处理结构和位置中的一个和多个的局部数据处理电路，模块包含单个公共处理器板、多个单独的板，模块中的一个和多个分散在多个板上，其中一些板还运行另一个模块的一些和全部计算，模块的软件代码被编写为单个集成软件代码，每个模块分成单独的子程序，或者一个模块的部分代码与另一个模块的一些或全部代码组合，数据和处理结构包括各种集中式或分布式数据处理和编程体系结构中的任何一种，控制器的输出，该控制器将经常试图求解一个特定的操纵器关节配置矢量q，以用于为这些高度可配置的从操纵器机构生成命令，操纵器联动装置通常具有足够的自由度，以便占据给定末端执行器状态的一系列关节状态，其中一个关节的致动直接被沿着运动链的不同关节的类似致动替换的结构，这些结构有时被称为具有过剩、额外或冗余的自由度，同时这些术语通常涵盖运动链，其中中间连杆能够移动而不改变末端执行器的方位，使用图40的速度控制器引导高度可配置的操纵器的移动时，第一模块的主要关节控制器经常试图确定或求解虚拟关节速度矢量qdot v，其能够被用于以使得末端执行器将准确地遵循主命令x m的方式驱动从操纵器164的关节，对于具有冗余自由度的从机构，逆雅可比矩阵通常不完全定义关节矢量解，在能够占据给定末端执行器状态的一系列关节状态的系统中，从笛卡尔命令xdot到关节运动qdot的映射是一对多的映射，因为机构是冗余的，所以存在数学上无数个解，其由逆生存的子空间表示，控制器使用列比行更多的雅可比矩阵来体现这种关系，将多个关节速度映射到相对较少的笛卡尔速度，通过由软件约束的远程运动中心298的概念被确定，通过具有计算软件枢轴点的能力，能够选择性地实现以系统的顺应性或刚度为特征的不同模式，在计算估计的枢轴点之后，实现在一定范围的枢轴点/中心上的不同系统模式，在固定枢轴实施方式中，能够将估计的枢轴点与期望的枢轴点进行比较以生成误差输出，该误差输出能够被用于将器械的枢轴驱动到期望的位置，相反地，在被动枢轴实施方式中，虽然期望的枢轴位置可能不是最重要的目标，但是估计的枢轴点能够被用于误差检测并因此用于安全性，因为估计的枢轴点位置的变化指示与方向盘分离或者传感器出现故障，从而使系统有机会采取纠正措施，处理器157包括第一控制器模块157和第二控制器模块160，第一模块157能够包含主要关节控制器，逆雅可比主-从控制器，第一模块157的主要关节控制器能够被配置用于响应于来自主输入装置156的输入产生期望的操纵器组件运动，操纵器联动装置具有一系列用于在空间中的给定末端执行器方位的替代配置，用于使末端执行器呈现给定方位的命令能够引起各种不同的关节运动和配置，第二模块160能够被配置为帮助将操纵器组件驱动到期望的配置，在主-从运动期间将操纵器朝向优选配置驱动，第二模块160将包含配置相关过滤器，第一模块157的主要关节控制器和第二模块160的配置相关过滤器都能够包含由处理器157使用来将关节的线性组合的控制权限传送到一个和多个目标或任务的实现的过滤器，假设X是关节运动的空间，则F(X)能够是对关节进行控制的过滤器，以i)提供期望的末端执行器运动，以及ii)在孔部位处提供器械轴的枢转运动，第一模块157的主要关节控制器可包含过滤器F(X)，从概念上讲，(1-F-1F)(X)能够描述配置相关的子空间过滤器，其给予与实现主要关节控制器的目标正交的 关节速度的线性组合的控制致动权限，这种配置相关过滤器能够由控制器157的第二模块160使用以实现第二目标，两个过滤器能够进一步细分为对应于实现更具体任务的更多过滤器，过滤器F(X)能够分成F1(X)和F2(X)，分别用于控制末端执行器和控制枢转轴运动，其中任何一个都能够被选择作为处理器的最高优先级任务，机器人处理器和控制技术将经常利用经配置用于第一控制器任务的主要关节控制器，以及配置相关过滤器，该配置相关过滤器利用由主要关节控制器生成的欠约束解以用于第二任务，参考第一模块描述主要关节控制器，而将参考第二模块描述配置相关过滤器，还能够包括附加功能和各种优先级的附加模块，第一模块和第二模块功能的硬件和编程代码，是完全集成、部分集成能够完全分离的，控制器157能够同时使用两个模块的功能，能够具有多种不同模式，其中一个或两个模块分开使用或以不同的方式使用，在主-从操纵期间，第一模块157能够在很少或没有来自第二模块160的影响的情况下使用，并且当末端执行器未被机器人驱动时，第二模块160在系统装配期间具有更大的作用，两个模块均能够在启用机器人运动的大部分或全部时间是活动的，通过将第一模块的增益设置为零，通过将x s设置为x s，actual和通过降低逆雅可比控制器中的矩阵秩使其不能控制太多并使配置相关过滤器具有更多控制权限，能够减少或消除第一模块对操纵器组件的状态的影响，从而将处理器157的模式变为抓持模式，第一模块157能够包含具有雅可比相关矩阵的某种形式的雅可比控制器，在端口抓持模式下，第二模块160能够从从操纵器158接收信号，该信号指示至少部分地由从操纵器联动装置的手动铰接产生的从动件的方位或速度，响应于该输入，第二模块160能够产生适于驱动从动件的关节的动力命令，以便允许从联动装置的手动铰接，同时配置处于期望的关节配置的从动件，在主-从末端执行器操纵期间，控制器能够使用第二模块160来帮助基于不同的信号bqdot o导出动力命令，这种到控制器157的第二模块160的替代输入信号可用于驱动操纵器联动装置，以便沿着操纵器结构维持或移动微创孔枢轴位置，从而避免多个操纵器之间的碰撞，从而增加操纵器结构的运动范围和避免奇点，以便产生操纵器的期望姿势等，使用来自MTM控制器的输入主动控制远程运动中心(RC)、手臂端197(C)和器械末端执行器(E)参照系的框图231，使用来自主操纵器控制器的输入主动控制器械末端执行器(E)系，同时使用次要输入装置控制远程中心(RC)和手臂端197(C)系的框图232，次要输入装置使用任意参照，而不一定是目系(EYE系)，参照系变换EYETREF能够被直接测量或由间接测量值计算，信号调节单元将这些输入组合在适当的公共系中，以供从操纵器控制器使用，有三个待由系统的控制器控制的参照系，参照系之一(C)是手臂端197的参照系，假设EYETE由主工具操纵器(MTM)控制器命令，远程中心参照系和手臂端197参照系的姿势规范来自以下来源中的一个或组合：(i)MTM控制器指定这些系/参照系在EYE系中，即 EVET RC和 EYET C，(ii)次要装置命令这些系姿势在方便的参照系中，即 REFT RC和 REFT C(其中能够确定 EYET RFF)，以及(iii)从侧控制器指定这些姿势在从臂的基系(base frame)中，即 WT RC和 WT C，系统212和213的示意框图，系统212和213用于使用计算机辅助的车辆远程驾驶体系258的第二处理器215来控制器械末端执行器193参照系和远程控制中心298参照系之间的关系，假设手臂端197参照系和远程控制中心298参照系重合，手臂端197参照系和远程控制中心298在物理上被约束为相对于器械末端执行器193仅沿着手臂端197和器械的纵向轴线移动，采用两种不同的策略，以控制器械末端执行器193的参照系(E系)和远程控制中心298的参照系(RC系)之间的关系，用于主动控制两个参照系无论E系是固定的还是移动的之间的相对距离(d)的一种策略使用来自力/扭矩传感器或三态开关的输入，使用框图来实现用于该模式的控制子系统，该控制子系统能够被描述为‘相对姿势控制器’，使用三态开关相对控制距尖端的距离的一般框图，对从操纵器的增量笛卡尔命令“slv_cart_delta”表示如下：slv_cart_delta＝S*slv_cart_vel*Ts(其中Ts是控制器的采样时间)，S取值[1，-1，0]，这取决于它命令控制方向盘235、档位400、刹车踏板401和油门踏板402的运动，力/扭矩或压力传感器相对控制手臂端197距尖端的距离的一般框图，对从操纵器的增量笛卡尔命令“slv_cart_delta”能够表示如下：slv_cart_delta＝f(F，p)“f”是使用感测到的力或压力F和一些用户定义的参数p作为输入的可编程函数，这是一个导纳控制器，这能够通过借助于关节扭矩传感器和臂运动学知识估计沿手臂端197轴线的笛卡尔力来实现，在这种估计之后，计算的估计值能够用作输入F以命令增量运动，信号F能够是基于用户与操纵器的交互的任何其他测量或计算的力的量，能够独立地控制RC系和E系的轨迹，其中管理这些轨迹的控制输入可能都来自主操纵器，该附加策略的控制子系统框图能够被称为‘独立姿势控制器’，能够概述插入(I/O)运动以允许远程控制中心298或手臂端197相对于器械尖端E的横向运动，远程控制中心298或手臂端197将需要围绕尖端枢转，同时驱动器械以补偿E的运动，这将允许RC和手臂端197在驾驶室内的运动，提供远程系统的故障反应、故障隔离和故障弱化的方法，机器人170的组件协同交互以执行在机器人170中的故障反应、故障隔离和故障弱化的各种方面，第一机械手182、第二机械手183、第三机械手184和第四机械手185均包括多个节点，每个节点控制多个马达，该马达驱动在机械手臂中的接头和连杆机构以影响机械手臂的运动自由度，每个节点也控制用于停止马达旋转的多个制动器，第一机械手182具有马达307、309、311和313；多个制动器308、310、312和314以及多个节点315、316和317，每个节点315，316控制单个马达/制动器对；节点317控制两个马达/制动器对，传感器处理单元318被包括以提供马达位移传感器信息到节点317用于控制目的，第二机械手183、第三机械手184、第四机械手185与第一机械手182类似被配置为具有马达、制动器和节点，每个机械手臂操作地耦连到手臂处理器，手臂处理器328操作地耦连到第一机械手182的节点，手臂处理器325操作地耦连到第二机械手183的节点，手臂处理器323操作地耦连到第三机械手184的节点，以及手臂处理器321操作地耦连到第四机械手185的节点，每个手臂处理器还包括接头位置控制器，用于将其操作地耦连的机械手臂的期望接头位置转变为用于驱动其操作地耦连的机械手臂中的马达的电流命令以将其相应的接头驱动到期望的接头位置，系统管理处理器320操作地耦连到手臂处理器328，325，323， 321；系统管理处理器320还将与机械手臂相关联的用户操纵输入装置平移到期望的接头位置尽管示为分离单元，手臂处理器328，325，323，321也通过程序代码被实施为系统管理处理器320的一部分，手臂管理处理器319操作地耦连到系统管理处理器320和手臂处理器328、325、323、321，该手臂管理处理器319开始、控制和监控手臂的某些协调活动以便于使系统管理处理器320免于必须这样做，手臂管理器319也通过程序代码被实施为系统管理处理器320的一部分，处理器和节点中的每个被配置成通过硬件、固件和软件编程的任一组合来执行本文的各种任务，它们的功能通过一个单元来被执行或者在许多子单元之中被分配，每个子单元又通过硬件、固件和软件编程的任一组合来实施，该系统管理处理器320被分配为遍及机器人170的子单元，比如远程控制台169、以及机器人170的基座173，系统管理处理器320、手臂管理处理器319以及每个手臂处理器328，325，323，321包括多个处理器以执行各种处理器和控制器任务和功能，每个节点和传感器处理单元包括发射器/接收器(TX/RX)对，以便于与其机械手臂的其它节点和操作地耦连到其机械手臂的手臂处理器进行通信，TX/RX以菊花链连入网络，在这种菊花链布置中，当每个节点的RX接收来自邻近节点的TX的信息包时，它核实在该包中的目的地字段以确定是否该包用于其节点，该包用于其节点，则该节点处理该包，该包用于另一节点，则节点的TX在与它所来之处的相反方向上传递所接收包到邻近节点的RX，通过使用包交换协议(packet-switching protocol)，信息以包的形式在菊花链网络上传递，故障反应逻辑(FRL)线路被提供在每个机械手臂中，故障通知通过手快速传送，第一机械手182包括与手臂处理器328和机械手臂315的节点315、316和317中的每个耦连的FRL线路，当手臂处理器328和节点315，316和317中的一个检测到影响它的故障时，该手臂处理器或节点拉高FRL线路329以快速传送故障通知到与该线路329耦连的其它组件，相反地，当该手臂处理器328将要传输恢复通知到第一机械手182的节点时，它拉低FRL线路329以快速传送该恢复通知到与该线路329耦连的其它组件，虚拟FRL线路329通过指定包中的一个或更多个字段包括这样的故障通知和恢复通知而被代替地使用，在327中，该方法检测在多个机械手臂的失效手臂中的故障，其中由于所检测的故障该机械手臂变成“失效手臂”，在328中，该方法然后将该失效手臂置入安全状态，其中“安全状态”指的是通过阻止手臂的进一步运动来隔离所检测故障的失效手臂的状态，在329中，该方法确定该故障应被看作系统故障还是局部故障，其中“系统故障”指的是影响多个机械手臂中的至少一个其他的机械手臂的性能的故障，并且“局部故障”指的是影响仅失效手臂的性能的故障，因为局部故障导致仅失效手臂被保持在安全状态下直到该故障被清除，所以它不是导致非失效机械手臂的不安全操作的故障类型，该故障是导致非失效手臂的不安全操作的类型，则该方法应产生所检测故障是系统故障的确定，其中在系统中的所有机械手臂将被置入安全状态，在330中，该方法在仅当该故障将被看作系统故障时将多个手臂中的非失效手臂置入安全状态，其中“非失效手臂”指的是在其中没有故障已经被检测到的多个机械手臂中的机械手臂，在331中，该方法确定所检测故障被分类为可恢复系统故障还是不可恢复系统故障，在332中，该故障被分类为可恢复系统故障，则该方法提供给系统用户恢复选项，在333中，该故障被分类为不可恢复系统故障，则该方法等待系统关闭，在329中的确定是该故障将被看作局部故障，然后在334中，该方法确定该故障被分类为可恢复局部故障还是不可恢复局部故障，在335中，该故障被分类为可恢复局部故障，则该方法提供给系统用户恢复选项和弱化操作选项，在336中，该故障被分类为不可恢复局部故障，该方法仅提供弱化操作选项，执行故障反应、故障隔离和故障弱化的方法的方面的流程图，该故障反应、故障隔离和故障弱化通过机器人170的多个机械手臂的每个节点315、316和317被执行，在337中，每个节点持续地监控在该节点中的信号和信息以使用常规故障检测方法来检测影响节点的故障，这种类型的所检测故障在本文被称为“局部故障”，因为它局限于节点，该节点也针对由其手臂处理器或其机械手臂内另一节点发出的故障通知监控FRL线路，这种类型的所检测故障在本文被称为“远程故障”，因为它不局限于该节点，所检测的故障是硬件、固件、软件、环境或相关的通信，其中故障已经被检测到的节点在本文被称为“失效节点”，它的机械手臂在本文被称为“失效手臂”，其中没有故障已经被检测到的节点在本文被称为“非失效节点”，其中没有故障已经被检测的机械手臂在本文被称为“非失效手臂”，在338中，在337中检测到故障，则节点将其自身置入安全状态，这通过停用该节点的一个或更多个受控马达来完成，这通过接合该节点的一个或多个受控制动器来完成，在339中，该节点确定所检测的故障是局部故障还是远程故障，如之前参照337，故障的来源确定它将被看作局部故障还是远程故障，该故障被确定为局部故障，则该节点是失效节点，在第一种情况下，该失效节点通过执行如下的343-346和341-342而持续，该故障被确定为远程故障，则该节点是非失效节点，在第二种情况下，非失效节点通过执行如下的340-342而持续，在343中，该失效节点在失效机械手臂中在上游方向和下游方向上传送故障通知到邻近节点，该“下游”方向指的是信息包远离该节点的手臂处理器行进的方向并且“上游”方向指的是信息包朝向该节点的手臂处理器行进的方向，节点完成该过程的一个方式是通过将FRL线路拉到高状态，在344中，该失效节点然后诊断故障并且发送错误消息到系统管理处理器320，该错误消息优选包括故障信息，它的错误代码、错误类和起源(origin)，可能发生的会影响到节点的每种错误类型都被分配错误代码，该错误代码被分类为错误类，有至少四种错误类：可恢复手臂故障、不可恢复手臂故障、可恢复系统故障以及不可恢复系统故障，使用“可恢复”意味着用户被提供有尝试从故障中恢复的选项，使用“不可恢复”意味着用户不被提供有尝试从故障中恢复的选项，故障的起源包括节点的身份的信息以及节点内故障的来源的可选附加信息，在345中，失效节点确定所检测故障是否为可恢复局部故障，在345中的确定为否，则在346中，该失效节点保持在它的安全状态下并且忽视它可能在FRL线路上随后接收的任何恢复通知，在345中的确定为是则该失效节点行进到341，在339中的确定是所检测故障将被看作远程故障，则在340中，虚拟FRL线路被使用，则该非失效节点在与从故障通知所来之处的相反方向上传送所接 收故障通知，在真实FRL线路的情况下，非失效节点不需要对故障通知的这种传送采取任何行动，在341中，失效节点和非失效节点两者等待将被接收的恢复通知，在342中，一旦恢复通知被接收，则节点将它自身从安全状态返回到它的正常操作状态，这通过颠倒在338中所采取的行动同时避免突然变化来完成，该节点返回以执行参照337的故障检测任务，执行故障反应、故障隔离和故障弱化的方法的各方面的流程图，该故障反应、故障隔离和故障弱化通过每个手臂处理器321、323、325和328被执行，该每个手臂处理器操作地耦连到机器人170的机械手臂，在347中，每个手臂处理器在执行它的正常操作任务时，也持续监控它自身操作并且注意在其操作地耦连的机械手臂中的失效节点所传送的故障通知，在监控它自身操作时故障被检测到，则该故障在本文被称为“局部故障”，故障通过接收到来自在其操作地耦连的机械手臂中的失效节点的故障通知而被检测，则该故障被称为“远程故障”，远程故障是通过手臂处理器操作地耦连到的机械手臂中的失效节点沿着FRL线路传送的故障通知，在347中故障已经被检测，在车辆视觉系统502中，手臂处理器通过将接头位置控制器的输出马达电流命令锁定到零来将它的接头位置控制器置入安全状态，这用于加强它们相应节点的安全状态，在349中，手臂处理器确定所检测故障是局部故障还是远程故障，在347中，故障的来源确定了故障将被看作局部故障还是远程故障，该故障被确定为局部故障，则手臂处理器被看作失效节点，手臂处理器通过执行如下353-356和351-352而持续，故障被确定为远程故障，则手臂处理器被看作非失效节点，手臂处理器通过执行350-352而持续，在353中，手臂处理器将故障通知向下游传送到其操作地连接的机械手臂的所有节点，该手臂处理器完成该过程的一个方式是通过将FRL线路拉到高状态，在354中，手臂处理器诊断故障并发送错误消息到系统管理处理器320，该错误消息包括故障信息，错误代码、错误类以及起源，发生的会影响到手臂处理器的每种错误类型都被分配错误代码，该错误代码被分类为错误类，有至少四种错误类：可恢复处理器故障、不可恢复处理器故障、可恢复系统故障以及不可恢复系统故障，故障的起源包括手臂处理器的身份的信息以及手臂处理器中故障的来源的可选附加信息，在355中，手臂处理器确定所检测故障是否为可恢复局部故障，通过故障的错误类来完成该确定，在355中的确定为否，则在356中，该失效手臂处理器的接头位置控制器保持在其安全状态并且手臂处理器忽视其可能在FRL线路上随后接收的任何恢复通知，在355中的确定为是，则该手臂处理器行进到350，在349中的确定是所检测故障将被看作远程故障，则在350中，手臂处理器等待从系统管理处理器320接收的恢复通知，在351中，一旦恢复通知被接收，则手臂处理器通过例如将它的FRL线路拉低来传送该恢复通知到在其操作地耦连的机械手臂中的所有节点，在352中，手臂处理器然后将它的接头位置控制器从安全状态返回到它的正常操作状态，该过程通过释放接头位置控制器的输出马达电流命令来完成，以便它们可再一次反映其操作地耦连的机械手臂的期望接头位置同时避免突然的变化，该手臂处理器然后返回以执行参照347的它的故障检测任务，执行故障反应、故障隔离和故障弱化的方法的各方面的流程图，该故障反应、故障隔离和故障弱化通过机器人170的系统管理处理器320被执行，在357中，系统管理处理器在执行它的正常操作任务时也等待以接收从机器人170的另一组件所传输的错误消息，错误消息在357被接收，则在358中，系统管理处理器为了安全目的通过例如命令机器人系统中的所有手臂处理器328、325、323和321的接头位置控制器以它们的电流值锁定它们各自的输出来停止该系统，没有新的电流命令输入被提供到机械手臂，直到接头位置控制器的输出被解锁，接头位置控制器的输出的这种锁定在本文被称为“软锁”接头位置控制器，该方法然后行进到359，在359中，该系统管理处理器确定所检测故障应被看作系统故障还是手臂故障，该系统管理处理器通过检查在错误消息中所提供的错误类信息来完成该步骤，系统故障包括被分类为或者可恢复系统故障或者不可恢复系统故障的所有故障，因为这些故障可应用于不只是失效机械手臂，相反地，手臂故障包括被分类为或者可恢复局部故障或者不可恢复局部故障的所有故障，因为这些故障仅可应用于失效机械手臂，针对将被看作手臂故障的所有故障，执行360-366，在360中，该系统管理处理器给机器人170的远程操作员171提供接受机器人170的弱化操作的选项，该局部故障是可恢复局部故障，则该系统管理处理器还给用户提供从故障中恢复的选项，除了每个所提供的选项之外，所检测故障的信息也通过系统管理处理器提供，以辅助远程操作员171确定是否接受该选项，该选项和故障信息被提供在远程控制台169的视觉显示器255上，在361中，该系统管理处理器等待远程操作员171选取在360中所提供的选项，一旦选项被远程操作员171选取，则在362中，该系统管理处理器确定所选取的选项是弱化操作选项还是恢复选项，恢复选项被提供并且该远程操作员171选取恢复选项，则在381中，该系统管理处理器发送恢复通知到失效机械手臂的手臂处理器，该失效机械手臂的手臂处理器将处理该恢复通知，包括传送恢复通知到失效手臂的所有节点，失效手臂的节点然后处理该恢复通知，在382中，系统管理处理器然后通过解锁所有手臂处理器的接头控制器的输出而释放接头控制器的软锁，以便接头控制器再次发出反映它们的操作地耦连的机械手臂的期望接头位置的马达电流命令然后，该系统管理处理器返回执行参照357的它的任务，远程操作员171选取弱化操作选项，则在363中，该系统管理处理器给远程操作员171提供从故障中恢复的选项，在这种情况下从故障中恢复不同于参照381-382的恢复，因为没有尝试恢复失效手臂，恢复仅应用于恢复非失效手臂的正常操作，在364中，该系统管理处理器等待用户选取在363中所提供的选项，一旦选项被远程操作员171选取，则在365中，该系统管理处理器发送消息到失效手臂的手臂处理器以加强该故障，故障的加强在这种情况下意味着附加步骤被采取以完全关闭失效机械手臂的操作，这种加强措施的一个示例是从主/从控制系统的其它部件操作地断开手臂处理器的接头位置控制器，该主/从控制系统产生其操作地耦连的机械手臂的期望接头位置另一加强措施是关闭到失效机械手臂的电源，在366中，该系统管理处理器随然后通过解锁所有非失效手臂的手臂处理器的接头控制器的输出来释放接头控制器的软锁，以便接头控制器再次发出反映其操作地耦连的机械手臂的期望接头位置的马达电流命令该系统管理处理器然后返回 执行参照357的任务，在367中，该系统管理处理器使系统FRL情况对在机器人170中的所有节点有效，通过引起FRL线路329、327、384和385被拉高从而使故障通知同时被提供给手臂处理器和第一机械手182、第二机械手183、第三机械手184和第四机械手185的节点来完成该步骤，在369中，该系统管理处理器然后确定该系统故障是否为可恢复系统故障，通过检查在所接收错误消息中的错误类来完成该步骤，在369中的确定为否，则在363中，该系统管理处理器没有采取进一步行动并且等待该系统被关闭，在369中的确定为是，则在370中，该系统管理处理器提供给用户从故障中恢复的选项，在371中，该管理处理器等待远程操作员171选取恢复选项，该选项被选取则在372中，该系统管理处理器发送恢复通知到机器人170的所有机械手臂的手臂处理器，在373中，该系统管理处理器在接收来自171的请求或行动时释放每个接头控制器的软锁以在其正常操作状态下操作接头控制器的手臂，以便所释放的接头控制器再一次发出反映其操作地耦连的机械手臂的期望接头位置的马达电流命令该系统管理处理器随后返回执行参照357的任务，执行故障反应、故障隔离和故障弱化的方法的各方面的流程图，该故障反应、故障隔离和故障弱化通过操作地耦连到机器人170的系统管理处理器和手臂处理器328、325、323和321的手臂管理处理器被执行，手臂管理处理器319开始、控制和监控机器人170的第一机械手臂第一机械手182、第二机械手183、第三机械手184、第四机械手185的某些协调活动，该手臂管理器319开始和监控启动制动器测试，其中该手臂管理器319与手臂处理器328、325和323中的每个通信，以便具有不同扭矩值的具体制动顺序被应用到它们的各自机械手臂的制动器，该活动的协调在这种情况下通过手臂管理器319进行，因为将其编码到每个手臂处理器的开销是多余的，在每个顺序的结束时，由每个手臂处理器所计算的最大扭矩值被传递回手臂管理器319，超出范围的错误发生时，则手臂管理器319将传递故障通知到失效手臂，手臂管理器命令该手臂处理器执行手臂活动，监控结果，并且决定该活动结果是否表示手臂失效，在374中，手臂管理器根据机械手臂各自手臂处理器的报告监控机械手臂的协调活动以检测在一个手臂中的故障，当所报告的测量超过预期值达阈值量时，该手臂管理器确定故障已经发生，在这种情况下所检测的故障是机械手臂的节点中的一个节点或手臂处理器一般不会检测的故障，在故障已经在374中被检测到之后，然后在375中，手臂管理器抑制到失效手臂的任何进一步命令，没有进一步命令将从手臂管理器被传输到失效手臂的手臂处理器，直到或者从系统管理器接收恢复通知或者重新启动该系统，在376中，该手臂管理器通过将失效手臂的FRL线路拉到高状态来传送故障通知到失效手臂，在虚拟FRL线路的情况下，手臂管理器通过失效手臂的节点中的一个节点或手臂处理器传输在与被指定用于传输故障通知的包字段相同或不同的包字段中的故障通知，在377中，该手臂管理器发送错误消息到系统管理器，该错误消息具有故障的可用细节，通过手臂管理器检测的每种故障类型被分配错误代码，并且该错误代码被分类为错误类，故障的起源包括失效手臂的身份信息以及故障的来源的可选附加信息，在357中该系统管理器然后开始处理错误消息，在378中该手臂管理器确定所检测故障是否为可恢复故障，根据故障的错误类来进行该确定，在378中的确定为否，则在381中，手臂管理器持续抑制到失效手臂的任何进一步命令并且忽视随后从系统管理器接收的任何恢复通知，在378中的确定为是，则在379中，该手臂管理器等待从管理处理器所接收的恢复通知，在380中，恢复通知被接收，该手臂管理器停止抑制到失效手臂的命令并且返回它的正常操作模式并且执行参照374的故障检测任务，第二和第三实施例双模驾驶模式的系统连接图，采用模块化设计思路无入驾驶控制器525，无人驾驶控制器525与车辆视觉系统502连接、无人驾驶控制器525与定位导航模块274、规划系统模块528、无人驾驶控制器525与车辆总线276连接，控制系统模块529与机器人170连接，各模块之间采用以太网和CAN总线通讯，无人驾驶控制器525采用可重构计算AI芯片，基于Linux系统平台进行程序编写，综合车辆整车参数和运营特点参数定制控制策略，具体控制策略包括：1、无人驾驶控制器525作为控制模式的仲裁控制器，决定当前是否处于无人驾驶模式，是否满足无人驾驶模式的条件；2、当处于远程驾驶模式下，控制系统模块屏蔽无人驾驶控制器发出的控制命令，响应远程操作命令；3、当处于无人驾驶控制模式下，根据无人驾驶控制器525收到的环境感知模块接受到的激光雷达、毫米波雷达和摄像头等的信息，经各传感器融合的优化信息，以及接收到的陀螺仪感知的车身姿态信息和DGPS定位导航模块提供的经纬度信息，结合经深度学习获知的循迹路线，通过并行建图与定位技术(SLAM)自适应路线规划行驶，第二实施例双模驾驶切换逻辑图，采用远程控制驾驶模式能够切换成智能无人驾驶模式，应急切换模式：“远程控制驾驶模式”信号中断自动发送请求517，此时无人驾驶控制器判断进入无人驾驶模式自动接管远程驾驶模式，执行518，不允许进入无人驾驶模式，反馈信号519，当需从“无人驾驶模式”进入“远程控制驾驶模式”时，通过复位外置无人驾驶控制开关或预先规划好的退出无人驾驶逻辑切换到正常驾驶模式，执行控制指令520，第三实施例双模驾驶切换逻辑图，采用远程控制驾驶模式能够切换成智能无人驾驶模式，正常切换模式：当需从“远程控制驾驶模式”进入“智能无人驾驶模式”时，按下外置无人驾驶控制开关向无人驾驶控制器发送请求521，此时无人驾驶控制器判断是否进入无人驾驶模式，允许进入无人驾驶模式，执行522，不允许进入无人驾驶模式，反馈信号523；当需从“无人驾驶模式”进入“远程控制驾驶模式”时，通过复位外置无人驾驶控制开关或预先规划好的退出无人驾驶逻辑切换到正常驾驶模式，执行控制指令524。     The data decompression system 448 includes a receiving module 263, a compressed data scanning unit 449, a compression logic judgment value and compression logic acquisition unit 450, a decompression reading unit 451, a second judgment unit 452, and an original byte data recovery unit 453. The compressed data scanning unit 449 is used to scan compressed data; the compression logic judgment value and compression logic acquisition unit 450 is used to perform a compression logic judgment operation on the n-th byte data of the compressed data to obtain the compression logic judgment value and the compression logic, Where n is a natural number greater than or equal to 1; the decompression reading unit 451 is used to decompress and read the compressed data according to the compression logic judgment value and the compression logic; the second judgment unit 452 is used to judge whether the compressed data is scanned Complete; the original byte data recovery unit 453 is used to recover the original byte data according to the read byte data, the display 255 displays the original byte data recovered by the original byte data recovery unit 453 through the various subsystems established by the above connection , The remote operator 171 remotely drives the vehicle 260 to walk with the following system: the distal end of the first link 139 is connected to the proximal end of the second link 137 at the joint providing the horizontal pivot axis 138, and the third link 124 The proximal end is connected to the distal end of the second link 137 at the rolling joint, so that the third link generally rotates at the joint 123 around an axis extending along the axis of both the second link and the third link. Rolling is performed distally after the pivot joint 125. The distal end of the fourth link 136 is connected to the instrument holder 136 through a pair of pivot joints 135, 134. The pivot joints 135, 134 together define the instrument holder 121, The translation or prismatic joint 132 of the robot 170 manipulator arm assembly 133 facilitates the axial movement of the instrument 126, enabling the instrument holder 131 to be attached to the cannula, through which the instrument 126 is slidably inserted, in the instrument holder On the distal side of 131, the second instrument 126 includes additional degrees of freedom. The actuation of the second instrument 126 is driven by the motor of the robot manipulator arm assembly 133. The interface can be arranged more proximally or distally along the kinematic chain of the manipulator arm assembly 133. The second instrument 126 includes the proximal rotary joint 130 of the pivot point PP, which is arranged at the required position. The distal side of 126 allows the end effector 128 to pivot around the instrument wrist axis 129, 127, and can control the angle θ between the end effector jaws 231 independently of the position and orientation of the end effector 128. Left-hand input The device 177 and the right main input device 178 are connected to and separated from the console 169 through wireless communication, the left hand-held input device 177 is connected to the second processor 215, the right hand-held input device 178 is connected to the second processor 215, and the remote operator 171 is in The remote console 169 activates the second processor 215 to start remote driving, and the left hand of the remote operator 171 controls the left hand input device At 177, the left hand-held input device 177 controls the movement of the arm end 197 through the second processor 215, the right hand of the remote operator 171 controls the right hand-held input device 178, and the right hand-held input device 178 controls the arm end 197 through the second processor 215 Move, the arm end 197 uses the first contact end 194 and the second contact end 196 in the end effector 193 to contact and hold the steering wheel 235. The left hand-held input device 177 and the right hand-held input device 178 can move in opposite directions. The steering wheel 235 rotates, the remote operator 171 uses the second processor 215 software of the remote console 169 to control the first manipulator 182 and the second manipulator 183 of the robot 170, and the remote operator 171 determines through measurement, model estimation, measurement and modeling The force applied to the steering wheel 235 on the first manipulator 182 and the second manipulator 183 of the robot 170. The first manipulator 182 and the second manipulator 183 provide tactile feedback to the remote operator 171 through the remote console 169. This tactile feedback can Simulate the manual manipulation arm end 197 for the remote operator 171 to control the steering wheel 235, and the remote operator 171 can simulate the reaction force experienced by the first manipulator 182 of the robot 170 corresponding to the steering wheel 235, the first contact end 194 in the end effector 193 And the second contact end 196, which pivot relative to each other so as to define a pair of end effector jaws 231. For instruments having end effector jaws 231, they are caused by squeezing the grip members of the input devices 177, 178 Moving the jaws 231, the robot 170 manipulates the first manipulator 182 and the second manipulator 183 to move the transmission assembly 195 on the upper part of the steering wheel 235, so that the shaft 187 extends and retracts to provide the desired movement of the end effector 193, and the robot 170 manipulates the first manipulator 182 and the second manipulator 183, the third manipulator 184 and the fourth manipulator 185 can move at the steering wheel 235, the gear position 400, the brake 401 pedal and the accelerator pedal 402 during remote driving, the first manipulator 182 and the second manipulator 183 Hold the steering wheel 235 to change the direction of the car. The first manipulator 182 is connected with an instrument holder 180, which is connected with the instrument 186 and the arm end 197, and the instrument holder 180 and the first manipulator 182 are connected by a motorized joint. The instrument holder 180 includes an instrument holder frame 188, a clamp 189, and an instrument holder bracket 190. The clamp 189 is fixed to the distal end of the instrument holder frame 188. The clamp 189 can be connected to and separated from the arm end 197. The instrument holder The bracket 190 is connected to the instrument holder frame 188. The linear translation of the instrument holder bracket 190 along the instrument holder frame 188 is a motorized translation movement controlled by the second processor 215. The instrument 186 includes a transmission assembly 195 and an elongated Shaft 187 and end effector 193, transmission assembly 195 and instrument holder The bracket 190 is connected, the shaft 187 extends distally from the transmission assembly 195, and the end effector 193 is provided at the distal end of the shaft 187. The shaft 187 defines a longitudinal axis 192, which coincides with the longitudinal axis of the arm end 197 and is aligned with the longitudinal axis of the arm end 197. The longitudinal axis defined by the arm end 197 coincides. When the instrument holder bracket 190 is translated along the instrument holder frame 188, the elongated shaft 187 of the instrument 186 moves along the longitudinal axis 192, and the end effector 193 can be extended from the working space. Retract, the remote operator 171 sends instructions through the second processor 215, the three-state switch 202 receives the activation signal, and the remote operator 171 uses the second processor 215 and the vehicle remote driving system 258 to connect the robot 170 to operate the arm of the first manipulator 182 The end 197 is to grip and move away from the steering wheel 235. The remote operator 171 uses the second processor 215 and the vehicle remote driving system 258 to connect the robot 170 to manipulate the second manipulator 183. The arm end 197 is to grip and move away from the steering wheel 235. The first contact end 194 and the second contact end 196 in the actuator 193 apply force to the steering wheel 235 to rotate the steering wheel 235, and release the three-state switch 202 to stop the movement of the arm end 197. When the arm end 197 is connected to the steering wheel 235, The remote operator 171 sends a signal to activate the second direction. The first direction is opposite to the second direction. The three-state switch 202 receives the signal to activate the second direction, and the arm end 197 moves toward the steering wheel 235. 151 connects the main input device 152 to the slave The simplified controller schematic diagram of the master/slave controller 153 of the manipulator 154 uses vector mathematical notation to describe the controller input, output and calculation, where the vector X will refer to the azimuth vector in Cartesian coordinates, and where the vector q will refer to the correlation The joint articulation configuration vector of the linked linkage device is sometimes called the linkage device orientation in the joint space. When there is ambiguity, subscripts can be appended to these vectors to identify specific structures, so that
    Figure PCTCN2020000015-appb-100004
    It is the position of the main input device in the associated main working space or coordinate system, and x s represents the position of the follower in the working space. The speed vector associated with the position vector consists of the point or vector above the vector and the bottom The word "dot" between the symbols means, for example, the xdot m of the main velocity vector. The velocity vector is mathematically defined as the change of the azimuth vector with time. The controller 153 contains the inverse Jacobian speed controller.
    Figure PCTCN2020000015-appb-100005
    When it is the position of the master input device and the speed of the master input device, the controller 153 calculates the power command for transmission to the slave manipulator 154 to realize the slave end effector movement corresponding to the input device from the master speed, and control The device 153 is capable of calculating the force reflection signal applied to the main input device and from there to the hand of the remote operator 171 from the bearing x s and from the speed. The first module 159 contains the inverse Jacobian speed controller, which has the function of The output of the calculation performed by the inverse Jacobian matrix modified from path 163. First, the virtual slave path is described. The vector associated with the virtual follower is usually represented by the v subscript, so that the virtual slave velocity in the joint space qdot v is integrated with Provide q v , use the inverse motion module 162 to process q v to generate the virtual slave joint orientation signal x v , the virtual slave orientation and the master input command x m are combined and use the forward motion 161 for processing, the use of virtual slaves is helpful For smooth control and force reflection when approaching the hard limit of the system, when exceeding the soft limit of the system, etc., other components and other controllers instructed by the first control module 159 and the second control module 160 and control the schematic diagram 165 The structure includes data processing hardware, software, and firmware. Such a structure includes reprogrammable software and data, which is embodied in machine-readable code and stored in a tangible medium for the second of the remote console 169 The processor 215 uses machine-readable codes to store in a variety of different configurations, including random access memory, non-volatile memory, write-once memory, magnetic recording media, and optical recording media, embodying the code and data associated with it. Signals are transmitted through various communication links, including the Internet, Intranet, Ethernet, wireless communication networks and links, electrical signals and conductors, and optical fibers and networks. The second processor 215 includes a remote console 169 And multiple data processors, including one or more local data processing circuits of manipulators, instruments, separate and remote processing structures and locations, modules include a single common processor board, multiple separate boards, and one of the modules And multiple are scattered on multiple boards, some of which also run some and all calculations of another module, the software code of the module is written as a single integrated software code, and each module is divided into separate subroutines, or parts of a module The code is combined with some or all of the code of another module, the data and processing structure includes any of various centralized or distributed data processing and programming architectures, the output of the controller, which will often try to solve a specific The manipulator joint configuration vector q is used to generate commands for these highly configurable slave manipulator mechanisms. Manipulator linkages usually have enough degrees of freedom to occupy a series of joint states for a given end effector state, where The actuation of a joint is directly replaced by similar actuations of different joints along the kinematic chain. These structures are sometimes referred to as having excess, extra or redundant degrees of freedom. At the same time, these terms usually cover the kinematic chain, where the middle link Pole energy The main joint controller of the first module often tries to determine or solve the virtual joint speed vector qdot v when using the speed controller of Fig. 40 to guide the movement of the highly configurable manipulator. Can be used to drive the joints of the slave manipulator 164 in such a way that the end effector will accurately follow the master command x m . For slave mechanisms with redundant degrees of freedom, the inverse Jacobian matrix usually does not completely define the joint vector solution. In a system that can occupy a series of joint states for a given end effector state, the mapping from the Cartesian command xdot to the joint motion qdot is a one-to-many mapping. Because the mechanism is redundant, there are countless solutions in mathematics. It is represented by the subspace of inverse survival. The controller uses a Jacobian matrix with more columns than rows to reflect this relationship, and maps multiple joint velocities to relatively few Cartesian velocities, through a remote motion center constrained by software The concept of 298 is determined. Through the ability to calculate software pivot points, different modes characterized by system compliance or stiffness can be selectively realized. After calculating the estimated pivot points, a certain range of pivots can be realized Different system modes on the point/center. In the fixed pivot implementation, the estimated pivot point can be compared with the expected pivot point to generate an error output that can be used to drive the instrument’s pivot To the desired position, conversely, in the passive pivot implementation, although the desired pivot position may not be the most important target, the estimated pivot point can be used for error detection and therefore for safety, because the estimated pivot point The change of the pivot point position indicates that the steering wheel is separated or the sensor is faulty, so that the system has the opportunity to take corrective measures. The processor 157 includes a first controller module 157 and a second controller module 160. The first module 157 can contain the main Joint controller, inverse Jacobian master-slave controller, the main joint controller of the first module 157 can be configured to generate the desired manipulator assembly movement in response to input from the main input device 156, the manipulator linkage device has a A series of alternative configurations for a given end effector orientation in space. Commands used to make the end effector assume a given orientation can cause various joint movements and configurations. The second module 160 can be configured to help The manipulator assembly is driven to the desired configuration, and the manipulator is driven toward the preferred configuration during the master-slave movement. The second module 160 will contain configuration related filters, the main joint controller of the first module 157 and the configuration of the second module 160 Related filters can include filters used by the processor 157 to transfer the control authority of the linear combination of joints to the realization of one or more goals or tasks. Assuming that X is the space of joint motion, F(X) can be The filter that controls the joints to i) provide the desired end effector movement, and ii) provide the pivoting movement of the instrument shaft at the hole. The main joint controller of the first module 157 may include the filter F(X ), conceptually, (1-F-1F)( X) The configuration-related subspace filter can be described, which gives the control actuation authority of the linear combination of joint speed orthogonal to the goal of the main joint controller. This configuration-related filter can be controlled by the second controller 157 The module 160 is used to achieve the second goal. The two filters can be further subdivided into more filters corresponding to more specific tasks. The filter F(X) can be divided into F1(X) and F2(X). For controlling the end effector and controlling the pivot axis movement, any one of them can be selected as the highest priority task of the processor, and the robot processor and control technology will often use the main joint controller configured for the first controller task , And a configuration related filter, which uses the underconstrained solution generated by the main joint controller for the second task. The main joint controller will be described with reference to the first module, and the configuration related filter will be described with reference to the second module It can also include additional functions and additional modules of various priority levels. The hardware and programming codes of the first and second module functions are fully integrated and partially integrated and can be completely separated. The controller 157 can use two modules at the same time The function can have a variety of different modes, in which one or two modules are used separately or in different ways. During the master-slave operation, the first module 157 can have little or no influence from the second module 160 When the end effector is not driven by the robot, the second module 160 has a greater role during system assembly. Both modules can be active during most or all of the time when the robot movement is enabled. The gain of the first module is set to zero. By setting x s to x s, actual and by reducing the matrix rank in the inverse Jacobian controller make it impossible to control too much and make the configuration related filters have more control rights. Reduce or eliminate the influence of the first module on the state of the manipulator assembly, thereby changing the mode of the processor 157 to the gripping mode. The first module 157 can contain some form of Jacobian controller with a Jacobian correlation matrix. In the port gripping mode, the second module 160 can receive a signal from the slave manipulator 158, which indicates the orientation or speed of the follower at least partially caused by the manual articulation of the slave manipulator linkage. In response to the input, the first The second module 160 can generate power commands suitable for driving the joints of the follower, so as to allow manual articulation of the slave linkage, while configuring the follower in the desired joint configuration. During the master-slave end effector manipulation, the controller The second module 160 can be used to help derive power commands based on different signals bqdot o . This alternative input signal to the second module 160 of the controller 157 can be used to drive the manipulator linkage to maintain or move along the manipulator structure Minimally invasive hole pivot position to avoid collisions between multiple manipulators, thereby increasing the range of motion of the manipulator structure and avoiding singularities, so as to generate the desired posture of the manipulator, etc., use the input from the MTM controller to actively control the remote Sports Center (R C) A block diagram 231 of the arm end 197 (C) and the instrument end effector (E) reference system, using the input from the main manipulator controller to actively control the instrument end effector (E) system, while using the secondary input device to control The block diagram 232 of the remote center (RC) and arm end 197(C) system, the secondary input device uses any reference, not necessarily the target system (EYE system), the reference system transformation EYETREF can be directly measured or calculated from indirect measurement values , The signal conditioning unit combines these inputs in an appropriate common system for use by the slave manipulator controller. There are three reference systems to be controlled by the system controller. One of the reference systems (C) is the reference of the arm end 197 System, assuming that EYETE is commanded by the master tool manipulator (MTM) controller, the posture specifications of the remote central reference frame and the arm end 197 reference frame come from one or a combination of the following sources: (i) The MTM controller specifies these systems/reference systems In the EYE system, namely EVE T RC and EYE T C , (ii) the secondary device commands these system positions in a convenient frame of reference, namely REF T RC and REF T C (where EYE T RFF can be determined), and ( iii) The slave side controller designates these postures in the base frame of the slave arm, namely W T RC and W T C , schematic block diagrams of systems 212 and 213, which are used for remote vehicle assistance using computer The second processor 215 of the driving system 258 controls the relationship between the reference frame of the instrument end effector 193 and the reference frame of the remote control center 298. Assuming that the reference frame of the arm end 197 and the reference frame of the remote control center 298 coincide, the reference frame of the arm end 197 And the remote control center 298 is physically constrained to move only along the arm end 197 and the longitudinal axis of the instrument relative to the instrument end effector 193. Two different strategies are adopted to control the reference frame of the instrument end effector 193 (E The relationship between the reference system (RC system) of the remote control center 298 and the reference system (RC system) of the remote control center 298, which is used to actively control the relative distance between the two reference systems regardless of whether the E system is fixed or moving (d). Force/torque sensor or three-state switch input, use the block diagram to realize the control subsystem for this mode, the control subsystem can be described as a'relative posture controller', using the three-state switch to relatively control the distance from the tip In general block diagram, the incremental Cartesian command "slv_cart_delta" of the slave manipulator is expressed as follows: slv_cart_delta=S*slv_cart_vel*Ts (where Ts is the sampling time of the controller), S takes the value [1, -1, 0], this The general block diagram of the distance between the force/torque or pressure sensor relative to the control arm end 197 and the tip depending on its commands to control the steering wheel 235, gear 400, brake pedal 401, and accelerator pedal 402. For the incremental Cartesian from the manipulator The command "slv_cart_delta" can It is expressed as follows: slv_cart_delta=f(F,p) "f" is a programmable function that uses the sensed force or pressure F and some user-defined parameters p as input. This is an admittance controller, which can be used by Based on the joint torque sensor and arm kinematics knowledge to estimate the Cartesian force along the axis of the arm end 197, after this estimation, the calculated estimated value can be used as the input F to command incremental movement. The signal F can be based on the user and Any other measured or calculated force amount of the interaction of the manipulator can independently control the trajectories of the RC system and the E system. The control input to manage these trajectories may come from the main manipulator. The control subsystem block diagram of the additional strategy can be Known as an'independent posture controller', it can outline the insertion (I/O) movement to allow lateral movement of the remote control center 298 or arm end 197 relative to the instrument tip E, the remote control center 298 or arm end 197 will need to surround the tip Pivoting, while driving the instrument to compensate for the movement of E, this will allow the movement of the RC and the arm end 197 in the cab, and provide a remote system fault response, fault isolation and fault weakening method. The components of the robot 170 interact in coordination to execute the Various aspects of fault response, fault isolation and fault weakening in the robot 170. The first manipulator 182, the second manipulator 183, the third manipulator 184, and the fourth manipulator 185 all include multiple nodes, and each node controls multiple motors. The motor drives the joint and linkage mechanism in the robot arm to affect the degree of freedom of the robot arm. Each node also controls multiple brakes for stopping the motor rotation. The first robot arm 182 has motors 307, 309, 311, and 313. Multiple brakes 308, 310, 312 and 314 and multiple nodes 315, 316 and 317, each node 315, 316 controls a single motor/brake pair; node 317 controls two motor/brake pairs, the sensor processing unit 318 is included In order to provide motor displacement sensor information to the node 317 for control purposes, the second manipulator 183, the third manipulator 184, and the fourth manipulator 185 are similarly configured with the first manipulator 182 to have motors, brakes, and nodes. The arm processor 328 is operatively coupled to the node of the first manipulator 182, the arm processor 325 is operatively coupled to the node of the second manipulator 183, and the arm processor 323 is operatively coupled to the third The node of the manipulator 184 and the arm processor 321 are operatively coupled to the node of the fourth manipulator 185. Each arm processor also includes a joint position controller for converting the desired joint position of the manipulator operatively coupled to it. To drive the current command of the motor in the operatively coupled robot arm to drive its corresponding joint to the desired joint position, the system management processor 320 is operatively coupled to the arm processors 328, 325, 323, 321; The system management processor 320 will also manipulate the user associated with the robotic arm The input device is translated to the desired joint position. Although shown as a separate unit, the arm processors 328, 325, 323, 321 are also implemented by program code as part of the system management processor 320, and the arm management processor 319 is operatively coupled to the system The management processor 320 and the arm processors 328, 325, 323, 321. The arm management processor 319 initiates, controls, and monitors certain coordinated activities of the arm in order to prevent the system management processor 320 from having to do so. The arm manager 319 is also implemented as a part of the system management processor 320 through program code. Each of the processor and node is configured to perform various tasks in this document through any combination of hardware, firmware, and software programming. One unit is executed or distributed among many sub-units, and each sub-unit is implemented by any combination of hardware, firmware and software programming. The system management processor 320 is distributed as sub-units throughout the robot 170, such as The remote console 169, and the base 173 of the robot 170, the system management processor 320, the arm management processor 319, and each arm processor 328, 325, 323, 321 include multiple processors to perform various processors and controls Each node and sensor processing unit includes a transmitter/receiver (TX/RX) pair to facilitate communication with other nodes of its robot arm and the arm processor operatively coupled to its robot arm. TX /RX is connected to the network in a daisy chain. In this daisy chain arrangement, when the RX of each node receives an information packet from the TX of a neighboring node, it verifies the destination field in the packet to determine whether the packet is used At its node, if the packet is used for its node, the node processes the packet, and the packet is used for another node, then the node's TX transfers the received packet to the RX of the neighboring node in the opposite direction to where it came from By using the packet-switching protocol, information is transmitted in the form of packets on the daisy chain network, fault response logic (FRL) lines are provided in each robot arm, and fault notifications are quickly transmitted by hand. The manipulator 182 includes FRL lines coupled to each of the nodes 315, 316, and 317 of the arm processor 328 and the robot arm 315, when the arm processor 328 and one of the nodes 315, 316, and 317 detects a fault affecting it When the arm processor or node pulls up the FRL line 329 to quickly transmit a fault notification to other components coupled to the line 329, on the contrary, when the arm processor 328 will transmit a recovery notification to the node of the first manipulator 182 , It pulls down the FRL line 329 to quickly transmit the recovery notification to other components coupled to the line 329. The virtual FRL line 329 is replaced by specifying one or more fields in the packet to include such failure notification and recovery notification. In 327, this method detects faults in failed arms of multiple robotic arms, Among them, the robot arm becomes a "failed arm" due to the detected fault. In 328, the method then puts the failed arm into a safe state, where the "safe state" refers to isolating the detected arm by preventing further movement of the arm The state of the failed arm of the fault. In 329, the method determines whether the fault should be regarded as a system fault or a partial fault, where “system fault” refers to the performance that affects at least one of the other robot arms. Failure, and "partial failure" refers to a failure that affects the performance of a fail-only arm. Because a partial failure causes only the failing arm to be kept in a safe state until the failure is cleared, it does not cause unsafe operation of a non-failed robotic arm If the fault is the type that causes the unsafe operation of the non-failure arm, the method should produce a determination that the detected fault is a system fault, in which all the robot arms in the system will be placed in a safe state, in 330 , This method puts the non-failed arm of the multiple arms into a safe state only when the fault will be regarded as a system failure, where the "non-failed arm" refers to multiple machines in which no failure has been detected The robotic arm in the arm. In 331, the method determines whether the detected fault is classified as a recoverable system fault or an unrecoverable system fault. In 332, the fault is classified as a recoverable system fault, and the method is provided to the system user Recovery option, in 333, the fault is classified as an unrecoverable system fault, then the method waits for the system to shut down, the determination in 329 is that the fault will be regarded as a partial fault, and then in 334, the method determines that the fault is It is classified as a recoverable partial fault or an unrecoverable partial fault. In 335, the fault is classified as a recoverable partial fault. The method provides the system user with recovery options and weakened operation options. In 336, the fault is classified as unrecoverable. To recover from a local fault, the method only provides options for weakening operations, and performs a flowchart of the aspects of the method of fault reaction, fault isolation and fault weakening, which pass through each node of the multiple robotic arms of the robot 170 315, 316, and 317 are executed. In 337, each node continuously monitors the signals and information in the node to use conventional fault detection methods to detect faults affecting the node. This type of detected fault is referred to herein as It is a "local fault" because it is limited to the node, which also monitors the FRL line for the fault notification issued by its arm processor or another node in its robotic arm. This type of detected fault is referred to herein as ""Remotefailure" because it is not limited to the node. The detected failure is hardware, firmware, software, environment, or related communications. The node whose failure has been detected is referred to as a "failed node" in this article. The arm is called "failed arm" in this article, the node where no fault has been detected is called "non-failed node" in this article, and the mechanical arm where no fault has been detected is called "non-failed arm" in this article , In 338, in 337 If a fault is detected, the node puts itself into a safe state. This is done by deactivating one or more controlled motors of the node. This is done by engaging one or more controlled brakes of the node, at 339 , The node determines whether the detected fault is a local fault or a remote fault. For example, referring to 337, the source of the fault determines whether it will be regarded as a local fault or a remote fault. If the fault is determined to be a local fault, the node is a failed node In the first case, the failed node continues by executing the following 343-346 and 341-342. If the failure is determined to be a remote failure, then the node is a non-failed node. In the second case, the non-failed node Continue by executing the following steps 340-342. In 343, the failed node transmits fault notifications to neighboring nodes in the failed robot arm in the upstream and downstream directions. The "downstream" direction refers to the information packets away from the node. The direction in which the arm processor travels and the "upstream" direction refers to the direction in which the packet travels towards the arm processor of the node. One way for the node to complete this process is by pulling the FRL line to the high state. In 344, the failure The node then diagnoses the fault and sends an error message to the system management processor 320. The error message preferably includes the fault information, its error code, error type, and origin. Each type of error that may occur that affects the node is Assign an error code. The error code is classified as an error type. There are at least four error types: recoverable arm failure, unrecoverable arm failure, recoverable system failure, and unrecoverable system failure. Using "recoverable" means that the user is provided There is an option to try to recover from the failure. Using "unrecoverable" means that the user is not provided with the option to try to recover from the failure. The origin of the failure includes information about the identity of the node and optional additional information about the source of the failure within the node. In 345, the failed node determines whether the detected fault is a recoverable partial fault, and the determination in 345 is no, then in 346, the failed node remains in its safe state and ignores that it may subsequently receive on the FRL line If the determination in 345 is yes, the failed node will proceed to 341, and the determination in 339 is that the detected failure will be regarded as a remote failure. Then in 340, the virtual FRL line is used, then the non- The failed node transmits the received failure notification in the opposite direction from where the failure notification came from. In the case of a real FRL line, the non-failed node does not need to take any action on this transmission of the failure notification. In 341, failure Both the node and the non-failed node wait for the recovery notification to be received. In 342, once the recovery notification is received, the node returns itself from the safe state to its normal operating state, which is achieved by reversing the approach taken in 338 The action is completed while avoiding sudden changes. The node returns to perform the fault detection task referred to in Section 337, and executes the flowchart of all aspects of the fault response, fault isolation and fault weakening methods, the fault response, fault isolation and Fault mitigation is performed by each arm processor 321, 323, 325, and 328, which is operatively coupled to the robotic arm of the robot 170. In 347, each arm processor is performing its normal operation During the task, it also continuously monitors its own operation and pays attention to the fault notification transmitted by the failed node in the robot arm to which it is operatively coupled. When the fault is detected when monitoring its own operation, the fault is called " Local fault”, the fault is detected by receiving a fault notification from the failed node in the robot arm that is operatively coupled, the fault is called “remote fault”, and the remote fault is operatively coupled by the arm processor. The faulty node in the connected robot arm sends a fault notification along the FRL line. The fault has been detected in 347. In the vehicle vision system 502, the arm processor locks the output motor current command of the joint position controller to zero To put its joint position controller into a safe state, which is used to strengthen the safe state of their corresponding nodes. In 349, the arm processor determines whether the detected fault is a local fault or a remote fault. In 347, the source of the fault is determined If the fault will be regarded as a local fault or a remote fault, if the fault is determined as a local fault, the arm processor is regarded as a failed node. The arm processor continues by executing the following 353-356 and 351-352, and the fault is determined as For remote failure, the arm processor is regarded as a non-failure node, and the arm processor continues by executing 350-352. In 353, the arm processor transmits the failure notification downstream to all nodes of the robotic arm that it is operatively connected to. One way for the arm processor to complete the process is by pulling the FRL line to a high state. In 354, the arm processor diagnoses the fault and sends an error message to the system management processor 320. The error message includes fault information, error code, Error type and origin. Each type of error that will affect the arm processor is assigned an error code. The error code is classified as an error type. There are at least four types of error: recoverable processor failure and unrecoverable processor Faults, recoverable system faults and non-recoverable system faults. The origin of the fault includes information about the identity of the arm processor and optional additional information about the source of the fault in the arm processor. In 355, the arm processor determines whether the detected fault is Partial faults can be recovered. The determination is done through the error category of the fault. If the determination in 355 is no, then in 356, the joint position controller of the failed arm processor remains in its safe state and the arm processor ignores its possibility Any subsequent notification of recovery received on the FRL line, the determination in 355 is yes, then the arm processor proceeds to 350, the determination in 349 is that the detected fault will be regarded as a remote fault, then in 350, the arm The processor waits for the recovery notification received from the system management processor 320. In 351, once the recovery notification is received, the arm processor transmits the recovery communication by, for example, pulling down its FRL line. Knowing all the nodes in its operatively coupled robotic arm, in 352, the arm processor then returns its joint position controller from the safe state to its normal operating state. This process is performed by releasing the joint position controller The output motor current command is completed so that they can once again reflect the desired joint position of the robotic arm to which it is operatively coupled while avoiding sudden changes. The arm processor then returns to perform its fault detection task with reference to 347, and executes A flowchart of various aspects of the methods of fault reaction, fault isolation and fault weakening. The fault reaction, fault isolation and fault weakening are executed by the system management processor 320 of the robot 170. In 357, the system management processor is executing its During normal operation tasks, it also waits to receive an error message transmitted from another component of the robot 170. If the error message is received at 357, then in 358, the system management processor for safety purposes, for example, instructs all arms in the robot system to process The joint position controllers of devices 328, 325, 323 and 321 lock their respective outputs with their current values to stop the system. No new current command input is provided to the robotic arm until the joint position controller output is unlocked. This locking of the output of the joint position controller is referred to herein as a "soft lock" joint position controller, and the method then proceeds to 359, where the system management processor determines that the detected fault should be considered a system fault It is also an arm failure. The system management processor completes this step by checking the error information provided in the error message. System failures include all failures classified as either recoverable system failures or unrecoverable system failures, because these failures can be It is not only applied to failed robotic arms. On the contrary, arm failures include all failures classified as either recoverable partial failures or unrecoverable partial failures, because these failures can only be applied to failed robotic arms, and for those that will be regarded as arm failures. For all failures, perform 360-366. In 360, the system management processor provides the remote operator 171 of the robot 170 with the option of accepting the weakened operation of the robot 170. If the partial failure is a recoverable partial failure, the system management processor The user is also provided with options to recover from the failure. In addition to each provided option, the information of the detected failure is also provided by the system management processor to assist the remote operator 171 in determining whether to accept the option, the option and the failure Information is provided on the visual display 255 of the remote console 169. In 361, the system management processor waits for the remote operator 171 to select the option provided in 360. Once the option is selected by the remote operator 171, in 362 , The system management processor determines whether the selected option is a weakened operation option or a recovery option, the recovery option is provided and the remote operator 171 selects the recovery option, then in 381, the system management processor sends a recovery notification to the failed robotic arm The arm processor of the failed robotic arm will process the The recovery notification includes sending the recovery notification to all nodes of the failed arm. The node of the failed arm then processes the recovery notification. In 382, the system management processor then releases the joint controller by unlocking the output of the joint controller of all the arm processors Soft-locked so that the joint controller again issues a motor current command reflecting the desired joint position of the robotic arm to which they are operatively coupled. Then, the system management processor returns to perform its task with reference to 357, and the remote operator 171 chooses to weaken Operation option, in 363, the system management processor provides the remote operator 171 with the option to recover from the failure. In this case, the recovery from the failure is different from the recovery with reference to 381-382, because there is no attempt to recover the failed arm. , The restoration is only used to restore the normal operation of the non-failed arm. In 364, the system management processor waits for the user to select the option provided in 363. Once the option is selected by the remote operator 171, in 365, the system management The processor sends a message to the arm processor of the failed arm to reinforce the fault. The reinforcement of the fault in this case means that additional steps are taken to completely shut down the operation of the failed robotic arm. An example of such a reinforce measure is from the master/ Operately disconnect the joint position controller of the arm processor from other parts of the control system. The master/slave control system generates the desired joint position of the operatively coupled robot arm. Another enhancement measure is to turn off the power to the failed robot arm. , In 366, the system management processor then releases the soft lock of the joint controller by unlocking the output of the joint controller of the arm processor of all non-failed arms, so that the joint controller again sends out a signal reflecting its operationally coupled The motor current at the desired joint position of the robot arm commands the system management processor to execute the task with reference to 357. In 367, the system management processor makes the system FRL status valid for all nodes in the robot 170 by causing the FRL circuit 329, 327, 384, and 385 are pulled high so that the fault notification is provided to the arm processor and the nodes of the first robot 182, second robot 183, third robot 184, and fourth robot 185 at the same time to complete this step. In 369 Then, the system management processor determines whether the system failure is a recoverable system failure, and completes this step by checking the error class in the received error message. If the determination in 369 is no, then in 363, the system The management processor takes no further action and waits for the system to be shut down. The determination in 369 is yes, then in 370, the system management processor provides the user with the option to recover from the failure. In 371, the management processor Waiting for the remote operator 171 to select the recovery option. If this option is selected, in 372, the system management processor sends a recovery notification to the arm processors of all the robot arms of the robot 170. In 373, the system management processor is receiving 171 releases the soft lock of each joint controller to operate in its normal operating state upon request or action. As the arm of the joint controller, so that the released joint controller once again sends out a motor current command reflecting the desired joint position of the robot arm to which it is operatively coupled. The system management processor then returns to perform the task of reference 357 and executes the fault response , A flowchart of various aspects of the method of fault isolation and fault weakening, which are operatively coupled to the system management processor of the robot 170 and the arms of the arm processors 328, 325, 323 and 321 The management processor is executed. The arm management processor 319 starts, controls, and monitors certain coordinated activities of the first manipulator 182, the second manipulator 183, the third manipulator 184, and the fourth manipulator 185 of the robot 170. The arm The manager 319 initiates and monitors the start of the brake test, where the arm manager 319 communicates with each of the arm processors 328, 325, and 323 so that specific braking sequences with different torque values are applied to their respective robot arms. Brake, the coordination of this activity is carried out by the arm manager 319 in this case, because the cost of encoding it to each arm processor is redundant, and is calculated by each arm processor at the end of each sequence The maximum torque value is transmitted back to the arm manager 319. When an out-of-range error occurs, the arm manager 319 will notify the failed arm of the transmission failure. The arm manager instructs the arm processor to perform arm activities, monitor the results, and determine the Does the activity result indicate arm failure? In 374, the arm manager monitors the coordinated activity of the robot arm according to the report of the respective arm processor of the robot arm to detect a fault in an arm. When the reported measurement exceeds the expected value by a threshold amount, The arm manager determines that the fault has occurred. In this case, the detected fault is one of the nodes of the robotic arm or a fault that the arm processor generally does not detect. After the fault has been detected in 374, then In 375, the arm manager suppresses any further commands to the disabled arm. No further commands will be transmitted from the arm manager to the arm processor of the disabled arm until either a recovery notification is received from the system manager or the system is restarted. In 376, the arm manager sends a fault notification to the failed arm by pulling the FRL line of the failed arm to the high state. In the case of a virtual FRL line, the arm manager passes one of the nodes of the failed arm or the arm processor The fault notification is transmitted in the same or different packet field as the packet field designated for the transmission of the fault notification. In 377, the arm manager sends an error message to the system manager, and the error message has the available details of the failure. Each fault type detected by the arm manager is assigned an error code, and the error code is classified as an error category. The origin of the fault includes the identification information of the failed arm and the optional additional information of the source of the fault. In 357, the system manager Then start to process error messages. In 378, the arm manager determines whether the detected fault is possible. Recover the fault. The determination is made according to the error class of the fault. If the determination in 378 is no, then in 381, the arm manager continues to suppress any further commands to the failed arm and ignores any subsequent recovery notifications received from the system manager , The determination in 378 is yes, then in 379, the arm manager waits for the recovery notification received from the management processor, and in 380, the recovery notification is received, the arm manager stops suppressing the command to the invalid arm and Return to its normal operating mode and perform the fault detection task referred to in 374. The system connection diagrams of the dual-mode driving mode of the second and third embodiments adopt the modular design concept of the non-entry driving controller 525, the unmanned driving controller 525 and The vehicle vision system 502 is connected, the unmanned driving controller 525 is connected to the positioning and navigation module 274, the planning system module 528, and the unmanned driving controller 525 are connected to the vehicle bus 276. The control system module 529 is connected to the robot 170. Ethernet is used between the modules. The unmanned driving controller 525 uses reconfigurable computing AI chips and is programmed based on the Linux system platform to communicate with the Internet and CAN bus. The control strategy is customized by integrating vehicle parameters and operating characteristics. The specific control strategies include: 1. Unmanned As the arbitration controller of the control mode, the driving controller 525 determines whether it is currently in the unmanned driving mode and whether it meets the conditions of the unmanned driving mode; 2. When in the remote driving mode, the control system module shields the unmanned driving controller's Control commands, in response to remote operation commands; 3. When in the unmanned driving control mode, according to the information of lidar, millimeter-wave radar and camera received by the environment sensing module received by the unmanned driving controller 525, the information is passed through each sensor The fusion optimization information, the body attitude information received by the gyroscope and the latitude and longitude information provided by the DGPS positioning and navigation module, combined with the tracking route learned through deep learning, and adaptive route planning through parallel mapping and positioning technology (SLAM) Driving, the second embodiment dual-mode driving switching logic diagram, using remote control driving mode can be switched to intelligent unmanned driving mode, emergency switching mode: "remote control driving mode" signal interruption automatic sending request 517, at this time unmanned driving control The controller judges to enter the unmanned driving mode and automatically takes over the remote driving mode, executes 518, does not allow to enter the unmanned driving mode, feedback signal 519, when it is necessary to enter the “unmanned driving mode” into the “remote control driving mode”, reset the external Unmanned driving control switch or pre-planned exit unmanned driving logic to switch to normal driving mode, execute control instruction 520, the third embodiment dual-mode driving switching logic diagram, using remote control driving mode can be switched to intelligent unmanned driving mode , Normal switching mode: when you need to enter the “intelligent driverless driving mode” from the "remote control driving mode", press the external driverless control switch to send a request 521 to the driverless controller. At this time, the driverless controller Judge whether to enter the unmanned driving mode, allow to enter the unmanned driving mode, execute 522, It is not allowed to enter the unmanned driving mode, feedback signal 523; when it is necessary to enter the “remote control driving mode” from the “unmanned driving mode”, reset the external unmanned driving control switch or the pre-planned exit unmanned driving logic switch To the normal driving mode, the control instruction 524 is executed.
PCT/CN2020/000015 2019-01-22 2020-01-07 Vehicle remote driving system established by primary and secondary wireless devices by means of internet of things connection WO2020151468A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910096218.4A CN111464978A (en) 2019-01-22 2019-01-22 Vehicle remote driving system established by connection of primary wireless equipment and secondary wireless equipment through Internet of things
CN201910096218.4 2019-01-22

Publications (1)

Publication Number Publication Date
WO2020151468A1 true WO2020151468A1 (en) 2020-07-30

Family

ID=71679897

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/000015 WO2020151468A1 (en) 2019-01-22 2020-01-07 Vehicle remote driving system established by primary and secondary wireless devices by means of internet of things connection

Country Status (2)

Country Link
CN (1) CN111464978A (en)
WO (1) WO2020151468A1 (en)

Cited By (44)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112130136A (en) * 2020-09-11 2020-12-25 中国重汽集团济南动力有限公司 Traffic target comprehensive sensing system and method
CN112134942A (en) * 2020-09-18 2020-12-25 北京市生态环境监测中心 Data quality detection device and method for heavy-duty vehicle remote monitoring platform
CN112269392A (en) * 2020-09-16 2021-01-26 广西电网有限责任公司电力科学研究院 Ground workstation system controlled by unmanned aerial vehicle cluster and control method thereof
CN112345531A (en) * 2020-10-19 2021-02-09 国网安徽省电力有限公司电力科学研究院 Transformer fault detection method based on bionic robot fish
CN112437414A (en) * 2020-09-30 2021-03-02 北方工业大学 Remote driving vehicle control information transmission and processing method
CN112492029A (en) * 2020-11-27 2021-03-12 河南汇祥通信设备有限公司 Comprehensive pipe gallery on-site multi-source comprehensive on-site positioning system and method
CN112714282A (en) * 2020-12-22 2021-04-27 北京百度网讯科技有限公司 Image processing method, apparatus, device and program product in remote control
CN112822734A (en) * 2020-12-31 2021-05-18 上海擎昆信息科技有限公司 High-speed rail line network access method and system
CN112918517A (en) * 2021-02-01 2021-06-08 中国神华能源股份有限公司神朔铁路分公司 Method and device for setting railway locomotive driving parameters, computer equipment and storage medium
CN113036915A (en) * 2021-03-04 2021-06-25 国网福建省电力有限公司厦门供电公司 Intelligent gateway-based remote monitoring and control method for park power supply and utilization equipment
CN113032003A (en) * 2021-04-08 2021-06-25 平安国际智慧城市科技股份有限公司 Development file export method and device, electronic equipment and computer storage medium
CN113112041A (en) * 2021-04-22 2021-07-13 深圳市阳谷医疗系统有限公司 Cloud computing medical system based on air purifier
CN113253274A (en) * 2021-04-30 2021-08-13 西南电子技术研究所（中国电子科技集团公司第十研究所） Fusion processing method for helicopter anti-collision ground surface power line
CN113254283A (en) * 2021-05-20 2021-08-13 中国兵器装备集团自动化研究所有限公司 Multi-CAN port test system optimization method, device, equipment and storage medium
CN113311835A (en) * 2021-05-21 2021-08-27 深圳裹动智驾科技有限公司 Method for reducing driving risk of automatic driving vehicle
CN113342169A (en) * 2021-06-10 2021-09-03 中国水利水电第七工程局有限公司 Tower crane operation virtual training system based on force feedback
CN113359124A (en) * 2021-05-20 2021-09-07 陕西长岭电子科技有限责任公司 Airborne hover indicator
CN113433548A (en) * 2021-06-24 2021-09-24 中国第一汽车股份有限公司 Data monitoring method, device, equipment and storage medium
CN113538759A (en) * 2021-07-08 2021-10-22 深圳创维－Rgb电子有限公司 Access control management method, device, equipment and storage medium based on display equipment
CN113715868A (en) * 2021-06-17 2021-11-30 上海应用技术大学 Remote track detection system based on time-space coupling
CN113741401A (en) * 2021-09-18 2021-12-03 南昌济铃新能源科技有限责任公司 Car as a house bus system
CN113865596A (en) * 2021-08-31 2021-12-31 广东省威汇智能科技有限公司 Frame pasting equipment based on automobile navigation field
CN113965619A (en) * 2021-10-21 2022-01-21 宜信普惠信息咨询(北京)有限公司 GPS device online judging method and device
CN114143086A (en) * 2021-11-30 2022-03-04 北京天融信网络安全技术有限公司 Web application identification method and device, electronic equipment and storage medium
CN114185348A (en) * 2021-12-03 2022-03-15 东风悦享科技有限公司 Remote driving control system and method for crawler-type patrol car
CN114475734A (en) * 2022-02-28 2022-05-13 中南大学 Wireless remote locomotive control method and system
CN114585054A (en) * 2022-02-23 2022-06-03 北京小米移动软件有限公司 WIFI connection control method and device and storage medium
CN114582154A (en) * 2020-11-30 2022-06-03 丰田自动车株式会社 Image display apparatus and method, non-transitory computer-readable medium
CN114726672A (en) * 2022-03-08 2022-07-08 广东空天科技研究院 High-speed aircraft ground station redundancy man-machine interaction system and method
CN114756007A (en) * 2022-04-20 2022-07-15 中国第一汽车股份有限公司 Evaluation method, device, equipment and storage medium
CN114863711A (en) * 2021-12-13 2022-08-05 广东电网有限责任公司 Internet of vehicles vehicle positioning method and system
CN114970608A (en) * 2022-05-06 2022-08-30 中国科学院自动化研究所 Human-computer interaction method and system based on electro-oculogram signals
CN115032969A (en) * 2022-06-27 2022-09-09 中国第一汽车股份有限公司 Ethernet test system of vehicle-mounted controller
CN115331190A (en) * 2022-09-30 2022-11-11 北京闪马智建科技有限公司 Road hidden danger identification method and device based on radar fusion
CN115324443A (en) * 2022-07-08 2022-11-11 卡斯柯信号有限公司 Vehicle door automatic alignment isolation system and method based on online detection
CN115396755A (en) * 2022-10-28 2022-11-25 高勘(广州)技术有限公司 Power pipe gallery operation and maintenance method, device, equipment and storage medium
CN115410233A (en) * 2022-11-01 2022-11-29 齐鲁工业大学 Gesture attitude estimation method based on Kalman filtering and deep learning
CN115599737A (en) * 2022-12-13 2023-01-13 南京芯驰半导体科技有限公司（Cn） Heterogeneous multi-core system, communication method, chip, equipment and storage medium
CN116112529A (en) * 2023-04-03 2023-05-12 三一汽车起重机械有限公司 Multi-mode signal transmission method and device and engineering machinery
WO2023220854A1 (en) * 2022-05-16 2023-11-23 Oppo广东移动通信有限公司 Access control method, terminal, chip, readable storage medium, and computer program product
US11835008B1 (en) 2023-01-12 2023-12-05 Ford Global Technologies, Llc Engine and engine exhaust control system
CN117473456A (en) * 2023-12-28 2024-01-30 北京卓视智通科技有限责任公司 Intelligent fusion method and system for thunder
CN117681852A (en) * 2024-02-02 2024-03-12 万向钱潮股份公司 Linear control brake control method based on corner sensor fault
CN117681852B (en) * 2024-02-02 2024-04-30 万向钱潮股份公司 Linear control brake control method based on corner sensor fault

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112364701B (en) * 2020-10-15 2022-11-04 东风汽车集团有限公司 Video image processing method and device applied to parallel driving system
CN112288906B (en) * 2020-10-27 2022-08-02 北京五一视界数字孪生科技股份有限公司 Method and device for acquiring simulation data set, storage medium and electronic equipment
CN113454613B (en) * 2020-10-30 2022-08-09 华为技术有限公司 Information transmission method, control device, electromagnetic signal transmitting and receiving device and signal processing equipment
CN113040757B (en) * 2021-03-02 2022-12-20 江西台德智慧科技有限公司 Head posture monitoring method and device, head intelligent wearable device and storage medium
CN113112844A (en) * 2021-03-18 2021-07-13 浙江金乙昌科技股份有限公司 Vehicle remote control system based on 5G communication and high-precision positioning and control device thereof
CN112734927B (en) * 2021-03-31 2021-06-25 湖北亿咖通科技有限公司 Method and device for simplifying high-precision map lane line and computer storage medium
CN115958996A (en) * 2021-04-14 2023-04-14 岳秀兰 Aircraft operation and protection system consisting of remote driving, energy supply and ground aircraft carrier
CN113434764B (en) * 2021-06-29 2023-10-24 青岛海尔科技有限公司 Content pushing method and device, storage medium and electronic device
CN113920762A (en) * 2021-10-08 2022-01-11 湖南湘江智能科技创新中心有限公司 Control method for prior passage of emergency vehicles based on intelligent network environment
CN114578188B (en) * 2022-05-09 2022-07-08 环球数科集团有限公司 Power grid fault positioning method based on Beidou satellite
CN115240292A (en) * 2022-06-24 2022-10-25 柳州铁道职业技术学院 Operation monitoring system, method, equipment and medium for coach bus
CN115297539B (en) * 2022-10-10 2023-02-03 联友智连科技有限公司 Internet of vehicles terminal time synchronization method, system and electronic equipment
CN115701894B (en) * 2022-12-27 2023-10-24 蔚来汽车科技(安徽)有限公司 Communication method, communication system and computer storage medium
CN116052121B (en) * 2023-01-28 2023-06-27 上海芯算极科技有限公司 Multi-sensing target detection fusion method and device based on distance estimation
CN116170779B (en) * 2023-04-18 2023-07-25 西安深信科创信息技术有限公司 Collaborative awareness data transmission method, device and system

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105869419A (en) * 2016-04-25 2016-08-17 张开冰 Intelligent travelling system
CN107010152A (en) * 2016-11-25 2017-08-04 罗轶 It is public voluntarily to wrap
CN108037760A (en) * 2017-12-12 2018-05-15 成都育芽科技有限公司 A kind of unmanned automatic driving automobile autonomous operation control method
CN108063797A (en) * 2017-11-17 2018-05-22 南京视莱尔汽车电子有限公司 A kind of operation posture complete monitoring method of autonomous driving vehicle
WO2018169590A1 (en) * 2017-03-17 2018-09-20 Autoliv Asp, Inc. Improved asil-classification by cooperative positioning

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU2006306523B2 (en) * 2005-10-21 2011-05-19 Deere & Company Systems and methods for switching between autonomous and manual operation of a vehicle
US9446517B2 (en) * 2013-10-17 2016-09-20 Intuitive Surgical Operations, Inc. Fault reaction, fault isolation, and graceful degradation in a robotic system
US9439232B2 (en) * 2014-01-17 2016-09-06 GM Global Technology Operations LLC Managing traditional Wi-Fi and Wi-Fi direct connections using a wireless device
CN106656195A (en) * 2015-11-04 2017-05-10 北京信威通信技术股份有限公司 Data compression decompression method and data compression decompression system
CN107305372B (en) * 2016-04-25 2020-06-19 岳秀兰 Electric automobile energy monitoring and supply network for remote monitoring of cloud computing network architecture
CN114767266A (en) * 2016-06-09 2022-07-22 直观外科手术操作公司 Computer-assisted teleoperation surgical system and method
CN106101590B (en) * 2016-06-23 2019-07-19 上海无线电设备研究所 The detection of radar video complex data and processing system and detection and processing method
US10162360B2 (en) * 2016-12-01 2018-12-25 GM Global Technology Operations LLC Vehicle environment imaging systems and methods
CN108995538A (en) * 2018-08-09 2018-12-14 金龙联合汽车工业（苏州）有限公司 A kind of Unmanned Systems of electric car
CN108811191A (en) * 2018-09-18 2018-11-13 南昌工程学院 microwave channel, mobile communication, satellite communication network access system and working method

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105869419A (en) * 2016-04-25 2016-08-17 张开冰 Intelligent travelling system
CN107010152A (en) * 2016-11-25 2017-08-04 罗轶 It is public voluntarily to wrap
WO2018169590A1 (en) * 2017-03-17 2018-09-20 Autoliv Asp, Inc. Improved asil-classification by cooperative positioning
CN108063797A (en) * 2017-11-17 2018-05-22 南京视莱尔汽车电子有限公司 A kind of operation posture complete monitoring method of autonomous driving vehicle
CN108037760A (en) * 2017-12-12 2018-05-15 成都育芽科技有限公司 A kind of unmanned automatic driving automobile autonomous operation control method

Cited By (71)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112130136B (en) * 2020-09-11 2024-04-12 中国重汽集团济南动力有限公司 Comprehensive perception system and method for traffic targets
CN112130136A (en) * 2020-09-11 2020-12-25 中国重汽集团济南动力有限公司 Traffic target comprehensive sensing system and method
CN112269392A (en) * 2020-09-16 2021-01-26 广西电网有限责任公司电力科学研究院 Ground workstation system controlled by unmanned aerial vehicle cluster and control method thereof
CN112269392B (en) * 2020-09-16 2023-08-22 广西电网有限责任公司电力科学研究院 Unmanned aerial vehicle cluster control ground workstation system and control method thereof
CN112134942A (en) * 2020-09-18 2020-12-25 北京市生态环境监测中心 Data quality detection device and method for heavy-duty vehicle remote monitoring platform
CN112134942B (en) * 2020-09-18 2023-12-19 北京市生态环境监测中心 Heavy vehicle remote monitoring platform data quality detection device and method
CN112437414B (en) * 2020-09-30 2023-06-30 北方工业大学 Remote driving vehicle control information transmission and processing method
CN112437414A (en) * 2020-09-30 2021-03-02 北方工业大学 Remote driving vehicle control information transmission and processing method
CN112345531B (en) * 2020-10-19 2024-04-09 国网安徽省电力有限公司电力科学研究院 Transformer fault detection method based on bionic robot fish
CN112345531A (en) * 2020-10-19 2021-02-09 国网安徽省电力有限公司电力科学研究院 Transformer fault detection method based on bionic robot fish
CN112492029A (en) * 2020-11-27 2021-03-12 河南汇祥通信设备有限公司 Comprehensive pipe gallery on-site multi-source comprehensive on-site positioning system and method
CN112492029B (en) * 2020-11-27 2022-12-23 河南汇祥通信设备有限公司 Comprehensive pipe gallery on-site multi-source comprehensive on-site positioning system and method
CN114582154B (en) * 2020-11-30 2023-12-29 丰田自动车株式会社 Image display device and method, non-transitory computer readable medium
CN114582154A (en) * 2020-11-30 2022-06-03 丰田自动车株式会社 Image display apparatus and method, non-transitory computer-readable medium
CN112714282A (en) * 2020-12-22 2021-04-27 北京百度网讯科技有限公司 Image processing method, apparatus, device and program product in remote control
CN112822734B (en) * 2020-12-31 2023-01-31 上海擎昆信息科技有限公司 High-speed rail line network access method and system
CN112822734A (en) * 2020-12-31 2021-05-18 上海擎昆信息科技有限公司 High-speed rail line network access method and system
CN112918517B (en) * 2021-02-01 2023-02-17 中国神华能源股份有限公司神朔铁路分公司 Method and device for setting railway locomotive driving parameters, computer equipment and storage medium
CN112918517A (en) * 2021-02-01 2021-06-08 中国神华能源股份有限公司神朔铁路分公司 Method and device for setting railway locomotive driving parameters, computer equipment and storage medium
CN113036915A (en) * 2021-03-04 2021-06-25 国网福建省电力有限公司厦门供电公司 Intelligent gateway-based remote monitoring and control method for park power supply and utilization equipment
CN113036915B (en) * 2021-03-04 2024-02-27 国网福建省电力有限公司厦门供电公司 Intelligent gateway-based park power supply equipment remote monitoring and control method
CN113032003A (en) * 2021-04-08 2021-06-25 平安国际智慧城市科技股份有限公司 Development file export method and device, electronic equipment and computer storage medium
CN113032003B (en) * 2021-04-08 2024-04-02 深圳赛安特技术服务有限公司 Development file export method, development file export device, electronic equipment and computer storage medium
CN113112041A (en) * 2021-04-22 2021-07-13 深圳市阳谷医疗系统有限公司 Cloud computing medical system based on air purifier
CN113253274B (en) * 2021-04-30 2024-02-06 西南电子技术研究所（中国电子科技集团公司第十研究所） Fusion processing method for anti-collision ground surface power line of helicopter
CN113253274A (en) * 2021-04-30 2021-08-13 西南电子技术研究所（中国电子科技集团公司第十研究所） Fusion processing method for helicopter anti-collision ground surface power line
CN113254283A (en) * 2021-05-20 2021-08-13 中国兵器装备集团自动化研究所有限公司 Multi-CAN port test system optimization method, device, equipment and storage medium
CN113359124A (en) * 2021-05-20 2021-09-07 陕西长岭电子科技有限责任公司 Airborne hover indicator
CN113359124B (en) * 2021-05-20 2024-02-23 陕西长岭电子科技有限责任公司 Airborne hover indicator
CN113311835A (en) * 2021-05-21 2021-08-27 深圳裹动智驾科技有限公司 Method for reducing driving risk of automatic driving vehicle
CN113311835B (en) * 2021-05-21 2023-12-29 深圳安途智行科技有限公司 Method for reducing running risk of automatic driving vehicle
CN113342169A (en) * 2021-06-10 2021-09-03 中国水利水电第七工程局有限公司 Tower crane operation virtual training system based on force feedback
CN113715868A (en) * 2021-06-17 2021-11-30 上海应用技术大学 Remote track detection system based on time-space coupling
CN113433548A (en) * 2021-06-24 2021-09-24 中国第一汽车股份有限公司 Data monitoring method, device, equipment and storage medium
CN113538759A (en) * 2021-07-08 2021-10-22 深圳创维－Rgb电子有限公司 Access control management method, device, equipment and storage medium based on display equipment
CN113538759B (en) * 2021-07-08 2023-08-04 深圳创维－Rgb电子有限公司 Gate inhibition management method, device and equipment based on display equipment and storage medium
CN113865596A (en) * 2021-08-31 2021-12-31 广东省威汇智能科技有限公司 Frame pasting equipment based on automobile navigation field
CN113741401A (en) * 2021-09-18 2021-12-03 南昌济铃新能源科技有限责任公司 Car as a house bus system
CN113965619B (en) * 2021-10-21 2024-03-01 宜信普惠信息咨询(北京)有限公司 GPS equipment online judgment method and device
CN113965619A (en) * 2021-10-21 2022-01-21 宜信普惠信息咨询(北京)有限公司 GPS device online judging method and device
CN114143086B (en) * 2021-11-30 2023-09-26 北京天融信网络安全技术有限公司 Web application identification method and device, electronic equipment and storage medium
CN114143086A (en) * 2021-11-30 2022-03-04 北京天融信网络安全技术有限公司 Web application identification method and device, electronic equipment and storage medium
CN114185348A (en) * 2021-12-03 2022-03-15 东风悦享科技有限公司 Remote driving control system and method for crawler-type patrol car
CN114863711B (en) * 2021-12-13 2023-09-29 广东电网有限责任公司 Vehicle networking vehicle positioning method and system
CN114863711A (en) * 2021-12-13 2022-08-05 广东电网有限责任公司 Internet of vehicles vehicle positioning method and system
CN114585054B (en) * 2022-02-23 2023-11-14 北京小米移动软件有限公司 WIFI connection control method, device and storage medium
CN114585054A (en) * 2022-02-23 2022-06-03 北京小米移动软件有限公司 WIFI connection control method and device and storage medium
CN114475734A (en) * 2022-02-28 2022-05-13 中南大学 Wireless remote locomotive control method and system
CN114475734B (en) * 2022-02-28 2023-11-28 中南大学 Wireless remote locomotive control method and system
CN114726672B (en) * 2022-03-08 2023-08-04 广东空天科技研究院 Human-computer interaction system and method for redundancy of ground station of high-speed aircraft
CN114726672A (en) * 2022-03-08 2022-07-08 广东空天科技研究院 High-speed aircraft ground station redundancy man-machine interaction system and method
CN114756007A (en) * 2022-04-20 2022-07-15 中国第一汽车股份有限公司 Evaluation method, device, equipment and storage medium
CN114970608B (en) * 2022-05-06 2023-06-02 中国科学院自动化研究所 Man-machine interaction method and system based on electro-oculogram signals
CN114970608A (en) * 2022-05-06 2022-08-30 中国科学院自动化研究所 Human-computer interaction method and system based on electro-oculogram signals
WO2023220854A1 (en) * 2022-05-16 2023-11-23 Oppo广东移动通信有限公司 Access control method, terminal, chip, readable storage medium, and computer program product
CN115032969A (en) * 2022-06-27 2022-09-09 中国第一汽车股份有限公司 Ethernet test system of vehicle-mounted controller
CN115324443B (en) * 2022-07-08 2024-04-09 卡斯柯信号有限公司 Automatic vehicle door alignment isolation system and method based on online detection
CN115324443A (en) * 2022-07-08 2022-11-11 卡斯柯信号有限公司 Vehicle door automatic alignment isolation system and method based on online detection
CN115331190B (en) * 2022-09-30 2022-12-09 北京闪马智建科技有限公司 Road hidden danger identification method and device based on radar vision fusion
CN115331190A (en) * 2022-09-30 2022-11-11 北京闪马智建科技有限公司 Road hidden danger identification method and device based on radar fusion
CN115396755A (en) * 2022-10-28 2022-11-25 高勘(广州)技术有限公司 Power pipe gallery operation and maintenance method, device, equipment and storage medium
CN115410233A (en) * 2022-11-01 2022-11-29 齐鲁工业大学 Gesture attitude estimation method based on Kalman filtering and deep learning
CN115599737A (en) * 2022-12-13 2023-01-13 南京芯驰半导体科技有限公司（Cn） Heterogeneous multi-core system, communication method, chip, equipment and storage medium
CN115599737B (en) * 2022-12-13 2023-02-28 南京芯驰半导体科技有限公司 Heterogeneous multi-core system, communication method, chip, equipment and storage medium
US11835008B1 (en) 2023-01-12 2023-12-05 Ford Global Technologies, Llc Engine and engine exhaust control system
CN116112529A (en) * 2023-04-03 2023-05-12 三一汽车起重机械有限公司 Multi-mode signal transmission method and device and engineering machinery
CN116112529B (en) * 2023-04-03 2023-06-09 三一汽车起重机械有限公司 Multi-mode signal transmission method and device and engineering machinery
CN117473456A (en) * 2023-12-28 2024-01-30 北京卓视智通科技有限责任公司 Intelligent fusion method and system for thunder
CN117473456B (en) * 2023-12-28 2024-02-27 北京卓视智通科技有限责任公司 Intelligent fusion method and system for thunder
CN117681852A (en) * 2024-02-02 2024-03-12 万向钱潮股份公司 Linear control brake control method based on corner sensor fault
CN117681852B (en) * 2024-02-02 2024-04-30 万向钱潮股份公司 Linear control brake control method based on corner sensor fault

Also Published As

Publication number Publication date
CN111464978A (en) 2020-07-28

Similar Documents

Publication Publication Date Title
WO2020151468A1 (en) Vehicle remote driving system established by primary and secondary wireless devices by means of internet of things connection
US11242066B2 (en) Vehicle control apparatus and vehicle control system and vehicle control method thereof
US20190064813A1 (en) Systems and Methods of Controlling an Autonomous Vehicle Using an Enhanced Trajectory Following Configuration
CN110568852A (en) Automatic driving system and control method thereof
JP7200946B2 (en) Information processing device, control system, information processing method and program
US10929995B2 (en) Method and apparatus for predicting depth completion error-map for high-confidence dense point-cloud
KR20160113039A (en) Vehicle combination and method for forming and operating a vehicle combination
US20220182498A1 (en) System making decision based on data communication
JP7320001B2 (en) Information processing device, information processing method, program, mobile body control device, and mobile body
CN110716549A (en) Autonomous navigation robot system for map-free area patrol and navigation method thereof
WO2022218219A1 (en) Aircraft operation guarantee system consisting of remote driving, energy supply, and ground carrier
CN112544061B (en) Data transmission method and device
CN104122891A (en) Intelligent robot inspection system for city underground railway detection
CA3136909A1 (en) Systems and methods for simultaneous localization and mapping using asynchronous multi-view cameras
WO2020116194A1 (en) Information processing device, information processing method, program, mobile body control device, and mobile body
CN114348020A (en) 5G remote and automatic driving safety redundancy system and control method
CN109177753A (en) Automobile-used aircraft and the automobile for carrying aircraft
CN108958248A (en) Standby system
CN113401116A (en) Method and system for autonomous driving of a vehicle
US20220222936A1 (en) Outside environment recognition device
US11110970B2 (en) Removable interior for reconfigurable vehicles
Lu et al. Teleoperation technologies for enhancing connected and autonomous vehicles
Bećirbašić et al. Video-processing platform for semi-autonomous driving over 5G networks
CN211669779U (en) Intelligent networking automobile teaching training platform
US20220094435A1 (en) Visible light communication apparatus, visible light communication method, and visible light communication program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20744228

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20744228

Country of ref document: EP

Kind code of ref document: A1