CN111065982B - Control method, control equipment, control system and storage medium - Google Patents

Control method, control equipment, control system and storage medium Download PDF

Info

Publication number
CN111065982B
CN111065982B CN201880036977.5A CN201880036977A CN111065982B CN 111065982 B CN111065982 B CN 111065982B CN 201880036977 A CN201880036977 A CN 201880036977A CN 111065982 B CN111065982 B CN 111065982B
Authority
CN
China
Prior art keywords
control
remote control
mobile robot
equipment
terminal equipment
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201880036977.5A
Other languages
Chinese (zh)
Other versions
CN111065982A (en
Inventor
邹旭
杨小虎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
SZ DJI Technology Co Ltd
Original Assignee
SZ DJI Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by SZ DJI Technology Co Ltd filed Critical SZ DJI Technology Co Ltd
Publication of CN111065982A publication Critical patent/CN111065982A/en
Application granted granted Critical
Publication of CN111065982B publication Critical patent/CN111065982B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02Control of position or course in two dimensions
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/10Simultaneous control of position or course in three dimensions

Landscapes

  • Engineering & Computer Science (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Selective Calling Equipment (AREA)
  • Telephonic Communication Services (AREA)
  • Telephone Function (AREA)
  • Manipulator (AREA)

Abstract

A remote control device of a mobile robot, comprising: the remote control device comprises a first interaction device and a processor, wherein the first interaction device is used for detecting control operation of a user, the processor generates a control instruction according to the control operation detected by the first interaction device, when the remote control device is in a mobile robot control mode, a mobile robot in communication connection with the control device is controlled according to the control instruction, and when the remote control device is in a terminal device control mode, a terminal device in communication connection with the remote control device is controlled according to the control instruction. The terminal equipment and the mobile robot are controlled through the interaction device of the remote control equipment, and the user is not required to switch. Also included are a control method for the remote control device, a mobile robot system including the remote control device, and a computer readable storage medium executing the control method.

Description

Control method, control equipment, control system and storage medium
Technical Field
The present invention relates to the field of control technologies, and in particular, to a control method, device, system, and storage medium.
Background
With the development of control technology, mobile robots become a relatively popular research topic at present, and are widely applied to the fields of plant protection, shooting, forest fire monitoring, security monitoring and the like. Currently, a user may control movement of a mobile robot through a remote control device and view sensor data collected by the mobile robot (e.g., image data collected by the mobile robot) through a terminal device communicatively connected to the remote control device.
However, when the user wants to control the mobile robot and control the terminal device, the interaction device of the remote control device and the interaction device of the terminal device need to be operated respectively, and this operation mode frequently switches the interaction device to realize the respective control of the mobile robot and the terminal device, which is complicated in operation mode and poor in interaction friendliness.
Disclosure of Invention
The embodiment of the invention provides a control method, equipment, a system and a storage medium, which can improve the convenience of controlling a mobile robot and terminal equipment.
In a first aspect, an embodiment of the present invention provides a remote control device for a mobile robot, including: a first interaction means and a processor, wherein,
the first interaction device is used for detecting control operation of a user;
the processor is configured to:
generating a control instruction according to the control operation detected by the first interaction device;
when the remote control equipment is in a mobile robot control mode, controlling a mobile robot in communication connection with the remote control equipment according to the control instruction;
when the remote control equipment is in a terminal equipment control mode, controlling terminal equipment in communication connection with the remote control equipment according to the control instruction.
In a second aspect, an embodiment of the present invention provides a control method applied to a remote control device of a mobile robot, where the remote control device includes a first interaction device, the method includes:
detecting a control operation of a user through a first interaction device;
generating a control instruction according to the control operation detected by the first interaction device;
when the remote control equipment is in a mobile robot control mode, controlling a mobile robot in communication connection with the remote control equipment according to the control instruction;
when the remote control equipment is in a terminal equipment control mode, controlling terminal equipment in communication connection with the remote control equipment according to the control instruction.
In a third aspect, an embodiment of the present invention provides a mobile robot system, including: the remote control device, the terminal device, and the mobile robot as described in the first aspect,
the terminal equipment at least comprises a touch display screen.
In a fourth aspect, embodiments of the present invention provide a computer-readable storage medium storing a computer program which, when executed by a processor, implements the control method as described in the above second aspect.
In the embodiment of the invention, the remote control equipment can detect the control operation of a user; generating a control instruction according to the detected control operation; when the remote control equipment is in a mobile robot control mode, controlling a mobile robot in communication connection with the remote control equipment according to the control instruction; when the remote control equipment is in a terminal equipment control mode, controlling the terminal equipment in communication connection with the remote control equipment according to the control instruction. By the mode, the terminal equipment and the mobile robot can be controlled through the interaction device of the remote control equipment, the user is not required to switch the interaction device, and a better man-machine interaction mode is provided for the user.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic structural diagram of a mobile robot system according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of another mobile robot system according to an embodiment of the present invention;
FIG. 3 is a schematic flow chart of a control method according to an embodiment of the present invention;
FIG. 4 is a flow chart of another control method according to an embodiment of the present invention;
FIG. 5 is a flow chart of yet another control method according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a remote control device according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Some embodiments of the present invention are described in detail below with reference to the accompanying drawings. The following embodiments and features of the embodiments may be combined with each other without conflict.
Currently, a remote control system of a mobile robot includes a remote control device and a terminal device, wherein the remote control device may include a remote controller, and the remote controller is generally used for controlling a moving state (such as a position, a speed, or an acceleration) of the mobile robot and/or a pan-tilt control of the mobile robot and/or a shooting action of the mobile robot. The terminal device may be a device configured with a touch display screen, and is generally used for displaying or editing an image captured by a photographing device of the mobile robot, starting an intelligent control mode (e.g., a tracking mode, a pointing movement mode), and the like. In the prior art, the remote control device and the terminal device generally operate independently, and the remote control device only sends a self-generated instruction and/or an instruction of the terminal device to the mobile robot through a communication device (such as a graphic transmission module) configured by the remote control device, and/or receives an image acquired by a shooting device of the mobile robot.
When a user wants to control the mobile robot and control the terminal equipment, the interaction device of the remote control equipment and the interaction device of the terminal equipment are required to be operated respectively, and the operation mode is frequently switched to realize the respective control of the mobile robot and the terminal equipment, so that the operation mode is complex and the interaction friendliness is poor. For example, in a specific application scenario, when a user performs an interactive operation of operating the remote control device with a glove in an outdoor cold scenario, the user controls the mobile robot, however, when the user thinks of operating the touch display screen of the terminal device, the user needs to take off the glove to perform a finger touch operation on the touch display screen, which results in inconvenient operation. In order to solve the problem, the embodiment of the invention provides a control method, which can be applied to a remote control device, wherein the remote control device can comprise a first interaction device, wherein the first interaction device is any device for interacting with a user, and particularly the first interaction device can comprise one or more of a keyboard, a key, a rocker and a wave wheel; generating a control instruction according to the control operation detected by the first interaction device; when the remote control equipment is in a mobile robot control mode, controlling a mobile robot in communication connection with the remote control equipment according to the control instruction; when the remote control equipment is in a terminal equipment control mode, controlling the terminal equipment in communication connection with the remote control equipment according to the control instruction. It can be seen that the remote control device can control the terminal device and the mobile robot in different control modes respectively, specifically, the mobile robot and the terminal device can be controlled through the first interaction device of the remote control device, so that convenience in controlling the terminal device and the mobile robot is improved, the control modes of the remote control device are enriched, the use value of the remote control device is improved, the interaction device does not need to be frequently switched, and a better man-machine interaction mode is provided for a user.
In addition, when the remote control device is in the terminal device control mode and the terminal device is required to execute a preset work task, for example, when a target application program (such as a photographing application program) is started, the user needs to operate the first interaction device of the remote control device, and does not need to switch to the operation of the interaction device of the terminal device, for example, the target application program is started through finger touch operation, so that the convenience of man-machine interaction is improved.
In order to facilitate understanding of the control method described in the present application, the embodiment of the present application further provides a mobile robot system, where the mobile robot system includes a control system of a mobile robot and a mobile robot as described above, and the remote control system of the mobile robot includes a remote control device and a terminal device as described above. The mobile robot may be an unmanned aerial vehicle, an unmanned ground robot, an unmanned ship, or the like, wherein the unmanned aerial vehicle may be a rotary-wing mobile robot, such as a four-rotor mobile robot, a six-rotor mobile robot, an eight-rotor mobile robot. The mobile robot comprises a power system, wherein the power system is used for providing mobile power for the mobile robot, the power system comprises one or more of an engine, a propeller, a motor and an electric regulator, the mobile robot can further comprise a cradle head and a shooting device, and the shooting device is mounted on a main body of the mobile robot through the cradle head. The shooting device is used for shooting images or videos in the moving process of the mobile robot, including but not limited to a multispectral imager, a hyperspectral imager, a visible light camera, an infrared camera and the like, the cradle head is a multi-shaft transmission and stability increasing system, the shooting angle of the imaging equipment is compensated by adjusting the rotation angle of the rotation shaft, and the shake of the imaging equipment is prevented or reduced by arranging a proper buffer mechanism.
In the embodiment of the invention, the remote control device can establish a wireless communication connection with the mobile robot, for example, the remote control device comprises a communication device, and the remote control device can establish the wireless communication connection with the mobile robot through the communication device, wherein the communication device can comprise a graph transmission module. The first interaction means of the remote control device may control the movement state (e.g. position, speed or acceleration etc.) of the mobile robot and/or the pan-tilt control of the mobile robot and/or the shooting action of the mobile robot via a wireless communication connection established with the mobile robot. Wherein the wireless communication connection may be based on WIFI, SDR, bluetooth or other proprietary protocols.
In the embodiment of the invention, the remote control device can also establish wired communication connection or wireless communication connection with the terminal device, wherein the wireless communication connection can be based on WIFI, SDR, bluetooth or other private protocols, and the wired communication connection can be based on USB lines. The first interaction means of the remote control device may control the terminal device through a wired or wireless communication connection established with the terminal device, for example, control one or more of starting a photographing function by the terminal device, starting a screen capturing operation by the terminal device, controlling an interaction interface displayed by a touch display screen of the terminal device (for example, controlling sliding of the interaction interface, etc.), and starting editing of an image by the terminal device.
The terminal device may be a device including at least a touch display screen, and in some embodiments, may be one or more of a smart phone, a tablet computer, a laptop computer, a ground station, and a wearable device (watch, bracelet), as described above. In some embodiments, the terminal device may be a customized device including a touch display screen. The operating system of the terminal device may refer to an android operating system, a Linux operating system or an IOS operating system, and the like.
The android operating system is a Linux-based free and open source operating system, and the system defaults to support the terminal equipment to establish communication connection with USB-HID (Human Interface Device) type remote control equipment. Thus, in one embodiment, the communication connection established by the remote control device with the terminal device may refer to a communication connection established through a USB interface.
The Linux operating system supports a bluetooth protocol stack, and also supports that the terminal device and the USB-HID remote control device establish communication connection on the bluetooth protocol stack, so in one embodiment, the communication connection established between the remote control device and the terminal device may be a communication connection established through bluetooth.
The USB-HID remote control device may be a device including a keyboard, a mouse, or a joystick.
In one embodiment, the terminal device and the remote control device may be physically separate devices, as shown in fig. 1, the mobile robot system includes a remote control device 11 and a terminal device 12, and the remote control device 11 and the terminal device 12 are physically separate devices. Specifically, the remote control device 11 and the terminal device 12 may be housed in different housings, for example, the remote control device 11 is housed in the housing 15, and the terminal device 12 is housed in the housing 14.
In one embodiment, the terminal device and the remote control device are housed in the same housing. As shown in fig. 2, the mobile robot system includes a remote control device 11, a terminal device 12, and a mobile robot 13, wherein the remote control device 11 and the terminal device 12 may be housed in the same housing 16.
Referring to fig. 3, fig. 3 is a flowchart of a control method according to an embodiment of the present invention, which may be performed by a remote control device, where a specific explanation of the remote control device is as described above. As shown in fig. 3, the control method may include the following steps.
S101, detecting control operation of a user through a first interaction device.
In the implementation of the invention, when the user needs the terminal equipment or the mobile robot to execute a certain work task, the user can control the first interaction device of the remote control equipment, for example, the user can input control operation in at least one mode of keys, rockers, thumb wheels and the like. The first interaction means may detect a control operation entered by the user.
In one embodiment, when the user inputs the control operation by means of a key, detecting the control operation of the user by the first interaction device includes: the control operation of the user is detected through the keyboard.
In one embodiment, when the user inputs the control operation by means of a joystick or a thumb wheel, detecting the control operation of the user by the first interaction means comprises: through the control operation of a rocker or a thumb wheel user.
S102, generating a control instruction according to the control operation detected by the first interaction device.
In the embodiment of the invention, after the first interaction device detects the control operation of the user, the processor of the remote control device may generate the control instruction according to the control operation detected by the first interaction device. Wherein the control operation of the user corresponds to the generated control instruction.
In one embodiment, when the user inputs the control operation by means of the keys, the control instruction may be generated according to parameters such as the strength of detecting the pressed key or the identification number of the pressed key in the keyboard. For example, the control instruction is generated based on the identification number of the pressed key. Specifically, the remote control device sets the corresponding relation between the identification number of the key and different control instructions in advance, and the remote control device can generate the control instructions according to the identification number of the pressed key and the corresponding relation.
In another embodiment, when the user inputs a control operation by means of a joystick or a thumb wheel, the remote control device may generate a control command based on detecting parameters such as a direction and/or angle in which the joystick or thumb wheel is rocked. For example, the control command is generated according to the shaking direction and/or angle of the rocker. Specifically, the remote control device presets the corresponding relation between different directions and/or angles of the rocking bar and different control instructions, and the remote control device can generate the control instructions according to the corresponding relation between the directions and/or angles of the rocking bar and the corresponding relation.
And S103, when the remote control equipment is in a mobile robot control mode, controlling the mobile robot in communication connection with the remote control equipment according to the control instruction.
In the embodiment of the invention, the remote control device can control the mobile robot and also can control the terminal device. When the remote control device is in different control modes, the remote control device can control different objects, namely the generated control instructions are used for controlling the different objects, and the control mode of the remote control device can determine the objects to be controlled. Specifically, when the remote control device is in the mobile robot control mode, it indicates that the remote control device can interact information with the mobile robot at this time, that is, the remote control device can send a control command to the mobile robot, so that the remote control device can control the mobile robot in communication connection with the remote control device according to the control command, so as to realize control of the mobile robot by the remote control device.
In one embodiment, the control mode of the remote control device may be set by the remote control device according to the distance of the remote control device from the mobile robot. For example, when the distance between the remote control device and the mobile robot is greater than a preset distance threshold (e.g., 500 m), the remote control device may set the control mode to the terminal device control mode, otherwise, the control mode is set to the mobile robot control mode, and the preset distance threshold may be set according to the distance that the remote control device is capable of controlling the mobile robot.
In one embodiment, the control mode of the remote control device may be set by the remote control device according to an application scenario. For example, in a scenario where the mobile robot needs to be controlled (for example, when the movement track of the mobile robot needs to be controlled), the remote control device may set the control mode to the mobile robot control mode; in a scenario where control of the terminal device is required (for example, when the interface displayed by the touch display screen of the terminal device is required to be controlled to slide), the remote control device may set the control mode to the terminal control mode.
In one embodiment, the control mode of the remote control device may be manually set by a user. For example, the remote control device may provide at least one control mode key, and when the remote control device detects a pressing operation on the control mode key in a preset time period (e.g. 5 s) when there is only one control mode key, the control mode is set to be the mobile robot control mode; and when the remote control device detects more than one pressing operation of the control mode key within a preset time period (such as 5 s), setting the control mode as a terminal device control mode. When the remote control device comprises two control mode keys, one is a mobile robot control mode key, the other is a terminal device control mode key, and if a pressing operation for the mobile robot control mode key is detected, the control mode is set to be a mobile robot control mode; and if the pressing operation of the terminal equipment control mode key is detected, setting the control mode as the terminal equipment control mode.
When the remote control device is in the mobile robot control mode, the control instruction may include an instruction for controlling a photographing device of the mobile robot to perform photographing, an instruction for controlling the mobile robot to send image data to the terminal device or the remote control device, an instruction for controlling the mobile robot to adjust a moving track, and the like. The remote control device may issue the control instruction to the mobile robot to cause the mobile robot to perform the work task indicated by the execution instruction.
And S104, when the remote control equipment is in a terminal equipment control mode, controlling the terminal equipment in communication connection with the remote control equipment according to the control instruction.
When the remote control equipment is in a terminal equipment control mode, the remote control equipment can exchange information with the terminal equipment at the moment, namely, the remote control equipment can send a control instruction to the terminal equipment, so that the remote control equipment can control the terminal equipment in communication connection with the remote control equipment according to the control instruction.
When the remote control device is in the terminal device control mode, the control instructions may include instructions for instructing the terminal device to start a photographing function, start a screen capturing operation, adjust an interactive interface displayed by the touch display screen (for example, adjust the interactive interface in a sliding manner, etc.), start editing an image, etc. The remote control device may send the control instruction to the terminal device to cause the terminal device to execute the work task indicated by the execution instruction.
Referring to fig. 4, fig. 4 is a flowchart of another control method according to an embodiment of the present invention, which may be performed by a remote control device, wherein a specific explanation of the remote control device is as described above. As shown in fig. 4, the control method may include the following steps on the basis of the foregoing embodiment.
S201, detecting control operation of a user through a first interaction device.
S202, generating a control instruction according to the control operation detected by the first interaction device.
S203, acquiring mode indication information. When the mode indication information is mobile robot control mode indication information, executing steps S204 to S205; when the mode indication information is terminal equipment control mode indication information, steps S206 to S207 are performed.
In one embodiment, the mode indication information may include connection state information between the remote control device and the mobile robot, and when the connection state information is normal connection state information, the remote control device is controlled to enter a mobile robot control mode; and when the connection state information is disconnection state information, controlling the remote control device to enter a terminal device control mode.
The remote control device may acquire connection state information between the remote control device and the mobile robot, wherein the connection state information may be used to indicate a network connection state between the remote control device and the mobile robot. When the connection state information is normal connection state information, the remote control device and the mobile robot are indicated to be in a connection state, the remote control device can conduct normal data interaction with the mobile robot, and the processor of the remote control device can control the remote control device to enter a mobile robot control mode. When the connection state information is disconnection state information, the processor of the remote control device can control the remote control device to enter a terminal device control mode, wherein the connection state information indicates that data interaction between the remote control device and the mobile robot cannot be performed due to network disconnection.
In one embodiment, the mode indication information is sent by the terminal device to the remote control device, and specifically, the remote control device may acquire the mode indication information sent by the terminal device, where the mode indication information is determined by the terminal device by detecting a control mode indication operation of the user.
The user can control the remote control device to enter a mobile robot control mode or a terminal device control mode in a mode that the terminal device performs control mode indication operation. Specifically, the control mode indication operation of the user includes a terminal device control mode indication operation and a mobile robot control mode indication operation, and when the interaction device of the terminal device detects the terminal device control mode indication operation of the user, the terminal device may generate terminal device control mode indication information; when the interaction means of the terminal device detects the mobile robot control mode indication operation of the user, the terminal device may generate mobile robot control mode indication information. The terminal device may send the generated mode indication information to the remote control device, and the processor of the remote control device may control the remote control device to perform a mobile robot control mode or a terminal device control mode indicated by the mode indication information. The interaction device of the terminal equipment can comprise one or more of a touch display screen, a rocker, a keyboard, keys and a wave wheel.
For example, the control mode indication operation includes detecting a pressing operation of a key by a user, for example, the interaction device of the terminal device includes a mobile robot control mode key and a terminal device control mode key, and when the terminal device detects that the user presses the mobile robot control mode key, the mobile robot control mode indication information is sent to the remote control device, and the remote control device receives the mobile robot control mode indication information and enters a mobile robot control mode; when the terminal equipment detects that the user presses a terminal equipment control mode key, terminal equipment control mode indication information is sent to the remote control equipment, and the remote control equipment receives the terminal equipment control mode indication information and enters a terminal equipment control mode.
For another example, the control mode indication operation includes receiving a voice control instruction of a user, for example, the user may send the control mode voice instruction to the terminal in a voice manner, the terminal device determines mode indication information according to the received control mode voice instruction, and sends the mode indication information to the remote control device, and the remote control device receives the mode indication information.
In one embodiment, the remote control device further comprises a second interaction means, the remote control device detects a control mode indication operation of the user through the second interaction means, and obtains the mode indication information according to the control mode indication operation detected by the second interaction means. In particular, the remote control device may include a second interaction means for setting a control mode, and the user may perform a control mode indication operation on the second interaction means to control whether the remote control device enters the mobile robot control mode or the terminal device control mode. The second interaction means detects a control mode indication operation of the user, the control mode indication operation including a mobile robot control mode indication operation and a terminal device control mode indication operation. The processor of the remote control device may obtain the mode indication information according to the control mode indication operation detected by the second interaction device, that is, the processor may obtain the mobile robot control mode indication information according to the mobile robot control mode indication operation detected by the second interaction device, and the processor may obtain the terminal device control mode indication information according to the terminal device control mode indication operation detected by the second interaction device.
In one embodiment, after step S203, further includes: and outputting the mode indication information.
In order to let the user know in which control mode the current control device is in, i.e. in order to let the user know whether the current control device is in the mobile robot control mode or the terminal device control mode. When the remote control device acquires the mode indication information, the mode indication information may be output. Wherein, outputting the mode indication information may include the following possible implementations:
one possible implementation: and controlling the display equipment of the remote control equipment to display prompt information according to the mode indication information. Specifically, the remote control device may include a display device, such as a display screen, an indicator light, or the like, and the processor of the remote control device may control the display device to display the prompt information corresponding to the mode indication information according to the acquired mode indication information. For example, the remote control device may include an indicator lamp, and when the mode indication information acquired by the processor of the remote control device is the mobile robot control mode indication information, the control indicator lamp emits green light, and when the mode indication information acquired by the processor of the remote control device is the terminal device control mode indication information, the control indicator lamp emits red light.
Another possible implementation is: and sending the mode indication information to the terminal equipment so that the terminal equipment displays prompt information according to the mode indication information. Specifically, the remote control device may transmit the mode indication information to the terminal device, and the terminal device may include a display device, such as a touch display screen, an indicator light, etc., and the processor of the terminal device may control the display device to display the prompt information corresponding to the mode indication information according to the received mode indication information. For example, the terminal device may include a touch display screen, and when the processor of the terminal device acquires that the mode indication information transmitted by the remote control device is the mobile robot control mode indication information, the touch display screen is controlled to display the "mobile robot control mode", and when the processor of the terminal device acquires that the mode indication information transmitted by the remote control device is the terminal device control mode indication information, the touch display screen is controlled to display the "terminal device control mode".
And S204, when the mode indication information is the mobile robot control mode indication information, controlling the remote control equipment to enter a mobile robot control mode.
S205, controlling the mobile robot in communication connection with the remote control equipment according to the control instruction.
In steps S204 to S205, when the mode indication information is mobile robot control mode indication information, it indicates that control of the mobile robot is required at this time, and therefore, the remote control device may control the remote control device to enter a mobile robot control mode, and control the mobile robot communicatively connected to the remote control device according to the control instruction.
In one embodiment, first task setting information sent by a terminal device is received, where the first task setting information is determined by the terminal device detecting a first task setting operation of a user, and step S205 includes: and controlling the mobile robot in communication connection with the remote control device according to the control instruction so as to enable the mobile robot to execute the work task indicated by the first work task setting information.
The user can set the corresponding work task for the control instruction generated by the remote control device in the mobile robot control mode by operating the terminal device, that is, set the control instruction to instruct the mobile robot to perform which work task, by this way, the first interaction device user can be set to instruct the mobile robot to perform which work task. Specifically, the user may perform a first task setting operation on the terminal device, for example, a control instruction generated by the user operating the terminal device instructs the mobile robot to perform a photographed task or a task for adjusting the posture of the pan-tilt, the terminal device may detect the first task setting operation of the user to generate first task setting information, and send the first task setting information to the remote control device, and the remote control device sets a task corresponding to the control instruction according to the first task setting information, and sets the control instruction to instruct the mobile robot to perform the task. When the remote control device is in a mobile robot control mode, the remote control device can control and send the generated control instruction to a mobile robot in communication connection with the remote control device, and the mobile robot executes a work task indicated by the first work task setting information according to the control instruction, for example, the mobile robot executes a shot work task indicated by the first work task setting information or a work task for adjusting the posture of the cradle head according to the control instruction.
S206, when the mode indication information is terminal equipment control mode indication information, controlling the remote control equipment to enter a terminal equipment control mode.
S207, controlling the terminal equipment in communication connection with the remote control equipment according to the control instruction.
In steps S206 to S207, when the mode indication information is terminal equipment control mode indication information, it indicates that control of the terminal equipment is required at this time, so that the remote control equipment is controlled to enter a terminal equipment control mode, and the terminal equipment communicatively connected to the remote control equipment is controlled according to the control instruction.
In one embodiment, second work task setting information sent by the terminal device is received, wherein the second work task setting information is determined by the terminal device detecting a second work task setting operation of a user; step S207 includes: and controlling the terminal equipment in communication connection with the remote control equipment according to the control instruction so as to enable the terminal equipment to execute the work task indicated by the second work task setting information.
The user can set the corresponding work task for the control instruction generated by the remote control device in the control mode of the terminal device by operating the terminal device, that is, set the control instruction to instruct the terminal device to execute which work task, by this way, the first interaction device user can be set to instruct the terminal device to execute which work task. Specifically, the user may perform a second task setting operation on the terminal device, for example, a control instruction generated by the user operating the setting on the terminal device instructs the terminal device to perform an editing task for starting an image or a task for adjusting an interactive interface displayed on the touch display screen, the terminal device may detect the second task setting operation of the user to generate second task setting information, and send the second task setting information to the remote control device, where the remote control device sets a task corresponding to the control instruction according to the second task setting information, that is, sets the task that the control instruction instructs the terminal device to perform. When the remote control device is in a terminal device control mode, the remote control device can control and send the generated control instruction to the terminal device in communication connection with the remote control device, and the terminal device executes the work task indicated by the second work task setting information according to the control instruction, for example, the terminal device executes the work task indicated by the second work task setting information for starting editing of the image or the work task for adjusting the interactive interface displayed by the touch display screen according to the control instruction.
In one embodiment, the remote control device may comprise a plurality of interaction means, one or more of the plurality of interaction means may be arranged as a first interaction means, i.e. the target interaction means of the plurality of interaction means may be arranged as a first interaction means to enable control of the mobile robot or the terminal device in different control modes. Specifically, referring to fig. 5, fig. 5 is a schematic diagram illustrating another control method according to an embodiment of the present invention, where the method includes:
s301, receiving interactive device selection information sent by the terminal equipment, wherein the interactive device selection information is determined by the terminal equipment detecting interactive device selection operation of a user.
The user may select one or more of the plurality of interaction means as the first interaction means by operating the terminal device. In particular, the remote control device may comprise a plurality of interaction means, e.g. the remote control device may comprise a joystick, a pulsator and keys. The user may perform an interactive device selection operation on the terminal apparatus, for example, perform an operation of selecting a rocker of the plurality of interactive devices, and the terminal apparatus may determine interactive device selection information according to the detected operation, wherein the interactive device selection information may include an identification number of the selected interactive device, for example, an identification number of the rocker, and the terminal apparatus may transmit the interactive device selection information to the remote control apparatus.
S302, determining one interactive device in a plurality of interactive devices as a first interactive device according to the interactive device selection information.
The remote control apparatus may determine, after receiving the interactive device selection information, which of the plurality of interactive devices the interactive device information indicates, and determine one of the interactive devices indicated by the interactive device selection information as the first interactive device. For example, the interaction device selection information includes an identification number of the rocker, and the rocker may be determined as the first interaction device according to the identification number.
S303, detecting control operation of a user through the first interaction device.
S304, generating a control instruction according to the control operation detected by the first interaction device.
S305, when the remote control equipment is in a mobile robot control mode, controlling the mobile robot in communication connection with the remote control equipment according to the control instruction.
When the control command is determined to be normal connection state information only from definition or connection state information, the remote control equipment is determined to be in a mobile robot control mode, and the mobile robot in communication connection with the remote control equipment is controlled according to the control command.
S306, when the remote control equipment is in a terminal equipment control mode, controlling the terminal equipment in communication connection with the remote control equipment according to the control instruction.
Based on the above description of a control method and system, an embodiment of the present invention provides a remote control device, as shown in fig. 6, where the remote control device includes a first interaction device 601, a processor 602, a second interaction device 603, an output device 604 and a communication device 605,
the first interaction device 601 is configured to detect a control operation of a user;
the processor 602 is configured to:
generating a control instruction according to the control operation detected by the first interaction device;
when the remote control equipment is in a mobile robot control mode, controlling a mobile robot in communication connection with the remote control equipment according to the control instruction;
when the remote control equipment is in a terminal equipment control mode, controlling terminal equipment in communication connection with the remote control equipment according to the control instruction.
In some embodiments, the processor 602 is specifically configured to, when controlling the mobile robot communicatively connected to the remote control device according to the control instruction:
the control communication device sends a control command to the mobile robot in communication connection with the remote control device so as to control the mobile robot according to the control command.
In some embodiments, when the processor 602 controls the terminal device communicatively connected to the remote control device according to the control instruction, the processor is specifically configured to:
The control communication device sends a control instruction to the terminal equipment in communication connection with the remote control equipment so as to control the terminal equipment according to the control instruction.
In some embodiments, the terminal device and the remote control device are housed in the same housing.
In some embodiments, the terminal device and the remote control device are physically separate devices.
In some embodiments, the terminal device location is based on a device of the android system, linux or IOS system.
In some embodiments, the processor 602 is further configured to:
acquiring mode indication information;
when the mode indication information is mobile robot control mode indication information, controlling the remote control equipment to enter a mobile robot control mode;
and when the mode indication information is terminal equipment control mode indication information, controlling the remote control equipment to enter a terminal equipment control mode.
In some embodiments, the mode indication information includes connection status information between the remote control device and the mobile robot,
the processor 602 is specifically configured to, when the mode indication information is mobile robot control mode indication information, control the remote control device to enter a mobile robot control mode:
When the connection state information is normal connection state information, controlling the remote control equipment to enter a mobile robot control mode;
the processor 602 is specifically configured to, when the mode indication information is terminal equipment control mode indication information, control the remote control equipment to enter a terminal equipment control mode:
and when the connection state information is disconnection state information, controlling the remote control equipment to enter a terminal equipment control mode.
In some embodiments, when the processor 602 obtains the mode indication information, the method specifically includes:
and acquiring mode indication information sent by the terminal equipment, wherein the mode indication information is determined by the terminal equipment through detecting control mode indication operation of a user.
In some embodiments, the second interaction device 603 is configured to detect a control mode indication operation of the user;
the processor 602 is further configured to: and acquiring mode indication information according to the control mode indication operation detected by the second interaction device.
In some embodiments, the output device 604 is configured to output the mode indication information.
In some embodiments, the apparatus comprises a plurality of interaction means, the processor 602 further configured to:
Receiving interactive device selection information sent by terminal equipment, wherein the interactive device selection information is determined by the terminal equipment detecting interactive device selection operation of a user;
and determining one interactive device in the plurality of interactive devices as a first interactive device according to the interactive device selection information.
In some embodiments, the processor 602 is further configured to:
receiving first work task setting information sent by terminal equipment, wherein the first work task setting information is determined by the terminal equipment by detecting first work task setting operation of a user;
the processor 602 is specifically configured to, when controlling the mobile robot communicatively connected to the remote control device according to the control instruction:
and controlling the mobile robot in communication connection with the remote control equipment according to the control instruction so as to enable the mobile robot to execute the work task indicated by the first work task setting information.
In some embodiments, the processor 602 is further configured to:
receiving second work task setting information sent by the terminal equipment, wherein the second work task setting information is determined by the terminal equipment by detecting second work task setting operation of a user;
the processor 602 is specifically configured to, when controlling the terminal device communicatively connected to the remote control device according to the control instruction:
And controlling the terminal equipment in communication connection with the remote control equipment according to the control instruction so as to enable the terminal equipment to execute the work task indicated by the second work task setting information.
It can be appreciated that the remote control device provided in the embodiment of the present invention may implement the method as described above, where specific principles and explanations refer to the foregoing parts, and are not repeated herein.
There is also provided in an embodiment of the present invention a computer readable storage medium storing a computer program to be executed by a processor to implement a method as described in any of fig. 3 to 5, which is not described in detail herein.
The computer readable storage medium may be an internal storage unit of the device according to any of the foregoing embodiments, for example, a hard disk or a memory of the device. The computer readable storage medium may also be an external storage device of the device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the device. Further, the computer readable storage medium may also include both internal storage units and external storage devices of the device. The computer-readable storage medium is used to store the computer program and other programs and data required by the terminal. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.
Those skilled in the art will appreciate that implementing all or part of the above-described methods in accordance with the embodiments may be accomplished by way of a computer program stored on a computer readable storage medium, which when executed may comprise the steps of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), or the like.
The foregoing disclosure is illustrative of the present invention and is not to be construed as limiting the scope of the invention, which is defined by the appended claims.

Claims (26)

1. A remote control apparatus of a mobile robot, comprising: a first interaction means and a processor, wherein,
the first interaction device is used for detecting control operation of a user;
the processor is configured to:
generating a control instruction according to the control operation detected by the first interaction device;
when the remote control equipment is in a mobile robot control mode, controlling a mobile robot in communication connection with the remote control equipment according to the control instruction;
When the remote control equipment is in a terminal equipment control mode, controlling terminal equipment in communication connection with the remote control equipment according to the control instruction.
2. The device of claim 1, wherein the terminal device and the remote control device are housed in the same housing.
3. The device of claim 1, wherein the terminal device and the remote control device are physically separate devices.
4. A device according to any of claims 1-3, characterized in that the terminal device is an android system, linux or mobile operating system IOS system based device.
5. The apparatus according to any one of claims 1 to 4, wherein,
the processor is further configured to:
acquiring mode indication information;
when the mode indication information is mobile robot control mode indication information, controlling the remote control equipment to enter a mobile robot control mode;
and when the mode indication information is terminal equipment control mode indication information, controlling the remote control equipment to enter a terminal equipment control mode.
6. The apparatus of claim 5, wherein the mode indication information includes connection state information between the remote control apparatus and a mobile robot,
The processor is specifically configured to, when the mode indication information is mobile robot control mode indication information, control the remote control device to enter a mobile robot control mode:
when the connection state information is normal connection state information, controlling the remote control equipment to enter a mobile robot control mode;
the processor is specifically configured to, when the mode indication information is terminal equipment control mode indication information and the remote control equipment is controlled to enter a terminal equipment control mode:
and when the connection state information is disconnection state information, controlling the remote control equipment to enter a terminal equipment control mode.
7. The apparatus according to claim 5 or 6, wherein,
when the processor acquires the mode indication information, the processor is specifically configured to:
and acquiring mode indication information sent by the terminal equipment, wherein the mode indication information is determined by the terminal equipment through detecting control mode indication operation of a user.
8. The device of any of claims 5-7, wherein the remote control device further comprises: a second interaction means, wherein,
the second interaction device is used for detecting control mode indication operation of a user;
The processor is further configured to:
and acquiring mode indication information according to the control mode indication operation detected by the second interaction device.
9. The apparatus according to any one of claims 5 to 8, further comprising: an output device, wherein,
the output device is used for outputting the mode indication information.
10. The device according to any of the claims 1-9, characterized in that the device comprises a plurality of interaction means,
the processor is further configured to:
receiving interactive device selection information sent by terminal equipment, wherein the interactive device selection information is determined by the terminal equipment detecting interactive device selection operation of a user;
and determining one interactive device in the plurality of interactive devices as a first interactive device according to the interactive device selection information.
11. The apparatus according to any one of claims 1 to 10, wherein,
the processor is further configured to:
receiving first work task setting information sent by terminal equipment, wherein the first work task setting information is determined by the terminal equipment by detecting first work task setting operation of a user;
the processor is specifically configured to, when controlling the mobile robot in communication connection with the remote control device according to the control instruction:
And controlling the mobile robot in communication connection with the remote control equipment according to the control instruction so as to enable the mobile robot to execute the work task indicated by the first work task setting information.
12. The apparatus according to any one of claims 1 to 11, wherein,
the processor is further configured to:
receiving second work task setting information sent by the terminal equipment, wherein the second work task setting information is determined by the terminal equipment by detecting second work task setting operation of a user;
the processor is specifically configured to, when controlling the terminal device communicatively connected to the remote control device according to the control instruction:
and controlling the terminal equipment in communication connection with the remote control equipment according to the control instruction so as to enable the terminal equipment to execute the work task indicated by the second work task setting information.
13. A control method, characterized in that it is applied to a remote control device of a mobile robot, said remote control device comprising first interaction means, characterized in that it comprises:
detecting a control operation of a user through a first interaction device;
generating a control instruction according to the control operation detected by the first interaction device;
when the remote control equipment is in a mobile robot control mode, controlling a mobile robot in communication connection with the remote control equipment according to the control instruction;
When the remote control equipment is in a terminal equipment control mode, controlling terminal equipment in communication connection with the remote control equipment according to the control instruction.
14. The method of claim 13, wherein the terminal device and the remote control device are housed in the same housing.
15. The method of claim 13, wherein the terminal device and the remote control device are physically separate devices.
16. The method according to any of claims 13-15, wherein the terminal device is an android system, linux or IOS system based device.
17. The method according to any one of claims 13-16, further comprising:
acquiring mode indication information;
when the mode indication information is mobile robot control mode indication information, controlling the remote control equipment to enter a mobile robot control mode;
and when the mode indication information is terminal equipment control mode indication information, controlling the remote control equipment to enter a terminal equipment control mode.
18. The method of claim 17, wherein the mode indication information includes connection status information between the remote control device and the mobile robot,
When the mode indication information is mobile robot control mode indication information, controlling the remote control device to enter a mobile robot control mode, including:
when the connection state information is normal connection state information, controlling the remote control equipment to enter a mobile robot control mode;
and when the mode indication information is terminal equipment control mode indication information, controlling the remote control equipment to enter a terminal equipment control mode, wherein the method comprises the following steps:
and when the connection state information is disconnection state information, controlling the remote control equipment to enter a terminal equipment control mode.
19. The method according to claim 17 or 18, wherein the acquisition mode indication information comprises:
and acquiring mode indication information sent by the terminal equipment, wherein the mode indication information is determined by the terminal equipment through detecting control mode indication operation of a user.
20. The method according to any of the claims 17-19, wherein the remote control device further comprises a second interaction means, the method further comprising:
detecting a control mode indication operation of a user through the second interaction device;
and acquiring mode indication information according to the control mode indication operation detected by the second interaction device.
21. The method according to any one of claims 17-20, further comprising:
and outputting the mode indication information.
22. The method of any of claims 13-21, wherein the apparatus comprises a plurality of interaction devices, the method further comprising:
receiving interactive device selection information sent by terminal equipment, wherein the interactive device selection information is determined by the terminal equipment detecting interactive device selection operation of a user;
and determining one interactive device in the plurality of interactive devices as a first interactive device according to the interactive device selection information.
23. The method according to any one of claims 13-22, further comprising:
receiving first work task setting information sent by terminal equipment, wherein the first work task setting information is determined by the terminal equipment by detecting first work task setting operation of a user;
the mobile robot which is in communication connection with the remote control equipment is controlled according to the control instruction, and the mobile robot comprises:
and controlling the mobile robot in communication connection with the remote control equipment according to the control instruction so as to enable the mobile robot to execute the work task indicated by the first work task setting information.
24. The method according to any one of claims 13-23, further comprising:
receiving second work task setting information sent by the terminal equipment, wherein the second work task setting information is determined by the terminal equipment by detecting second work task setting operation of a user;
the terminal equipment which is in communication connection with the remote control equipment is controlled according to the control instruction, and the terminal equipment comprises:
and controlling the terminal equipment in communication connection with the remote control equipment according to the control instruction so as to enable the terminal equipment to execute the work task indicated by the second work task setting information.
25. A mobile robotic system, comprising: the remote control device, terminal device, and mobile robot according to any one of claim 1 to 12,
the terminal equipment at least comprises a touch display screen.
26. A computer readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the control method according to any one of claims 13 to 24.
CN201880036977.5A 2018-07-30 2018-07-30 Control method, control equipment, control system and storage medium Active CN111065982B (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/097729 WO2020024092A1 (en) 2018-07-30 2018-07-30 Control method, device, system, and storage medium

Publications (2)

Publication Number Publication Date
CN111065982A CN111065982A (en) 2020-04-24
CN111065982B true CN111065982B (en) 2023-07-14

Family

ID=69230493

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201880036977.5A Active CN111065982B (en) 2018-07-30 2018-07-30 Control method, control equipment, control system and storage medium

Country Status (2)

Country Link
CN (1) CN111065982B (en)
WO (1) WO2020024092A1 (en)

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4814803B2 (en) * 2007-01-12 2011-11-16 富士通株式会社 Bidirectional control device using remote control signal by computer and home appliance
KR20110003963A (en) * 2009-07-07 2011-01-13 삼성전자주식회사 Multi-functional remote controller for handheld mobile terminal device
CN103853348A (en) * 2012-11-29 2014-06-11 华为技术有限公司 Operation method, operation control method and related device of air input device
CN103543752B (en) * 2013-10-09 2017-03-15 深圳市大疆创新科技有限公司 A kind of remote control thereof and remote control systems
WO2017096546A1 (en) * 2015-12-09 2017-06-15 SZ DJI Technology Co., Ltd. Imaging system and method for unmanned vehicles
CN108236376A (en) * 2016-12-27 2018-07-03 徐州山川净水设备有限公司 Water dispenser based on wireless control
CN108255161A (en) * 2016-12-28 2018-07-06 昊翔电能运动科技(昆山)有限公司 Aircraft control device and telecontrolled aircraft
CN207615924U (en) * 2017-09-19 2018-07-17 泉州信息工程学院 A kind of all position welding connection device of PLC controls

Also Published As

Publication number Publication date
CN111065982A (en) 2020-04-24
WO2020024092A1 (en) 2020-02-06

Similar Documents

Publication Publication Date Title
US10569874B2 (en) Flight control method and apparatus
CN105830436B (en) Information processing apparatus, information processing method, and program
US20190092345A1 (en) Driving method, vehicle-mounted driving control terminal, remote driving terminal, and storage medium
JP5523233B2 (en) Communication terminal and data transmission method thereof
US20190317490A1 (en) Control method, device, and remote control for vr apparatus
US20150109437A1 (en) Method for controlling surveillance camera and system thereof
US10863072B2 (en) Image processing system, method, apparatus and device of processing image data
CN109151392B (en) Shooting monitoring equipment and cradle head system comprising same
US9870118B2 (en) Non-transitory storage medium encoded with computer readable information processing program, information processing apparatus, method of controlling information processing apparatus, and information processing system, capable of controlling virtual camera while grasping overall condition of virtual camera arranged in virtual space
US9661229B2 (en) Method for capturing a picture in a portable terminal by outputting a notification of an object being in a capturing position
EP3989118A1 (en) Target tracking method and system, readable storage medium and moving platform
KR101305944B1 (en) A method for remote controlling robot using wrap around image and an apparatus thereof
WO2020154959A1 (en) Multi-load image transmission method, control system, control terminal, unmanned aerial vehicle, and server
US20170269590A1 (en) Controlled device and communication system and method utilizing the same
CN110313174B (en) Shooting control method and device, control equipment and shooting equipment
JP6503734B2 (en) INFORMATION PROCESSING APPARATUS, CONTROL METHOD, AND PROGRAM
CN111343425A (en) Virtual reality live broadcast control method based on unmanned aerial vehicle
KR101553503B1 (en) Method for controlling a external device using object recognition
US20160182860A1 (en) Methods for performing image capture and real-time image display on physically separated or separable devices and apparatus therefor
CN111065982B (en) Control method, control equipment, control system and storage medium
CN112204943B (en) Photographing method, apparatus, system, and computer-readable storage medium
CN108496351B (en) Unmanned aerial vehicle and control method thereof, control terminal and control method thereof
KR102617252B1 (en) Electronic Device and the Method for Automatically Switching to Panorama Capture Mode thereof
CN112219176A (en) Remote control movable platform control method, device and computer readable storage medium
JP2016225875A (en) Travel apparatus operation terminal, travel apparatus operation method and travel apparatus operation program

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant