CN111065982A - Control method, device, system and storage medium - Google Patents

Control method, device, system and storage medium Download PDF

Info

Publication number
CN111065982A
CN111065982A CN201880036977.5A CN201880036977A CN111065982A CN 111065982 A CN111065982 A CN 111065982A CN 201880036977 A CN201880036977 A CN 201880036977A CN 111065982 A CN111065982 A CN 111065982A
Authority
CN
China
Prior art keywords
control
remote control
mobile robot
terminal
equipment
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201880036977.5A
Other languages
Chinese (zh)
Inventor
邹旭
杨小虎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
SZ DJI Technology Co Ltd
Original Assignee
SZ DJI Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by SZ DJI Technology Co Ltd filed Critical SZ DJI Technology Co Ltd
Priority to PCT/CN2018/097729 priority Critical patent/WO2020024092A1/en
Publication of CN111065982A publication Critical patent/CN111065982A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/02Control of position or course in two dimensions
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05DSYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00Control of position, course or altitude of land, water, air, or space vehicles, e.g. automatic pilot
    • G05D1/10Simultaneous control of position or course in three dimensions

Abstract

A remote control device of a mobile robot, comprising: the system comprises a first interaction device and a processor, wherein the first interaction device is used for detecting control operation of a user, the processor generates a control instruction according to the control operation detected by the first interaction device, when the remote control equipment is in a mobile robot control mode, the mobile robot in communication connection with the control equipment is controlled according to the control instruction, and when the remote control equipment is in a terminal equipment control mode, the terminal equipment in communication connection with the remote control equipment is controlled according to the control instruction. The terminal equipment and the mobile robot are controlled through the interaction device of the remote control equipment, and a user does not need to switch. A control method for the remote control device, a mobile robot system including the remote control device, and a computer-readable storage medium performing the control method are also included.

Description

Control method, device, system and storage medium Technical Field
The present invention relates to the field of control technologies, and in particular, to a control method, device, system, and storage medium.
Background
With the development of control technology, the mobile robot becomes a popular research topic at present, and is widely applied to the fields of plant protection, shooting, forest fire alarm monitoring, security monitoring and the like. Currently, a user can control the movement of the mobile robot through a remote control device and view sensor data (e.g., image data collected by the mobile robot) collected by the mobile robot through a terminal device communicatively connected to the remote control device.
However, when a user wants to control the mobile robot and the terminal device, the user needs to operate the interaction device of the remote control device and the interaction device of the terminal device respectively, and this operation mode frequently switches the interaction devices to realize the respective control of the mobile robot and the terminal device, which is tedious in operation mode and poor in interaction friendliness.
Disclosure of Invention
The embodiment of the invention provides a control method, equipment, a system and a storage medium, which can improve the convenience of controlling a mobile robot and terminal equipment.
In a first aspect, an embodiment of the present invention provides a remote control device for a mobile robot, including: a first interaction device and a processor, wherein,
the first interaction device is used for detecting the control operation of a user;
the processor is configured to:
generating a control instruction according to the control operation detected by the first interaction device;
when the remote control equipment is in a mobile robot control mode, controlling a mobile robot in communication connection with the remote control equipment according to the control instruction;
and when the remote control equipment is in a terminal equipment control mode, controlling the terminal equipment in communication connection with the remote control equipment according to the control instruction.
In a second aspect, an embodiment of the present invention provides a control method, which is applied to a remote control device of a mobile robot, where the remote control device includes a first interaction apparatus, and the method includes:
detecting a control operation of a user through a first interaction device;
generating a control instruction according to the control operation detected by the first interaction device;
when the remote control equipment is in a mobile robot control mode, controlling a mobile robot in communication connection with the remote control equipment according to the control instruction;
and when the remote control equipment is in a terminal equipment control mode, controlling the terminal equipment in communication connection with the remote control equipment according to the control instruction.
In a third aspect, an embodiment of the present invention provides a mobile robot system, including: the remote control device, the terminal device and the mobile robot according to the first aspect,
the terminal equipment at least comprises a touch display screen.
In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the computer program implements the control method according to the second aspect.
In the embodiment of the invention, the remote control equipment can detect the control operation of a user; generating a control instruction according to the detected control operation; when the remote control equipment is in a mobile robot control mode, controlling a mobile robot in communication connection with the remote control equipment according to the control instruction; and when the remote control equipment is in a terminal equipment control mode, controlling the terminal equipment in communication connection with the remote control equipment according to the control instruction. By the method, the terminal equipment and the mobile robot can be controlled through the interaction device of the remote control equipment, and a user does not need to switch the interaction device, so that a better man-machine interaction mode is provided for the user.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
Fig. 1 is a schematic structural diagram of a mobile robot system according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of another mobile robotic system according to an embodiment of the present invention;
FIG. 3 is a flow chart of a control method according to an embodiment of the present invention;
FIG. 4 is a schematic flow chart of another control method provided by the embodiment of the invention;
FIG. 5 is a schematic flow chart illustrating another control method according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a remote control device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Some embodiments of the invention are described in detail below with reference to the accompanying drawings. The embodiments described below and the features of the embodiments can be combined with each other without conflict.
Currently, a remote control system of a mobile robot includes a remote control device and a terminal device, wherein the remote control device may include a remote controller, and the remote controller is generally used for controlling a moving state (such as a position, a velocity, an acceleration, or the like) of the mobile robot and/or a pan-tilt of the mobile robot and/or a photographing action of the mobile robot. The terminal device may be a device configured with a touch display screen, and the terminal device is generally used to display or edit an image captured by a camera of the mobile robot, to initiate an intelligent control mode (e.g., a tracking mode, a pointing movement mode), and the like. In the prior art, a remote control device and a terminal device are generally operated independently of each other, and the remote control device sends a command generated by the remote control device and/or a command of the terminal device to a mobile robot through a communication device (such as an image transmission module) configured by the remote control device and/or receives an image acquired by a shooting device of the mobile robot.
When a user wants to control the mobile robot and the terminal device, the user needs to operate the interaction device of the remote control device and the interaction device of the terminal device respectively, the interaction devices are frequently switched in the operation mode to realize the respective control of the mobile robot and the terminal device, the operation mode is complicated, and the interaction friendliness is poor. For example, in a specific application scenario, when a user is in an outdoor cold scene, the user carries gloves to operate the interactive operation of the remote control device to control the mobile robot, however, when the user thinks to operate the touch display screen of the terminal device, the user also needs to take off the gloves to facilitate the finger touch operation on the touch display screen, which results in inconvenient operation. In order to solve the problem, an embodiment of the present invention provides a control method, where the control method may be applied to a remote control device, where the remote control device may include a first interaction device, where the first interaction device is any device that interacts with a user, where the first interaction device may specifically include one or more of a keyboard, a key, a joystick, and a pulsator, and in the control method, the remote control device may detect a control operation of the user through the first interaction device; generating a control instruction according to the control operation detected by the first interaction device; when the remote control equipment is in a mobile robot control mode, controlling a mobile robot in communication connection with the remote control equipment according to the control instruction; and when the remote control equipment is in a terminal equipment control mode, controlling the terminal equipment in communication connection with the remote control equipment according to the control instruction. Therefore, the remote control equipment can respectively control the terminal equipment and the mobile robot in different control modes, specifically, the mobile robot and the terminal equipment can be controlled through the first interaction device of the remote control equipment, the convenience of controlling the terminal equipment and the mobile robot is improved, the control modes of the remote control equipment are enriched, the use value of the remote control equipment is improved, the interaction devices do not need to be frequently switched, and a better man-machine interaction mode is provided for a user.
In addition, when the remote control device is in the terminal device control mode, when the terminal device is required to execute a preset work task, for example, when a target application program (such as a photographing application program) is started, a user needs to operate the first interaction device of the remote control device, and the user does not need to switch to the operation of the interaction device of the terminal device, for example, the target application program is started through finger touch operation, so that the convenience of man-machine interaction is improved.
In order to facilitate understanding of the control method described in the present application, an embodiment of the present application further provides a mobile robot system, where the mobile robot system includes the control system of the mobile robot and the mobile robot as described above, and the remote control system of the mobile robot includes the remote control device and the terminal device as described above. The mobile robot may be an unmanned aerial vehicle, an unmanned ground robot, an unmanned ship, etc., wherein the unmanned aerial vehicle may be a rotary wing type mobile robot, such as a four-rotor mobile robot, a six-rotor mobile robot, an eight-rotor mobile robot. The mobile robot comprises a power system, the power system is used for providing mobile power for the mobile robot, the power system comprises one or more of an engine, a propeller, a motor and an electric regulator, the mobile robot further comprises a holder and a shooting device, and the shooting device is carried on a main body of the mobile robot through the holder. The shooting device is used for shooting images or videos in the moving process of the mobile robot and comprises but is not limited to a multispectral imager, a hyperspectral imager, a visible light camera, an infrared camera and the like, the cradle head is a multi-axis transmission and stability augmentation system, the cradle head motor compensates the shooting angle of the imaging equipment by adjusting the rotating angle of the rotating shaft, and the shaking of the imaging equipment is prevented or reduced by arranging a proper buffer mechanism.
In an embodiment of the present invention, the remote control device may establish a wireless communication connection with the mobile robot, for example, the remote control device includes a communication device, and the remote control device may establish a wireless communication connection with the mobile robot through the communication device, where the communication device may include a mapping module. Through the wireless communication connection established with the mobile robot, the first interaction device of the remote control equipment can control the moving state (such as position, speed or acceleration) of the mobile robot and/or the pan-tilt control of the mobile robot and/or the shooting action of the mobile robot. Wherein the wireless communication connection may be based on WIFI, SDR, bluetooth or other proprietary protocols.
In the embodiment of the present invention, the remote control device may further establish a wired communication connection or a wireless communication connection with the terminal device, where the wireless communication connection may be based on WIFI, SDR, bluetooth or other proprietary protocols, and the wired communication connection may be based on a USB cable. Through the wired or wireless communication connection established with the terminal device, the first interaction device of the remote control device can control the terminal device, for example, control the terminal device to start one or more of a photographing function, control the terminal device to start a screen capture operation, control an interaction interface displayed by a touch display screen of the terminal device (for example, control the sliding of the interaction interface, etc.), and control the terminal device to start editing an image.
As mentioned above, the terminal device may be a device at least including a touch display screen, and in some embodiments, the terminal device may be specifically one or more of a smart phone, a tablet computer, a laptop computer, a ground station, and a wearable device (watch, bracelet). In some embodiments, the terminal device may be a customized device that includes a touch screen display. The operating system of the terminal device may refer to an android operating system, a Linux operating system, an IOS operating system, or the like.
The android operating system is a Linux-based free and open source code operating system, and the system supports the terminal device to establish communication connection with a USB-HID (human Interface device) -type remote control device by default. Therefore, in one embodiment, the communication connection established between the remote control device and the terminal device may refer to a communication connection established through a USB interface.
The Linux operating system supports a bluetooth protocol stack, and also supports the terminal device to establish a communication connection with the USB-HID remote control device on the bluetooth protocol stack, so in one embodiment, the communication connection established between the remote control device and the terminal device may be a communication connection established through bluetooth.
The USB-HID remote control device may refer to a device including a keyboard, a mouse, or a joystick.
In one embodiment, the terminal device and the remote control device may be physically separate devices, as shown in fig. 1, the mobile robot system includes the remote control device 11 and the terminal device 12, and the remote control device 11 and the terminal device 12 are physically separate devices. Specifically, the remote control device 11 and the terminal device 12 may be housed in different housings, for example, the remote control device 11 is housed in the housing 15, and the terminal device 12 is housed in the housing 14.
In one embodiment, the terminal device and the remote control device are housed in the same housing. As shown in fig. 2, the mobile robot system includes a remote control device 11, a terminal device 12, and a mobile robot 13, wherein the remote control device 11 and the terminal device 12 can be housed in the same housing 16.
Referring to fig. 3, fig. 3 is a flowchart illustrating a control method according to an embodiment of the present invention, where the method may be executed by a remote control device, and a specific explanation of the remote control device is as described above. As shown in fig. 3, the control method may include the following steps.
S101, detecting control operation of a user through a first interaction device.
In the implementation of the present invention, when a user needs a terminal device or a mobile robot to execute a certain work task, the user may perform a control operation on a first interaction device of a remote control device, for example, the user may input the control operation in at least one of a key, a rocker, or a dial wheel. The first interaction means may detect a control operation input by a user.
In one embodiment, when the user inputs the control operation by means of the key, the detecting the control operation of the user by the first interaction device includes: and detecting the control operation of the user through the keyboard.
In one embodiment, when the user inputs the control operation by means of a joystick or a thumbwheel, the detecting the control operation of the user by the first interaction device includes: the control operation of a user is realized through a rocker or a thumb wheel.
And S102, generating a control instruction according to the control operation detected by the first interaction device.
In the embodiment of the present invention, after the first interaction device detects the control operation of the user, the processor of the remote control apparatus may generate the control instruction according to the control operation detected by the first interaction device. Wherein the control operation of the user corresponds to the generated control instruction.
In one embodiment, when the user inputs the control operation by a key, the control instruction may be generated according to parameters such as the detected force for pressing the keyboard or the identification number of the pressed key in the keyboard. For example, the control command is generated according to the identification number of the pressed key. Specifically, the remote control device presets a correspondence between the identification number of the key and different control instructions, and the remote control device may generate the control instruction according to the identification number of the pressed key and the correspondence.
In another embodiment, when the user inputs the control operation by means of the rocker or the dial wheel, the remote control device may generate the control command according to the parameters such as the direction and/or the angle of the rocker or the dial wheel. For example, the control command is generated according to the rocking direction and/or angle of the rocker. Specifically, the remote control device presets a corresponding relationship between different directions and/or angles of the rocking lever and different control instructions, and the remote control device can generate the control instructions according to the corresponding relationship and the directions and/or angles of the rocking lever.
And S103, when the remote control equipment is in a mobile robot control mode, controlling the mobile robot in communication connection with the remote control equipment according to the control instruction.
In the embodiment of the invention, the remote control equipment can control the mobile robot and can also control the terminal equipment. When the remote control device is in different control modes, the remote control device may control different objects, i.e. the generated control instructions are used to control different objects, and the control mode of the remote control device may determine the object to be controlled. Specifically, when the remote control device is in the mobile robot control mode, it indicates that the remote control device can interact information with the mobile robot at this time, that is, the remote control device can send a control instruction to the mobile robot, and therefore the remote control device can control the mobile robot in communication connection with the remote control device according to the control instruction, so as to control the mobile robot by the remote control device.
In one embodiment, the control mode of the remote control device may be set by the remote control device according to the distance of the remote control device from the mobile robot. For example, when the distance between the remote control device and the mobile robot is greater than a preset distance threshold (e.g., 500m), the remote control device may set the control mode to the terminal device control mode, otherwise, the control mode is set to the mobile robot control mode, and the preset distance threshold may be set according to the distance at which the remote control device can control the mobile robot.
In one embodiment, the control mode of the remote control device may be set by the remote control device according to an application scenario. For example, in a scene where the mobile robot needs to be controlled (for example, when the moving track of the mobile robot needs to be controlled), the remote control device may set the control mode to the mobile robot control mode; in a scene that the terminal device needs to be controlled (for example, when an interface displayed on a touch display screen of the terminal device needs to be controlled slides), the remote control device may set the control mode to the terminal control mode.
In one embodiment, the control mode of the remote control device may be manually set by the user. For example, the remote control device may provide at least one control mode key, and when only one control mode key is provided, the remote control device detects one pressing operation for the control mode key within a preset time period (for example, 5s), and sets the control mode as the mobile robot control mode; and when the remote control device detects more than one pressing operation aiming at the control mode key within a preset time period (such as 5s), setting the control mode as the terminal device control mode. When the remote control equipment comprises two control mode keys, one is a mobile robot control mode key, and the other is a terminal equipment control mode key, if the pressing operation aiming at the mobile robot control mode key is detected, the control mode is set as a mobile robot control mode; and if the pressing operation aiming at the terminal equipment control mode key is detected, setting the control mode as the terminal equipment control mode.
When the remote control device is in the mobile robot control mode, the control instruction may include an instruction for controlling a camera of the mobile robot to perform photographing, an instruction for controlling the mobile robot to transmit image data to the terminal device or the remote control device, an instruction for controlling the mobile robot to adjust a movement track, and the like. The remote control device may issue the control instruction to the mobile robot to cause the mobile robot to perform the work task indicated by the execution instruction.
And S104, when the remote control equipment is in a terminal equipment control mode, controlling the terminal equipment in communication connection with the remote control equipment according to the control instruction.
When the remote control equipment is in the terminal equipment control mode, the remote control equipment can interact information with the terminal equipment at the moment, namely the remote control equipment can send a control instruction to the terminal equipment, so that the remote control equipment can control the terminal equipment in communication connection with the remote control equipment according to the control instruction.
When the remote control device is in the terminal device control mode, the control instruction may include an instruction instructing the terminal device to start a photographing function, start a screen capture operation, adjust an interactive interface displayed by the touch display screen (for example, adjust the interactive interface in a sliding manner, etc.), start editing an image, and the like. The remote control device can send the control instruction to the terminal device to enable the terminal device to execute the work task indicated by the execution instruction.
Referring to fig. 4, fig. 4 is a flowchart illustrating another control method according to an embodiment of the present invention, where the method may be executed by a remote control device, and the remote control device is specifically explained as described above. As shown in fig. 4, on the basis of the foregoing embodiment, the control method may include the following steps.
S201, detecting control operation of a user through a first interaction device.
And S202, generating a control instruction according to the control operation detected by the first interaction device.
S203, acquiring mode indication information. When the mode indication information is mobile robot control mode indication information, executing steps S204-S205; and when the mode indication information is the terminal equipment control mode indication information, executing the steps S206 to S207.
In one embodiment, the mode indication information may include connection state information between the remote control device and the mobile robot, and when the connection state information is normal connection state information, the remote control device is controlled to enter a mobile robot control mode; and when the connection state information is disconnection state information, controlling the remote control equipment to enter a terminal equipment control mode.
The remote control device may obtain connection state information between the remote control device and the mobile robot, where the connection state information may be used to indicate a network connection state between the remote control device and the mobile robot. When the connection state information is normal connection state information, the remote control device and the mobile robot are in a connection state, the remote control device can perform normal data interaction with the mobile robot, and a processor of the remote control device can control the remote control device to enter a mobile robot control mode. When the connection state information is disconnection state information, the data interaction between the remote control device and the mobile robot cannot be carried out due to the fact that the network is disconnected, and the processor of the remote control device can control the remote control device to enter a terminal device control mode.
In one embodiment, the mode indication information is sent by the terminal device to the remote control device, and specifically, the remote control device may obtain the mode indication information sent by the terminal device, where the mode indication information is determined by the terminal device through detecting a control mode indication operation of the user.
The user can control the remote control equipment to enter a mobile robot control mode or a terminal equipment control mode by means of the terminal equipment performing control mode indication operation. Specifically, the control mode indication operation of the user includes a terminal device control mode indication operation and a mobile robot control mode indication operation, and when the interaction device of the terminal device detects the terminal device control mode indication operation of the user, the terminal device may generate terminal device control mode indication information; when the interaction means of the terminal device detects a mobile robot control mode indication operation of the user, the terminal device may generate mobile robot control mode indication information. The terminal device may send the generated mode indication information to the remote control device, and the processor of the remote control device may control the remote control device to perform the mobile robot control mode or the terminal device control mode indicated by the mode indication information. The interaction device of the terminal equipment can comprise one or more of a touch display screen, a rocker, a keyboard, a key and a wave wheel.
For example, the control mode indicating operation includes detecting a pressing operation of a key by a user, for example, the interaction device of the terminal device includes a mobile robot control mode key and a terminal device control mode key, when the terminal device detects that the user presses the mobile robot control mode key, sending mobile robot control mode indicating information to the remote control device, and the remote control device receiving the mobile robot control mode indicating information and entering the mobile robot control mode; when the terminal equipment detects that a user presses a terminal equipment control mode key, terminal equipment control mode indication information is sent to the remote control equipment, and the remote control equipment receives the terminal equipment control mode indication information and enters a terminal equipment control mode.
For another example, the operation of controlling the mode indication includes receiving a voice control instruction of the user, for example, the user may send a control mode voice instruction to the terminal in a voice manner, the terminal device determines the mode indication information according to the received control mode voice instruction, and sends the mode indication information to the remote control device, and the remote control device receives the mode indication information.
In one embodiment, the remote control device further includes a second interaction means, the remote control device detects a control mode indication operation of the user through the second interaction means, and acquires the mode indication information according to the control mode indication operation detected by the second interaction means. Specifically, the remote control device may include a second interaction means for setting a control mode, and the user may perform a control mode indication operation on the second interaction means to control whether the remote control device enters the mobile robot control mode or the terminal device control mode. The second interaction device detects control mode indication operations of a user, wherein the control mode indication operations comprise mobile robot control mode indication operations and terminal equipment control mode indication operations. The processor of the remote control device may obtain the mode indication information according to the control mode indication operation detected by the second interaction means, that is, the processor may obtain the mobile robot control mode indication information according to the mobile robot control mode indication operation detected by the second interaction means, and the processor may obtain the terminal device control mode indication information according to the terminal device control mode indication operation detected by the second interaction means.
In one embodiment, after step S203, the method further includes: and outputting the mode indication information.
In order to let the user know in which control mode the current control device is, i.e. to let the user know whether the current control device is in the mobile robot control mode or the terminal device control mode. When the remote control device acquires the mode indication information, the mode indication information may be output. Wherein the outputting the mode indication information may include the following feasible implementation manners:
one possible implementation is: and controlling display equipment of the remote control equipment to display prompt information according to the mode indication information. Specifically, the remote control device may include a display device, such as a display screen, an indicator light, and the like, and the processor of the remote control device may control the display device to display prompt information corresponding to the mode indication information according to the acquired mode indication information. For example, the remote control device may include an indicator light, and the indicator light is controlled to emit green light when the mode indication information acquired by the processor of the remote control device is the mobile robot control mode indication information, and to emit red light when the mode indication information acquired by the processor of the remote control device is the terminal device control mode indication information.
Another possible implementation: and sending the mode indication information to the terminal equipment so that the terminal equipment displays prompt information according to the mode indication information. Specifically, the remote control device may transmit the mode indication information to the terminal device, the terminal device may include a display device, such as a touch display screen, an indicator light, and the like, and the processor of the terminal device may control the display device to display prompt information corresponding to the mode indication information according to the received mode indication information. For example, the terminal device may include a touch display screen, and when the processor of the terminal device acquires that the mode indication information sent by the remote control device is the mobile robot control mode indication information, the processor of the terminal device controls the touch display screen to display the "mobile robot control mode", and when the processor of the terminal device acquires that the mode indication information sent by the remote control device is the terminal device control mode indication information, the processor of the terminal device controls the touch display screen to display the "terminal device control mode".
And S204, when the mode indication information is mobile robot control mode indication information, controlling the remote control equipment to enter a mobile robot control mode.
And S205, controlling the mobile robot in communication connection with the remote control equipment according to the control instruction.
In steps S204 to S205, when the mode indication information is the mobile robot control mode indication information, it indicates that the mobile robot needs to be controlled at this time, so the remote control device can control the remote control device to enter the mobile robot control mode, and control the mobile robot in communication connection with the remote control device according to the control instruction.
In one embodiment, first job task setting information sent by a terminal device is received, where the first job task setting information is determined by the terminal device detecting a first job task setting operation of a user, and step S205 includes: and controlling the mobile robot in communication connection with the remote control equipment according to the control instruction so that the mobile robot executes the work task indicated by the first work task setting information.
The user can set a corresponding work task for a control instruction generated by the remote control device in the mobile robot control mode by operating the terminal device, that is, the control instruction is set to instruct the mobile robot to execute which work task. Specifically, a user may perform a first work task setting operation on the terminal device, for example, a control instruction generated by the user through operation setting on the terminal device instructs the mobile robot to execute a photographed work task or adjust a work task of a pan-tilt posture, the terminal device may detect the first work task setting operation of the user to generate first work task setting information, and send the first work task setting information to the remote control device, the remote control device sets a work task corresponding to the control instruction according to the first work task setting information, and sets the control instruction to instruct the mobile robot to execute the work task. When the remote control device is in the mobile robot control mode, the remote control device may control and send the generated control instruction to the mobile robot in communication connection with the remote control device, and the mobile robot executes the work task indicated by the first work task setting information according to the control instruction, for example, the mobile robot executes the shot work task indicated by the first work task setting information or the work task for adjusting the posture of the pan/tilt head according to the control instruction.
S206, when the mode indication information is terminal equipment control mode indication information, controlling the remote control equipment to enter a terminal equipment control mode.
And S207, controlling the terminal equipment in communication connection with the remote control equipment according to the control instruction.
In steps S206 to S207, when the mode indication information is the terminal device control mode indication information, it indicates that the terminal device needs to be controlled at this time, so that the remote control device is controlled to enter the terminal device control mode, and the terminal device in communication connection with the remote control device is controlled according to the control instruction.
In one embodiment, second work task setting information sent by a terminal device is received, wherein the second work task setting information is determined by the terminal device detecting a second work task setting operation of a user; step S207 includes: and controlling the terminal equipment in communication connection with the remote control equipment according to the control instruction so as to enable the terminal equipment to execute the work task indicated by the second work task setting information.
The user can set a corresponding work task for a control instruction generated by the remote control device in the terminal device control mode by operating the terminal device, namely, the control instruction is set to indicate which work task the terminal device executes, and by the way, the user of the first interaction device can set which work task the terminal device instructs to execute. Specifically, a user may perform a second work task setting operation on the terminal device, for example, a control instruction generated by the user through operation setting on the terminal device instructs the terminal device to execute an edit work task for starting an image or a work task for adjusting an interactive interface displayed on the touch display screen, the terminal device may detect the second work task setting operation of the user to generate second work task setting information and send the second work task setting information to the remote control device, and the remote control device sets a work task corresponding to the control instruction according to the second work task setting information, that is, sets the work task that the control instruction instructs the terminal device to execute. When the remote control device is in the terminal device control mode, the remote control device may control and send the generated control instruction to the terminal device in communication connection with the remote control device, and the terminal device executes the work task indicated by the second work task setting information according to the control instruction, for example, the terminal device executes the work task indicated by the second work task setting information, which starts editing of an image, or adjusts the work task of an interactive interface displayed on the touch display screen according to the control instruction.
In one embodiment, the remote control device may comprise a plurality of interaction means, one or more of the plurality of interaction means may be set as the first interaction means, i.e. a target interaction means of the plurality of interaction means may be set as the first interaction means to enable control of the mobile robot or the terminal device in different control modes. Specifically, referring to fig. 5, fig. 5 is a control method according to another embodiment of the present invention, and on the basis of the foregoing embodiment, the method includes:
s301, receiving interactive device selection information sent by the terminal equipment, wherein the interactive device selection information is determined by the terminal equipment detecting the interactive device selection operation of the user.
The user may select one or more of the plurality of interaction means as the first interaction means by operating the terminal device. In particular, the remote control device may comprise a plurality of interaction means, for example the remote control device may comprise a joystick, a wave wheel and keys. The user may perform an interaction device selection operation on the terminal device, for example, perform an operation of selecting a joystick of the plurality of interaction devices, and the terminal device may determine interaction device selection information according to the detected operation, where the interaction device selection information may include an identification number of the selected interaction device, for example, an identification number of the joystick, and the terminal device may transmit the interaction device selection information to the remote control apparatus.
S302, determining one interactive device in the plurality of interactive devices as a first interactive device according to the interactive device selection information.
The remote control apparatus may determine, after receiving the interacting device selection information, which one of the plurality of interacting devices the interacting device information indicates, and determine one of the interacting devices indicated by the interacting device selection information as the first interacting device. For example, if the interactive device selection information includes an identification number of a joystick, the joystick may be determined as the first interactive device according to the identification number.
S303, detecting the control operation of the user through the first interaction device.
And S304, generating a control instruction according to the control operation detected by the first interaction device.
And S305, when the remote control equipment is in a mobile robot control mode, controlling the mobile robot in communication connection with the remote control equipment according to the control instruction.
And when the control instruction is determined to be only defined or the connection state information is the normal connection state information, determining that the remote control equipment is in a mobile robot control mode, and controlling the mobile robot in communication connection with the remote control equipment according to the control instruction.
And S306, when the remote control equipment is in a terminal equipment control mode, controlling the terminal equipment in communication connection with the remote control equipment according to the control instruction.
Based on the above description of a control method and system, an embodiment of the present invention provides a remote control device, as shown in fig. 6, which includes a first interaction means 601, a processor 602, a second interaction means 603, an output means 604 and a communication means 605,
the first interaction device 601 is configured to detect a control operation of a user;
the processor 602 is configured to:
generating a control instruction according to the control operation detected by the first interaction device;
when the remote control equipment is in a mobile robot control mode, controlling a mobile robot in communication connection with the remote control equipment according to the control instruction;
and when the remote control equipment is in a terminal equipment control mode, controlling the terminal equipment in communication connection with the remote control equipment according to the control instruction.
In some embodiments, the processor 602, when controlling the mobile robot communicatively connected to the remote control device according to the control instruction, is specifically configured to:
and the control communication device sends a control instruction to the mobile robot in communication connection with the remote control equipment so as to control the mobile robot according to the control instruction.
In some embodiments, when the processor 602 controls the terminal device communicatively connected to the remote control device according to the control instruction, the method is specifically configured to:
and the control communication device sends a control instruction to the terminal equipment in communication connection with the remote control equipment so as to control the terminal equipment according to the control instruction.
In some embodiments, the terminal device and the remote control device are housed in the same housing.
In some embodiments, the terminal device and the remote control device are physically separate devices.
In some embodiments, the terminal device is a device based on an android system, Linux, or IOS system.
In some embodiments, the processor 602 is further configured to:
acquiring mode indication information;
when the mode indication information is mobile robot control mode indication information, controlling the remote control equipment to enter a mobile robot control mode;
and when the mode indication information is terminal equipment control mode indication information, controlling the remote control equipment to enter a terminal equipment control mode.
In some embodiments, the mode indication information includes connection status information between the remote control device and the mobile robot,
when the mode indication information is mobile robot control mode indication information, and the processor 602 controls the remote control device to enter a mobile robot control mode, specifically, the processor is configured to:
when the connection state information is normal connection state information, controlling the remote control equipment to enter a mobile robot control mode;
when the mode indication information is terminal device control mode indication information and the remote control device is controlled to enter a terminal device control mode, the processor 602 is specifically configured to:
and when the connection state information is disconnection state information, controlling the remote control equipment to enter a terminal equipment control mode.
In some embodiments, when the processor 602 acquires the mode indication information, it is specifically configured to:
acquiring mode indication information sent by the terminal equipment, wherein the mode indication information is determined by the terminal equipment through detecting the control mode indication operation of a user.
In some embodiments, said second interaction means 603 is adapted to detect a control mode indicating operation by a user;
the processor 602 is further configured to: and acquiring mode indication information according to the control mode indication operation detected by the second interaction device.
In some embodiments, the output device 604 is configured to output the mode indication information.
In some embodiments, the apparatus comprises a plurality of interaction devices, and the processor 602 is further configured to:
receiving interactive device selection information sent by terminal equipment, wherein the interactive device selection information is determined by the interactive device selection operation of a user detected by the terminal equipment;
and determining one interactive device in the plurality of interactive devices as a first interactive device according to the interactive device selection information.
In some embodiments, the processor 602 is further configured to:
receiving first work task setting information sent by terminal equipment, wherein the first work task setting information is determined by first work task setting operation of a user detected by the terminal equipment;
when the processor 602 controls the mobile robot in communication connection with the remote control device according to the control instruction, the processor is specifically configured to:
and controlling the mobile robot in communication connection with the remote control equipment according to the control instruction so that the mobile robot executes the work task indicated by the first work task setting information.
In some embodiments, the processor 602 is further configured to:
receiving second work task setting information sent by the terminal equipment, wherein the second work task setting information is determined by a second work task setting operation of a user detected by the terminal equipment;
when the processor 602 controls the terminal device in communication connection with the remote control device according to the control instruction, the following steps are specifically performed:
and controlling the terminal equipment in communication connection with the remote control equipment according to the control instruction so as to enable the terminal equipment to execute the work task indicated by the second work task setting information.
It is understood that the remote control device provided by the embodiment of the present invention can implement the method described above, and the detailed principle and explanation thereof refer to the foregoing parts and are not described herein again.
In an embodiment of the present invention, a computer-readable storage medium is further provided, where the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of fig. 3 to 5, which is not described herein again.
The computer readable storage medium may be an internal storage unit of the device according to any of the foregoing embodiments, for example, a hard disk or a memory of the device. The computer readable storage medium may also be an external storage device of the device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), etc. provided on the device. Further, the computer-readable storage medium may also include both an internal storage unit and an external storage device of the apparatus. The computer-readable storage medium is used for storing the computer program and other programs and data required by the terminal. The computer readable storage medium may also be used to temporarily store data that has been output or is to be output.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
The above disclosure is only for the purpose of illustrating the preferred embodiments of the present invention, and it is therefore to be understood that the invention is not limited by the scope of the appended claims.

Claims (26)

  1. A remote control apparatus for a mobile robot, comprising: a first interaction device and a processor, wherein,
    the first interaction device is used for detecting the control operation of a user;
    the processor is configured to:
    generating a control instruction according to the control operation detected by the first interaction device;
    when the remote control equipment is in a mobile robot control mode, controlling a mobile robot in communication connection with the remote control equipment according to the control instruction;
    and when the remote control equipment is in a terminal equipment control mode, controlling the terminal equipment in communication connection with the remote control equipment according to the control instruction.
  2. The device of claim 1, wherein the terminal device and the remote control device are housed in the same housing.
  3. The device of claim 1, wherein the terminal device and the remote control device are physically separate devices.
  4. The device according to any one of claims 1 to 3, wherein the terminal device is a device based on an android system, Linux, or a mobile operating system IOS system.
  5. The apparatus according to any one of claims 1 to 4,
    the processor is further configured to:
    acquiring mode indication information;
    when the mode indication information is mobile robot control mode indication information, controlling the remote control equipment to enter a mobile robot control mode;
    and when the mode indication information is terminal equipment control mode indication information, controlling the remote control equipment to enter a terminal equipment control mode.
  6. The apparatus of claim 5, wherein the mode indication information includes connection status information between the remote control device and a mobile robot,
    when the mode indication information is mobile robot control mode indication information, and the processor controls the remote control device to enter a mobile robot control mode, the processor is specifically configured to:
    when the connection state information is normal connection state information, controlling the remote control equipment to enter a mobile robot control mode;
    when the mode indication information is terminal device control mode indication information and the remote control device is controlled to enter a terminal device control mode, the processor is specifically configured to:
    and when the connection state information is disconnection state information, controlling the remote control equipment to enter a terminal equipment control mode.
  7. The apparatus according to claim 5 or 6,
    when the processor acquires the mode indication information, the processor is specifically configured to:
    acquiring mode indication information sent by the terminal equipment, wherein the mode indication information is determined by the terminal equipment through detecting the control mode indication operation of a user.
  8. The device of any of claims 5-7, wherein the remote control device further comprises: a second interaction means for, wherein,
    the second interaction device is used for detecting the control mode indication operation of the user;
    the processor is further configured to:
    and acquiring mode indication information according to the control mode indication operation detected by the second interaction device.
  9. The apparatus according to any one of claims 5 to 8, characterized in that it further comprises: an output device, wherein,
    the output device is used for outputting the mode indication information.
  10. An apparatus according to any of claims 1-9, characterized in that the apparatus comprises a plurality of interacting means,
    the processor is further configured to:
    receiving interactive device selection information sent by terminal equipment, wherein the interactive device selection information is determined by the interactive device selection operation of a user detected by the terminal equipment;
    and determining one interactive device in the plurality of interactive devices as a first interactive device according to the interactive device selection information.
  11. The apparatus according to any one of claims 1 to 10,
    the processor is further configured to:
    receiving first work task setting information sent by terminal equipment, wherein the first work task setting information is determined by first work task setting operation of a user detected by the terminal equipment;
    the processor is used for controlling the mobile robot in communication connection with the remote control equipment according to the control instruction, and is specifically used for:
    and controlling the mobile robot in communication connection with the remote control equipment according to the control instruction so as to enable the mobile robot to execute the work task indicated by the first work task setting information.
  12. The apparatus according to any one of claims 1 to 11,
    the processor is further configured to:
    receiving second work task setting information sent by the terminal equipment, wherein the second work task setting information is determined by a second work task setting operation of a user detected by the terminal equipment;
    when the processor controls the terminal device in communication connection with the remote control device according to the control instruction, the processor is specifically configured to:
    and controlling the terminal equipment in communication connection with the remote control equipment according to the control instruction so as to enable the terminal equipment to execute the work task indicated by the second work task setting information.
  13. A control method, characterized in that, applied to a remote control apparatus of a mobile robot, the remote control apparatus includes a first interaction device, characterized in that the method includes:
    detecting a control operation of a user through a first interaction device;
    generating a control instruction according to the control operation detected by the first interaction device;
    when the remote control equipment is in a mobile robot control mode, controlling a mobile robot in communication connection with the remote control equipment according to the control instruction;
    and when the remote control equipment is in a terminal equipment control mode, controlling the terminal equipment in communication connection with the remote control equipment according to the control instruction.
  14. The method of claim 13, wherein the terminal device and the remote control device are housed in the same housing.
  15. The method of claim 13, wherein the terminal device and the remote control device are physically separate devices.
  16. The method according to any one of claims 13 to 15, wherein the terminal device is a device based on an android system, Linux, or IOS system.
  17. The method according to any one of claims 13-16, further comprising:
    acquiring mode indication information;
    when the mode indication information is mobile robot control mode indication information, controlling the remote control equipment to enter a mobile robot control mode;
    and when the mode indication information is terminal equipment control mode indication information, controlling the remote control equipment to enter a terminal equipment control mode.
  18. The method of claim 17, wherein the mode indication information includes connection status information between the remote control device and a mobile robot,
    when the mode indication information is mobile robot control mode indication information, controlling the remote control device to enter a mobile robot control mode, including:
    when the connection state information is normal connection state information, controlling the remote control equipment to enter a mobile robot control mode;
    when the mode indication information is terminal device control mode indication information, controlling the remote control device to enter a terminal device control mode, including:
    and when the connection state information is disconnection state information, controlling the remote control equipment to enter a terminal equipment control mode.
  19. The method according to claim 17 or 18, wherein the obtaining the mode indication information comprises:
    acquiring mode indication information sent by the terminal equipment, wherein the mode indication information is determined by the terminal equipment through detecting the control mode indication operation of a user.
  20. The method of any one of claims 17-19, wherein the remote control device further comprises a second interaction means, the method further comprising:
    detecting a control mode indication operation of a user through the second interaction device;
    and acquiring mode indication information according to the control mode indication operation detected by the second interaction device.
  21. The method according to any one of claims 17-20, further comprising:
    and outputting the mode indication information.
  22. The method according to any of claims 13-21, wherein the apparatus comprises a plurality of interactive devices, the method further comprising:
    receiving interactive device selection information sent by terminal equipment, wherein the interactive device selection information is determined by the interactive device selection operation of a user detected by the terminal equipment;
    and determining one interactive device in the plurality of interactive devices as a first interactive device according to the interactive device selection information.
  23. The method according to any one of claims 13-22, further comprising:
    receiving first work task setting information sent by terminal equipment, wherein the first work task setting information is determined by first work task setting operation of a user detected by the terminal equipment;
    the mobile robot that is connected with remote control equipment communication according to control command control includes:
    and controlling the mobile robot in communication connection with the remote control equipment according to the control instruction so that the mobile robot executes the work task indicated by the first work task setting information.
  24. The method according to any one of claims 13-23, further comprising:
    receiving second work task setting information sent by the terminal equipment, wherein the second work task setting information is determined by a second work task setting operation of a user detected by the terminal equipment;
    the terminal device which is in communication connection with the remote control device according to the control instruction comprises:
    and controlling the terminal equipment in communication connection with the remote control equipment according to the control instruction so as to enable the terminal equipment to execute the work task indicated by the second work task setting information.
  25. A mobile robotic system, comprising: the remote control device, the terminal device and the mobile robot according to any one of claims 1-12,
    the terminal equipment at least comprises a touch display screen.
  26. A computer-readable storage medium, in which a computer program is stored, which, when being executed by a processor, carries out the control method according to any one of claims 13 to 24.
CN201880036977.5A 2018-07-30 2018-07-30 Control method, device, system and storage medium Pending CN111065982A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/097729 WO2020024092A1 (en) 2018-07-30 2018-07-30 Control method, device, system, and storage medium

Publications (1)

Publication Number Publication Date
CN111065982A true CN111065982A (en) 2020-04-24

Family

ID=69230493

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201880036977.5A Pending CN111065982A (en) 2018-07-30 2018-07-30 Control method, device, system and storage medium

Country Status (2)

Country Link
CN (1) CN111065982A (en)
WO (1) WO2020024092A1 (en)

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108292365A (en) * 2015-12-09 2018-07-17 深圳市大疆创新科技有限公司 Imaging system and method for unmanned apparatus of transport
CN108236376A (en) * 2016-12-27 2018-07-03 徐州山川净水设备有限公司 Water dispenser based on wireless control
CN108255161A (en) * 2016-12-28 2018-07-06 昊翔电能运动科技(昆山)有限公司 Aircraft control device and telecontrolled aircraft
CN207615924U (en) * 2017-09-19 2018-07-17 泉州信息工程学院 A kind of all position welding connection device of PLC controls

Also Published As

Publication number Publication date
WO2020024092A1 (en) 2020-02-06

Similar Documents

Publication Publication Date Title
US9493232B2 (en) Remote control method and terminal
KR101655056B1 (en) Wireless video camera
JP2018173992A (en) Information processing device, information processing method, computer readable medium and imaging system
CN107719673B (en) Remote control method and terminal
WO2017032126A1 (en) Unmanned aerial vehicle photographing control method and apparatus, and electronic device
US8937667B2 (en) Image communication apparatus and imaging apparatus
US10241592B2 (en) Image projector device
EP2991338A1 (en) Method and device for switching cameras
KR20160138806A (en) Glass type terminal and method for controlling the same
KR102165450B1 (en) The Apparatus and Method for Portable Device controlling Unmanned Aerial Vehicle
KR100812506B1 (en) Remotely-operated robot, and robot self position identifying method
KR101444407B1 (en) Apparatus for controlling device based on augmented reality using local wireless communication and method thereof
KR20170006559A (en) Mobile terminal and method for controlling the same
CN106598071B (en) Flight control method and device, the unmanned plane of trailing type
CN106558310B (en) Virtual reality voice control method and device
KR101078057B1 (en) Mobile terminal had a function of photographing control and photographing control system used image recognition technicque
US10551833B2 (en) Method for image capturing using unmanned image capturing device and electronic device supporting the same
WO2015098051A1 (en) Information processing device, information processing method, and program
KR102067057B1 (en) A digital device and method of controlling thereof
JP5110805B2 (en) Communication terminal, communication method and program capable of wired and wireless communication
JP2014060501A (en) Photographing control device and control method of imaging device
WO2018077050A1 (en) Target tracking method and aircraft
US9787887B2 (en) Camera peripheral device for supplemental audio capture and remote control of camera
US20090227283A1 (en) Electronic device
US10771686B2 (en) Method and terminal for acquire panoramic image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination