WO2019000380A1

WO2019000380A1 - Method for controlling following of movable device, control device, and following system

Info

Publication number: WO2019000380A1
Application number: PCT/CN2017/091063
Authority: WO
Inventors: 陈一
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2017-06-30
Filing date: 2017-06-30
Publication date: 2019-01-03
Also published as: CN108496141B; CN108496141A

Abstract

A method for controlling the following of a movable device (230), a corresponding control device (220), a control system, and a computer-readable storage medium. The method for controlling the following of the movable device (230) comprises: identifying an action of a target (210) (S101); determining whether the action of the target (210) is a first preset action (S102); and when the action of the target (210) is the first preset action, issuing a following command to the movable device (230) (S103).

Description

Method for controlling movable device following, control device and following system

Copyright statement

The disclosure of this patent document contains material that is subject to copyright protection. This copyright is the property of the copyright holder. The copyright owner has no objection to the reproduction of the patent document or patent disclosure contained in the official records and files of the Patent and Trademark Office.

Technical field

The present disclosure relates to the field of remote control, and more particularly to a method of controlling movable device following, a control device, and a following system.

Background technique

Unmanned aerial vehicles (UAVs), also commonly referred to as "unmanned flight systems (UAS)", "unmanned aerial vehicles" or several other names, are aircraft that have no human pilots on them. When the drone follows the target, it can realize the functions of tracking shooting or tracking the moving position of the target. Tracking is generally controlled by the remote controller or by installing the corresponding software on the smart terminal: for example, manually controlling the drone through the remote controller Follow, follow the application software (App) installed on the phone and then operate on the phone interface to follow.

When using the remote control for follow-up control, it is necessary to manually operate the remote control button to control the following action. In particular, when tracking shots are taken, if the controller follows the target itself, it is often necessary to distract the operation, making the tracking shot unnatural. When using the App for follow-up control, the operation mode also requires manual operation of the smart terminal.

Under the above circumstances, how to simplify and optimize the follow-up control of the drone to make the follow-up effect more natural has become an urgent problem to be solved.

Summary of the invention

According to a first aspect of the present disclosure, a method of controlling a followability of a mobile device is provided, comprising:

Identify the action of the target;

Determining whether the action of the target is a first preset action;

When the action of the target is the first preset action, a follow command is issued to the mobile device.

According to a second aspect of the present disclosure, there is provided a control device for controlling a mobile device to follow, comprising:

a memory for storing instructions;

a processor for executing the instructions, the instructions, when executed by the processor, cause the processor to:

Identify the action of the target;

Determining whether the action of the target is a first preset action;

When the action of the target is the first preset action, a follow command is issued to the movable device.

According to a third aspect of the present disclosure, a tracking system is provided, comprising:

a control device, configured to recognize an action of the target, and issue a follow command to the movable device when the action of the target is the first preset action;

The mobile device is configured to receive the follow command and perform a follow operation according to the follow command.

According to a fourth aspect of the present disclosure, a computer readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method of the first aspect.

DRAWINGS

For a more complete understanding of the embodiments of the present disclosure and its advantages, reference will now be made to the following description

FIG. 1 illustrates a flow chart of a method of controlling a mobile device to follow in accordance with an embodiment of the present disclosure.

FIG. 2 illustrates a method application scenario of controlling a mobile device to follow in accordance with an embodiment of the present disclosure.

FIG. 3 illustrates a flow chart of a method of controlling mobile device following in accordance with another embodiment of the present disclosure.

FIG. 4 shows an application scenario of the control device startup phase of the present disclosure.

FIG. 5 illustrates an application scenario when a signal between the control device and the mobile device of the present disclosure can satisfy a bootable follow condition.

FIG. 6 shows an application scenario in which the control device prompts the following object to make a second preset action.

FIG. 7 illustrates that the mobile device of the present disclosure identifies a follow object and enters a follow-up application scenario.

FIG. 8 illustrates a first scenario of controlling a mobile device to follow an anomaly in accordance with an embodiment of the present disclosure.

FIG. 9 illustrates a second scenario of controlling a mobile device to follow an anomaly in accordance with an embodiment of the present disclosure.

FIG. 10 illustrates a third scenario of controlling a mobile device to follow an anomaly in accordance with an embodiment of the present disclosure.

FIG. 11 illustrates a fourth scenario of controlling a mobile device to follow an anomaly in accordance with an embodiment of the present disclosure.

FIG. 12 shows a hardware schematic of a control device of an embodiment of the present disclosure.

FIG. 13 shows a hardware schematic of a follow system of an embodiment of the present disclosure.

In addition, the drawings are not necessarily to scale,

Specific embodiment

Other aspects, advantages and salient features of the present disclosure will become apparent to those skilled in the <

In the present disclosure, the various embodiments described below for describing the principles of the present disclosure are merely illustrative and should not be construed as limiting the scope of the disclosure. The following description with reference to the drawings is intended to be a The description below includes numerous specific details to assist the understanding, but these details should be considered as merely exemplary. Accordingly, it will be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the disclosure. In addition, descriptions of well-known functions and constructions are omitted for clarity and conciseness. Further, the same reference numerals are used throughout the drawings for the same or similar functions and operations. Moreover, while solutions having different features may be described in different embodiments, those skilled in the art will appreciate that all or a portion of the features of the various embodiments can be combined to form no A new embodiment that departs from the spirit and scope of the present disclosure.

In the present disclosure, the terms "comprising" and "including" and their derivatives are intended to be inclusive and not limiting.

The techniques described in this disclosure may be applied to a mobile device, such as a mobile device in the air (eg, a fixed-wing aircraft such as an airplane, a taxi, a rotorcraft, such as a helicopter, or other aircraft, such as a soft airship). , balloons), mobile devices in the water (eg, submarines, boats, ships), mobile devices on land (eg, motor vehicles, such as cars, motorcycles, buses, trucks, trucks; or poles, such as fish竿 or other types of movable supports or frames; and trains, subways, etc.), mobile devices in space (eg, satellites, space stations, spacecraft). The mobile device can move freely in an environment (such as on land, in water, in the air, in space), or along a preset path, track, or in an agreed manner. The mobile device can be moved in one, two or three dimensions. The mobile device can automatically move in response to the signal without manual movement. In some embodiments, the mobile device can be a transportation vehicle, such as an aerial vehicle, a vehicle on land, a vehicle in water, a vehicle in space, or Their combination. The vehicle can move freely in one or more specified environments, or along a track or a fixed path. The vehicle can include a power system. The power system utilizes an electric motor, an engine, electronic components, magnetic mechanisms, gravity, wind, combustion, and/or other power mechanisms. In some embodiments, a manual power system, a human power system, or a power system by means of other organisms may also be applied to a mobile device, such as a vehicle. In certain embodiments, the moveable device may be a rotorcraft that may be driven or controlled by rotation of one or more blades. The moveable device can be driven or repositioned by means of one or more rotating blades, propellers, wheels, magnets, tracks, or other mechanisms. In some embodiments, the mobile device is an unmanned vehicle, such as an Unmanned Aerial Vehicle (UAV), which may also be referred to as a drone. Unmanned aerial vehicles can hover, adjust direction, and/or follow.

The mobile device can be remotely controlled without requiring someone to be inside or on the mobile device. The mobile device can be remotely manipulated through the terminal. Alternatively, a person may be within the mobile device or over the mobile device to assist in controlling the mobile device. The mobile device can be used to carry loads. In some embodiments, the load carried by the mobile device can include a load and/or carrier that can cause the load to move relative to the mobile device.

The object in the present disclosure refers to an animal that can be identified as making an action, the target may be an animal having a limb and a head, a mobile robot, an animal simulation robot (including but not limited to a robot dog) or a person, on the limb, the torso or When the head moves, the corresponding action can be recognized; the target can also be a moving object that can make an action to form a motion trajectory, and an action of forming a corresponding motion trajectory can also be recognized.

The following object in the present disclosure is the following object, animal or person when the mobile device performs the following, in the following state, the movable device aligns with the following object, and keeps synchronized or approximately synchronous motion with the following object. The following object may be the same person, object or animal as the above target, or may be different.

The ordinal numbers used in the present disclosure, such as "first", "second" and the like, are used to modify different types of preset actions, setting modes, scenes or vibration frequencies, etc., which do not themselves contain and represent corresponding pre- Set any previous ordinal number, such as action, setting mode, scene or vibration frequency, and does not represent a preset action, setting mode, scene or vibration frequency and another preset action, setting mode, scene or vibration frequency. Waiting for the order before or after, or the order of operations, the use of these ordinals is only used to make a preset action, setting mode, scene or vibration frequency with a certain name and other preset actions with the same naming A clear distinction can be made between the mode, the scene or the vibration frequency.

Note that although the following embodiments describe the drone as a mobile device, the watch as a control device, and the user as a target, the present disclosure is not limited thereto. In fact, the mobile device can be in any of the forms described above. In addition, the control device can also be a variety of wearable devices including, but not limited to, wristbands, glasses, helmets, smart shoes, smart clothing, school bags, crutches, or accessories, such as devices that can recognize the action of the target. Moreover, the target can also be other types as described above.

Before some embodiments of the present disclosure are formally described, some of the terms that are to be used herein will first be described.

First Person View (FPV): It is based on the UAV's installation of video recording devices (such as wireless camera back-transfer equipment), that is, the perspective of the drone seen on the remote end. The effect that can be achieved is that the environment of the following object or the surrounding object can be observed from the perspective of the drone in an immersive manner, and the control can be performed more accurately.

Tracking: means that the moving image recording device is aligned with the following object during shooting so that the following object is still moving under the lens field of view of the recording device.

Global Positioning System (GPS): A system that uses GPS to locate satellites and locate and navigate in real time on a global scale. It is called Global Positioning System (GPS). GPS can provide global users with low-cost, high-precision navigation information such as three-dimensional position, speed and precise timing. It is a model application of satellite communication technology in the field of navigation. It greatly enhances the information level of the earth society and strongly promotes it. The development of the digital economy, for example, can provide location information for control devices.

As mentioned earlier, there is a need for a simple and easy to use drone control follower. In the common drone following control mode, it is usually operated by a handheld remote control or a handheld smart terminal such as a mobile phone. For example, you can control the buttons on the remote control or through the touch phone interface, but at the same time, the hands and eyes are focused on the control end, and there are problems such as not being able to quickly start following.

For example, when the user uses the remote control to control the drone to follow his own shooting, the line of sight needs to be aligned with the remote control, so that the shooting is not good, and the facial expressions are unnatural during the follow-up and follow-up process; for example, When the user installs the App and uses the mobile phone to control it, perhaps the line of sight is aligned with the display screen, and the finger needs to be touched accordingly, and the operation mode is not simple enough. It can be seen that in the prior art, the following problems are required: the line of sight alignment control device is required to follow the startup, and the button or the screen of the control device needs to be touched by finger touch or the like, and the manipulation mode is complicated and inconvenient.

In order to at least partially solve or alleviate the above problems, a following system in accordance with an embodiment of the present disclosure has been proposed. This system recognizes the action of the target, initiates drone follow-up when it is a specific action, and reduces the operation to improve the following effect. Through the simple following manner of the system, the control device such as the wearable device can recognize the action of the target and send a follow command to the mobile device to implement corresponding control.

Next, a process of controlling a portable device (for example, a drone) to follow by a control device (for example, a watch) according to an embodiment of the present disclosure will be described in detail with reference to FIGS. 1 to 2.

FIG. 1 illustrates a flow chart of a method of controlling a mobile device to follow in accordance with an embodiment of the present disclosure. FIG. 2 illustrates a method application scenario of controlling a mobile device to follow in accordance with an embodiment of the present disclosure. As shown in FIG. 1, the method for controlling the following of the mobile device by the control device control includes:

S101: an action of identifying a target;

S102: Determine whether the action of the target is a first preset action;

S103: If the action of the target is the first preset action, issue a follow command to the mobile device.

The action of the target in step S101, as shown in FIG. 2, is an exemplary picker as the target 210, but the present disclosure is not limited thereto. The target 210 may also have other forms, and may be an animal having a limb and a head, a mobile robot, an animal simulation robot (including but not limited to a robot dog), or a person. When the limb, the torso, or the head moves, the corresponding action may be The target may also be a moving object (including but not limited to a car, a train or a ship) capable of making an action to form a trajectory, and an action of forming a corresponding trajectory may also be recognized. In Fig. 2, the arm of the target 210 can act (see the direction of the arrow in Fig. 2, the arm can swing back and forth), and the control device 220 worn on the arm can recognize the action made.

The manner of identifying the target 210 may be: synchronizing with the movement of the specific part of the target 210, sensing the acceleration, angular velocity and/or displacement variation of the specific part of the target; and synchronizing with the overall movement of the target 210, sensing the overall acceleration of the target , angular velocity and / or displacement variation. The identification can be achieved by the target wearing the control device 220 of the present disclosure, and the control device includes corresponding associated functional units that detect the aforementioned acceleration, angular velocity, and/or displacement variation.

In the embodiment of the present disclosure, the first preset action in step S103 may include the first preset action including at least one of the following: when the target is an animal, a mobile robot, an animal simulation robot, or a person, according to the first setting manner When the limb, the trunk or the head is swung; and the target is a moving object, the motion trajectory is made according to the second setting manner, but the present disclosure is not limited thereto. It should be understood that the first preset action may be to swing the limb, the torso, the head Part or any combination, or through a specific identifier, such as a logo with a specific pattern, a specific shape of the identification object, etc., but preferably with a discriminative and simple action to improve handling efficiency.

In some embodiments, swinging the limb, torso or head according to the first setting comprises at least one of: swinging the limb, torso or head at a set frequency (eg, swinging the arm at a frequency of 3 back and forth per second); Set the angular velocity to swing the limb, torso or head (for example, rotate the arm at a rate of 1 revolution per second); swing the limb, torso or head according to the set pattern (for example, the arm swings clockwise out of a semicircle); A range of angles swings the limb, torso, or head (such as an arm swing angle of 160-180°).

In some embodiments, the motion trajectory is made according to the second setting manner and includes at least one of: swinging itself according to the set frequency (for example, the car swings 2-3 times every 10 times); swinging itself according to the set angular velocity (for example, a car) Rotate at a speed of 1 lap every 10 seconds; move according to the set graph (for example, the car goes out of the figure 8); and move according to the set angle range (for example, the swing angle of the car is 160-180° for the arm swing angle).

In some embodiments, other portions of the target 210 may also be actuated, including but not limited to feet, waist, head, or target entirety, or a combination of any of the above-described types of actions. Accordingly, in some embodiments, to improve the accuracy and accuracy of the identification, the portion of the action is carried with the control device 220.

In some embodiments, the control device 220 may also be other wearable devices other than a watch, such as a wristband, glasses, helmet, smart shoes, smart clothing, school bag, crutches, or accessories, and the disclosure is not limited thereto.

In step S103, for some embodiments, issuing a follow command to the mobile device 230 includes: transmitting a reference coordinate, transmitting a command to cause the movable device to approach the reference coordinate according to the set movement parameter, and aligning the image capturing device in the movable device with the reference Around the coordinates. Please note: In this article, since the reference coordinates are the coordinates of the control device, unless otherwise stated, the coordinates of the reference coordinates and the control device will not be distinguished below. In other words, "reference coordinates" and "coordinates of the first device" are used interchangeably hereinafter unless otherwise stated. The follow command is issued to automatically issue the result of the action based on the target.

In general, in some embodiments, if it is determined that the action of the target is not the first preset action, proceeding to step S101 to continue the action of identifying the target.

Next, some correlation processes for controlling a mobile device (for example, a drone) to follow and an abnormal process occurring by a control device (for example, a watch) according to an embodiment of the present disclosure will be described in detail with reference to FIGS. 3 to 11.

FIG. 3 illustrates a flow chart of a method of controlling mobile device following in accordance with another embodiment of the present disclosure. Step S303 is the same as step S101, step S304 is the same as S102, and step S305 is the same as step S103. I will not repeat them. It should be understood that in addition to steps S303, S304, and S305 in FIG. 3, other steps may be arbitrarily selected to combine to form a plurality of technical solutions according to embodiments of the present disclosure.

In some embodiments, before step S303, the method of controlling the following of the mobile device may further include step S301: initiating a control device (such as a watch) to be connected to the mobile device. Further, in a specific embodiment, as shown in FIG. 3, step S301 may further include displaying first-person perspective information of the mobile device to the target: height, distance, power of the removable device 230, shooting mode, GPS, and/or The signal strength between the control device and the mobile device is only an exemplary description, and the disclosure is not limited thereto.

FIG. 4 illustrates an application scenario of the startup phase of the control device 420 of the present disclosure. The upper left of the figure shows the content displayed on the interface of the control device 420, and the lower right is the specific scene. As shown in FIG. 4, basic first person view information (including but not limited to height, distance, power of the mobile device 230, shooting mode, GPS, and/or signal strength between the control device and the mobile device is displayed. Combination), since the follow-up has not yet been entered, the shooting mode is not turned on. In this scenario, the target 410 can learn information about the first person perspective of the removable device 430.

FIG. 5 illustrates an application scenario when a signal between the control device and the mobile device of the present disclosure can satisfy a bootable follow condition. The upper left of the figure shows the content displayed on the interface of the control device 520, and the lower right is the specific scene. In some embodiments, as shown in FIG. 3, before the action of identifying the target in step S303, step S302 may be further included: prompting the target 510 to make a first preset action. For example, a dynamic map appears on the interface of the control device 520, showing the first preset action, at which point the mobile device 530 is still in a waiting state and has not moved closer to the control device 520.

In this state, the target 510 may perform a corresponding first preset action (such as shaking the arm) according to the dynamic image, so that the control device 520 recognizes, and after identifying the first preset action, sends the mobile device 530 to the mobile device 530. Follow the order. In some embodiments, issuing the following command includes transmitting the reference coordinates, transmitting the command to cause the movable device to approach the reference coordinates in accordance with the set movement parameter, and aligning the image capture device in the movable device about the reference coordinates. After receiving the following command, the mobile device 530 approaches the control device 520 with the preset movement parameters and aligns the image recording device with the control device 520.

In some embodiments, the manner in which the prompting target makes the first preset action includes at least one of: issuing a voice prompt to the target 510 through the control device 520, or displaying the first preset action to the target 510 through the control device 520. a graphic (such as an animation or a picture), prompting the target 510 by vibration (for example, vibrating at a first vibration frequency), and signaling a target to the target (when the target 510 is a movable object, and the target can receive control wirelessly or by wire) Signal sent by device 520).

FIG. 6 shows an application scenario in which the control device prompts the following object to make a second preset action. The upper left of the figure shows the content displayed on the interface of the control device 620, and the lower right is the specific scene. In some embodiments, after issuing the following command to the mobile device in step S305, step S306 may be further included: prompting the following object 640 to make a second preset action. The prompt mode is set by referring to the prompt mode of the first preset action described above, and details are not described herein. It should be noted that the target 610 and the following object 640 may be the same or different, because the image capturing device 631 of the movable device 630 is aligned with the control device 620, and the control device 620 is located in the region (the image capturing device) The moving person or object within the 631 lens (within the dashed line in FIG. 6) can be the following object 640, including but not limited to a movable person, object or animal. The following object 640 and the target 610 illustrated in FIG. 6 are different people, and of course, the following object 640 and the target 610 may be the same. The prompting manner of the second preset action may be different or the same as the prompting manner of the first preset action, preferably different prompting manners (for example, displaying different moving images on the control device 620), and there is a difference in the prompting manner, and Avoid confusion between different types of prompts.

In some embodiments, the second preset action comprises at least one of: swinging the limb, the torso or the head according to the third setting manner when the following object is an animal, a mobile robot, an animal simulation robot or a person; and the following object is moving When the object is in accordance with the fourth setting method, the motion trajectory is made.

In a further embodiment, swinging the limb, torso or head according to the third setting manner comprises at least one of: swinging the limb, torso or head at a set frequency; swinging the limb, torso or head at a set angular velocity; Set the graphic to swing the limb, torso or head; and swing the limb, torso or head to a set angle range.

In a further embodiment, the motion trajectory is made according to the fourth setting mode and includes at least one of: swinging itself according to the set frequency; oscillating itself according to the set angular velocity; moving according to the set graphic; and moving according to the set angle range.

It should be noted that the second preset action and the first preset action may be the same or different because they are recognized by different objects (the first preset action is recognized by the control device and the second preset action is recognized by the mobile device). .

In this state, the following object 640 can perform a corresponding second preset action (such as waving a double arm) according to the dynamic image displayed on the upper left of FIG. 6 to enable the removable device 630 to recognize, and after identifying the second preset action, The following object 640 is identified and the following object 640 is followed. In some embodiments, issuing the following command includes transmitting the reference coordinates, transmitting the command to cause the movable device to approach the reference coordinates in accordance with the set movement parameter, and aligning the image capture device in the movable device about the reference coordinates. After receiving the following command, the mobile device 630 approaches the control device 620 with the preset movement parameters and aligns the image recording device with the control device 520.

FIG. 7 illustrates that the mobile device of the present disclosure identifies a follow object and enters a follow-up application scenario. The upper left of the figure shows the content of the control device interface, and the lower right is the specific scene. In some embodiments, as shown in FIG. 3, after prompting the following object to make the second preset action in step S306, step S307 may be further included: prompting to follow the object 740 or the target, the following object has been identified, and the mobile device 730 is Follow the object to follow.

In this state, the control device may prompt the following object or the target, and the prompting manner includes at least one of: issuing a voice prompt to the target or following object 740 through the control device, or displaying the target or following object 740 through the control device. Following a state graphic (such as an animation or a picture), prompting a target by vibrating or following an object 740 (eg, vibrating at a second vibration frequency) and signaling a target or following object (when the target or following object is a movable object, And the signal transmitted by the control device can be received in a wireless or wired manner).

The above describes some application methods when the control device performs follow-up control, but it is not limited to this, and some application modes may be added or deleted according to actual needs. Several application scenarios for following anomalies will be described below with reference to Figures 8-11.

FIG. 8 illustrates a first scenario of controlling a mobile device to follow an anomaly in accordance with an embodiment of the present disclosure. In some embodiments, the mobile device 830 may not be able to be moved to the mobile device 830 because the control device 820 carried by the target 810 is too far apart or has a signal shield between the mobile device 830 or the geographical environment of the control device 820 is poor. A follow command is issued and communication is not possible.

In this state, the control device 820 can indicate to the target 810 that the follow-up cannot be initiated. The target 810 can adjust its position, adjust the position of the control device, and/or the position of the movable device 830 according to the actual geographical condition, and restart the follow-up.

FIG. 9 illustrates a second scenario of controlling a mobile device to follow an anomaly in accordance with an embodiment of the present disclosure. In some embodiments, when the mobile device 930 follows the following object 940, a following anomaly may occur (eg, the mobile device 930 is faster than the following object 940, the distance between the two is too close; the control device 820 is movable The devices 830 are spaced too far apart; the following object 940 is not within the lens range of the image capture device 631; there is an obstacle (eg, a tree) between the movable device 930 and the following object 940.

In this state, the control device may prompt the target or following object 940 to follow the abnormality, and the target or following object 940 may adjust its position, adjust the position of the control device, and/or the position of the movable device 930 according to the above actual situation, and restart. follow.

FIG. 10 illustrates a third scenario of controlling a mobile device to follow an anomaly in accordance with an embodiment of the present disclosure. In some embodiments, when the mobile device 1030 follows the following object 1040, due to the occurrence of a special abnormality, such as the insufficient energy of the mobile device 1030, the distance between the movable device 1030 and the following object 1040, the loss is followed.

In this state, the control device can prompt the target or follow object 1040 to follow the abnormality, cannot retrieve the lost target, and needs to restart.

FIG. 11 illustrates a fourth scenario of controlling a mobile device to follow an anomaly in accordance with an embodiment of the present disclosure. When the abnormality shown in FIG. 9 occurs, the control device 1120 may prompt the target or the following object 1140 whether to determine the retrieval, and the following object 1140 or the target selects whether or not according to the actual situation (for example, the mobile device 1130 has appeared nearby, Then select Yes, if you can't retrieve the lost removable device, you need to start over).

The specific functions and compositions of the control device of the present disclosure will be described below in conjunction with the above drawings in conjunction with FIG.

FIG. 12 shows a hardware schematic of a control device of an embodiment of the present disclosure. A control device, as shown in Figure 12, for controlling a mobile device to follow, comprising: a memory for storing instructions; a processor for executing the instructions, the instructions being executed by the processor The processor is caused to: recognize an action of the target; and if the action of the target is the first preset action, issue a follow command to the mobile device.

The control device can include a processor 1206 (eg, a central processing unit (CPU), a digital signal processor (DSP), a microcontroller unit (MCU), etc.). The processor 1206 can be a single processing unit or a plurality of processing units for performing different acts of the processes described herein. The control device may also include an input unit 1202 for receiving signals from other entities and an output unit 1204 for providing signals to other entities. The input unit 1202 and the output unit 1204 may be arranged as a single entity or as separate entities.

Moreover, the control device can include at least one readable storage medium 1208 in the form of a non-volatile or volatile memory, such as an electrically erasable programmable read only memory (EEPROM), flash memory, and/or a hard drive. The readable storage medium 1208 includes computer program instructions 1210 that include code/computer readable instructions that, when executed by the processor 1206 in the control device, cause the control device to perform the following methods:

Identify the action of the target;

If the action of the target is the first preset action, a follow command is issued to the mobile device.

The action of the above object, as shown in FIG. 2, is an exemplary selection person as the target 210, but the present disclosure is not limited thereto. The target 210 can also have other forms, which can be an animal with a limb and a head, a mobile robot, Animal simulation robots (including but not limited to robot dogs) or humans, when moving in the limbs, torso or head, the corresponding actions can be identified; the target can also be a moving object that can make an action to form a motion trajectory (including but not limited to The action of forming a corresponding motion trajectory can also be recognized by a car, a train or a ship. In Fig. 2, the arm of the target 210 can act (see the direction of the arrow in Fig. 2, the arm can swing back and forth), and the control device 220 worn on the arm can recognize the action made.

In the embodiment of the present disclosure, the first preset action may include the first preset action including at least one of the following: when the target is an animal, a mobile robot, an animal simulation robot, or a person, swinging the limb and the trunk according to the first setting manner Or the head; and when the target is a moving object, the motion trajectory is made according to the second setting manner, but the present disclosure is not limited thereto. It should be understood that the first preset action may be any combination of three kinds of swinging limbs, torso or head, but it is preferable to adopt a discriminating and simple action to improve the handling efficiency.

In some embodiments, swinging the limb, torso or head according to the first setting comprises at least one of the following:

Swing your limbs, torso, or head at a set frequency (for example, swing your arm at a rate of 3 round trips per second); swing your limbs, torso, or head at a set angular velocity (for example, rotate your arm at 1 lap per second); Set the graphic to swing the limb, torso or head (such as the arm swinging out of a semicircle clockwise); and swing the limb, torso or head according to the set angle range (such as the arm swing angle of 160-180°).

For some embodiments, issuing a follow command to the mobile device 230 includes transmitting reference coordinates, transmitting a command to cause the mobile device to approach the reference coordinates in accordance with the set movement parameters, and aligning the image capture device in the movable device about the reference coordinates. Please note: In this article, since the reference coordinates are the coordinates of the control device, unless otherwise stated, the coordinates of the reference coordinates and the control device will not be distinguished below. In other words, "reference coordinates" and "coordinates of the first device" are used interchangeably hereinafter unless otherwise stated. The follow command is issued to automatically issue the result of the action based on the target.

Next, other instructions related to the follow-up performed by the control device according to an embodiment of the present disclosure will be described in detail with reference to FIGS. 3 to 11.

In some embodiments, when the processor executes computer program instructions 1210, it is also possible to activate a control device (eg, a watch) to connect to the mobile device. FIG. 4 illustrates an application scenario of the startup phase of the control device 420 of the present disclosure. The upper left of the figure shows the content displayed on the interface of the control device 420, and the lower right is the specific scene. As shown in FIG. 4, basic first person view information (including but not limited to height, distance, power of the mobile device 230, shooting mode, GPS, and/or any combination of signal strength between the control device and the mobile device) is shown. Since the follow-up has not yet entered, the shooting mode is not turned on. In this scenario, the target 410 can learn information about the first person perspective of the removable device 430.

In some embodiments, when the processor executes the computer program instructions 1210, the target 510 may also be prompted to make a first preset action before the action of identifying the target. FIG. 5 illustrates an application scenario when a signal between the control device and the mobile device of the present disclosure can satisfy a bootable follow condition. The upper left of the figure shows the content displayed on the interface of the control device 520, and the lower right is the specific scene. For example, a dynamic map appears on the interface of the control device 520, showing the first preset action, at which point the mobile device 530 is still in a waiting state and has not moved closer to the control device 520.

In some embodiments, the manner in which the prompting target makes the first preset action includes at least one of: issuing a voice prompt to the target 510 through the control device 520, or moving to the target through the control device 520. 510 displays a first preset motion pattern (eg, an animation or a picture), prompts the target 510 by vibration (eg, vibrates at a first vibration frequency), and sends a prompt signal to the target (when the target 510 is a movable object, and the target can The signal transmitted by the control device 520 is received in a wireless or wired manner).

In some embodiments, when the processor executes the computer program instructions 1210, it is also possible to prompt the following object to make a second preset motion after issuing the following command to the mobile device. FIG. 6 shows an application scenario in which the control device prompts the following object to make a second preset action. The upper left of the figure shows the content displayed on the interface of the control device 620, and the lower right is the specific scene. The prompt mode is set by referring to the prompt mode of the first preset action described above, and details are not described herein. It should be noted that the target 610 and the following object 640 may be the same or different, because the image capturing device 631 of the movable device 630 is aligned with the control device 620, and the control device 620 is located in the region (the image capturing device) The moving person or object within the 631 lens (within the dashed line in FIG. 6) can be the following object 640, including but not limited to a movable person, object or animal. The following object 640 and the target 610 illustrated in FIG. 6 are different people, and of course, the following object 640 and the target 610 may be the same. The prompting manner of the second preset action may be different or the same as the prompting manner of the first preset action, preferably different prompting manners (for example, displaying different moving images on the control device 620), and there is a difference in the prompting manner, and Avoid confusion between different types of prompts.

It should be noted that since they are recognized by different objects (the first preset action is recognized by the control device and the second preset action is recognized by the mobile device), the second preset action and the first preset action may be the same. Can be different.

In this state, the following object 640 can sit on the display dynamic picture according to FIG. 6 to make a corresponding second preset action (such as waving a double arm), so that the mobile device 520 recognizes, after identifying the second preset action, Knowledge The follow object 640 is not followed, and the follow object 640 is followed. In some embodiments, issuing the following command includes transmitting the reference coordinates, transmitting the command to cause the movable device to approach the reference coordinates in accordance with the set movement parameter, and aligning the image capture device in the movable device about the reference coordinates. After receiving the following command, the mobile device 530 approaches the control device 520 with the preset movement parameters and aligns the image recording device with the control device 520.

In some embodiments, when the processor executes the computer program instructions 1210, it may also be possible to prompt the following object or the target to issue a follow command to the mobile device after prompting the follower object to make the second preset action. FIG. 7 illustrates that the mobile device of the present disclosure identifies a follow object and enters a follow-up application scenario. The upper left of the figure shows the content of the control device interface, and the lower right is the specific scene.

In this state, the control device may prompt the following object or the target object 740, and the prompting manner includes at least one of: issuing a voice prompt to the target or following object 740 by the control device, or to the target or following object 740 by the control device. Displaying a follow-up state graphic (eg, an animation or a picture), prompting the target by vibration or following the object 740 (eg, vibrating at a second vibration frequency) and signaling a target or following object (when the target or following object is a movable object) And can receive signals transmitted by the control device in a wireless or wired manner).

The computer program instructions 1210 related to the following performed by the control device are described above, but are not limited thereto, and some instructions may be added or deleted according to actual needs. Related computer program instructions 1210 following an exception are described below in conjunction with Figures 8-11.

In some embodiments, computer program instructions 1210 may also include instructions related to following an exception. FIG. 8 illustrates a first scenario of controlling a mobile device to follow an anomaly in accordance with an embodiment of the present disclosure. In some embodiments, the mobile device 830 may not be able to be moved to the mobile device 830 because the control device 820 carried by the target 810 is too far apart or has a signal shield between the mobile device 830 or the geographical environment of the control device 820 is poor. A follow command is issued and communication is not possible.

In this state, the processor of the control device 820 can execute the command-to-target 810 to indicate that the follow-up cannot be initiated. The target 810 can adjust its position, adjust the position of the control device, and/or the position of the movable device 830 according to the actual geographical situation, and Start following.

In this state, the processor of the control device can execute an instruction to prompt the target or follow object 940 to follow the abnormality, and the target 910 or the following object 940 can adjust its position, adjust the position of the control device, and/or the movable device according to the above actual situation. 930's position, restart to follow.

In this state, the processor of the control device can execute an instruction to prompt the target or follow object 1040 to follow the exception, unable to retrieve the lost target, and needs to restart.

FIG. 11 illustrates a fourth scenario of controlling a mobile device to follow an anomaly in accordance with an embodiment of the present disclosure. When the exception shown in FIG. 9 occurs, the processor of the control device can execute an instruction to the target or the following object 1140 to prompt whether to determine the retrieval, follow the object 1140 or the target actual situation to select yes or no (eg, the mobile device 1130 has appeared In the vicinity, select Yes, if you can't retrieve the lost removable device, you need to start over).

In some embodiments, the control device further includes a motion recognition unit 1212, including at least one of: an acceleration sensor for recognizing an amount of acceleration change of the motion of the target; an angular velocity sensor for recognizing an angular velocity variation of the motion of the target; A displacement sensor that is used to identify the displacement change of the target.

In some embodiments, the control device further includes an action locating unit 1214 for transmitting location information of the control device to the mobile device. Positioning unit 1214 can include, but is not limited to, a GPS.

The processor may be a single CPU (Central Processing Unit), but may also include two or more processing units. For example, a processor can include a general purpose microprocessor, an instruction set processor, and/or a related chipset and/or a special purpose microprocessor (eg, an application specific integrated circuit (ASIC)). The processor may also include an onboard memory for caching purposes. Computer program instructions may be hosted by a computer program instruction product coupled to the processor. The computer program instructions product can comprise a computer readable medium having stored thereon computer program instructions. For example, the computer program instructions product can be flash memory, random access memory (RAM), read only memory (ROM), EEPROM, and the computer program instructions modules described above can be distributed in the form of memory within the UE to alternative embodiments. Different computer program instruction products.

It should be noted that the functions described herein as being implemented by pure hardware, software and/or firmware may also be implemented by means of dedicated hardware, a combination of general hardware and software, and the like. For example, functions described as being implemented by dedicated hardware (eg, Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), etc.) may be implemented by general purpose hardware (eg, central processing unit (CPU), digital signal processing (DSP) is implemented in a way that is combined with software and vice versa.

Next, the following system of the present disclosure will be described with reference to FIG. 13 and the accompanying drawings of the above application scenarios.

FIG. 13 shows a hardware schematic of a follow system of an embodiment of the present disclosure. The following system 1300 includes a control device 1310 and a removable device 1320, wherein the introduction of the control device 1310 has been described above, and the mobile device 1320 of the controlled terminal is highlighted here.

A tracking system shown in FIG. 13 includes a control device 1310 for recognizing an action of a target, such as a target action being a first preset action, issuing a follow command to the mobile device, and a removable device 1320 for Receiving the following command to preset the movement parameter to be close to the control device, the movable device 1320 further comprising an image recording device 1323 for aligning the image capturing device after approaching the control device with a preset movement parameter controlling device.

In some embodiments, the video recording device 1323 may be a video camera, but may also be a camera. The disclosure is not limited thereto, and any device capable of recording an image is included herein.

In some embodiments, the mobile device 1320 further includes a pan/tilt that can be used to adjust an alignment angle of the image capture device; in particular, after the control device 1310 sends a follow command to the mobile device 1320, the mobile device After the 1320 moves to the movable device 1320 according to the set movement parameter, the movable device 1320 starts to adjust the parameters such as the pitch angle of the image capturing device through the pan/tilt, so that the camera framing range is aligned with the vicinity of the movable device 1320.

In some embodiments, the preset movement parameters include at least one of: a relative position to the control device, a ground height, and an alignment angle of the image recording device.

In some embodiments, the movable device 1320 is further configured to recognize an action of following the object, such as confirming the following object for the second preset action. The second preset action includes at least one of: when the object to be followed is an animal, a mobile robot, an animal simulation robot or a person, swinging the limb, the trunk or the head according to the third setting manner; and when the following object is a moving object, The motion trajectory is made according to the fourth setting mode. In a further embodiment, swinging the limb, torso or head according to the third setting manner comprises at least one of: swinging the limb, torso or head at a set frequency; swinging the limb, torso or head at a set angular velocity; Set graphic swing limb Body, torso or head; and swing the limb, torso or head at a set angle range. To implement the above functions, the removable device 1320 includes an identification unit 1324 for identifying the second preset action.

In some embodiments, the removable device 1320 further includes an input unit 1325 for receiving a follow command issued by the control device, ending the follow command.

In some embodiments, the removable device 1320 further includes an output unit 1322 that transmits image data collected by the image capture device to the control device.

In some embodiments, the mobile device 1320 is further configured to recognize an action following the object, such as ending the following for a third set action. The setting and the recognition mode of the third action refer to the second setting action, but the second setting action and the third setting action should not be identical. At this time, the identification unit 1324 is further configured to identify the third setting action.

The above units are all connected to the processor 1321 of the removable device 1320, and the processor 1321 controls the input unit 1325, the output unit 1322, the image recording device 1323, the recognition unit 1324, and the pan/tilt when executing the command.

Although the present disclosure has been shown and described with respect to the specific exemplary embodiments of the present disclosure, it will be understood by those skilled in the art Various changes in form and detail are made to the present disclosure. Therefore, the scope of the present disclosure should not be limited to the above-described embodiments, but should be determined not only by the appended claims but also by the equivalents of the appended claims.

Claims

A method of controlling the following of a mobile device, comprising:

Identify the action of the target;

Determining whether the action of the target is a first preset action; and

When the action of the target is the first preset action, a follow command is issued to the mobile device.
The method of claim 1, wherein the first preset action comprises at least one of the following:

When the target is an animal, a mobile robot, an animal simulation robot or a human, swing the limb, torso or head according to the first setting;

When the target is a moving object, the motion trajectory is made according to the second setting mode.
The method of claim 2 wherein swinging the limb, torso or head in accordance with the first setting comprises at least one of the following:

Swing the limb, torso or head at a set frequency;

Swing the limb, torso or head at a set angular velocity;

Swing the limb, torso or head in a set pattern;

Swing the limb, torso or head at a set angle range.
The method according to claim 2, wherein the motion trajectory is made according to the second setting manner and includes at least one of the following:

Swing yourself at the set frequency;

Swing yourself at the set angular velocity;

Move by the setting graphic;

Move by the set angle range.
The method according to any one of claims 1 to 4, wherein the action of identifying the target comprises at least one of the following:

Synchronizing with the movement of a particular part of the target, sensing the acceleration, angular velocity and/or amount of change of a particular part of the target;

Synchronizing with the overall motion of the target, the acceleration, angular velocity, and/or displacement variation of the target as a whole is sensed.
The method according to any one of claims 1 to 5, wherein the method further comprises: prompting the target to make a first preset action before the action of identifying the target.
The method according to claim 6, wherein the prompting the target to make the first preset action comprises at least one of the following:

A voice prompt is sent to the target, a first preset motion graphic is displayed to the target, the target is prompted by vibration, and a prompt signal is sent to the target.
The method according to any one of claims 1 to 7, wherein the method further comprises: before the action of identifying the target, prompting the target that the action of identifying the target cannot be performed.
The method according to any one of claims 1-8, wherein the issuing a follow command to the mobile device comprises: transmitting a reference coordinate, sending a command to cause the movable device to approach the set according to the set moving parameter Reference coordinates and aligning the image capture device in the movable device around the reference coordinates.
The method according to any one of claims 1 to 9, wherein the method further comprises: after issuing a follow command to the mobile device, prompting the follower object to make a second preset action to make the mobile device Identify and confirm the following objects.
The method according to claim 10, wherein the second preset action comprises at least one of the following:

When the subject is an animal, a mobile robot, an animal simulation robot or a person, swing the limb, torso or head according to the third setting;

When the following object is a moving object, the motion trajectory is made according to the fourth setting mode.
The method of claim 11 wherein swinging the limb, torso or head in accordance with the third setting comprises at least one of the following:

Swing the limb, torso or head at a set frequency;

Swing the limb, torso or head at a set angular velocity;

Swing the limb, torso or head in a set pattern;

Swing the limb, torso or head at a set angle range.
The method according to claim 11, wherein the making of the motion trajectory according to the fourth setting manner comprises at least one of the following:

Swing yourself at the set frequency;

Swing yourself at the set angular velocity;

Move by the setting graphic;

Move by the set angle range.
The method according to claim 10, wherein the prompting the following object to make the second preset action comprises at least one of the following:

A voice prompt is issued to the following object, a second preset motion graphic is displayed to the following object, the following object is prompted by the vibration mode, and a prompt signal is sent to the following object.
The method according to claim 10, wherein the method further comprises: after the mobile device recognizes and confirms the following object, following the target to prompt the target to follow the normal, prompting the target to follow the abnormality, and prompting the target. Whether to retrieve the following object or prompt the target to follow the object is lost.
The method according to claim 15, wherein said following prompt for the target comprises at least one of the following:

A voice prompt is sent to the target to show the target a normal follow-up, follow an abnormality, or follow the lost object, and alert the target by vibration.
The method of claim 10, wherein the method further comprises: after the mobile device identifies and confirms the following object, transmitting an end following command to the mobile device.
A control device for controlling a mobile device to follow, including:

a memory for storing instructions;

a processor for executing the instructions, the instructions, when executed by the processor, cause the processor to:

Identify the action of the target;

Determining whether the action of the target is a first preset action;

When the action of the target is the first preset action, a follow command is issued to the movable device.
The control device of claim 18, wherein said control device is a wearable device.
The control device according to claim 18 or 19, wherein the first preset action comprises at least one of the following:

When the target is an animal, a mobile robot, an animal simulation robot or a human, swing the limb, torso or head according to the first setting;

When the target is a moving object, the motion trajectory is made according to the second setting mode.
The control apparatus according to claim 20, wherein the swinging the limb, the torso or the head according to the first setting manner comprises at least one of the following:

Swing the limb, torso or head at a set frequency;

Swing the limb, torso or head at a set angular velocity;

Swing the limb, torso or head in a set pattern;

Swing the limb, torso or head at a set angle range.
The control device according to claim 21, wherein the motion trajectory is made according to the second setting manner and includes at least one of the following:

Swing yourself at the set frequency;

Swing yourself at the set angular velocity;

Move by the setting graphic;

Move by the set angle range.
The control device according to any one of claims 18 to 22, wherein the action of identifying the target comprises at least one of the following:

Synchronizing with the movement of a specific part of the target, sensing the amount of acceleration change of a specific part of the target;

Synchronize with the overall motion of the target to sense the amount of acceleration change of the target as a whole.
The control device according to any one of claims 18 to 23, wherein the instructions, when executed by the processor, further cause the processor to: prompt the target to make a first before the action of identifying the target Preset action.
The control device according to claim 24, wherein the prompting manner of the prompting target in the first preset action comprises at least one of the following:

A voice prompt is sent to the target, a first preset motion graphic is displayed to the target, the target is prompted by vibration, and a prompt signal is sent to the target.
The control device according to any one of claims 18 to 25, wherein the instruction, when executed by the processor, further causes the processor to: prompt the target to fail to perform recognition before recognizing the action of the target The action of the target.
The control device according to any one of claims 18 to 26, wherein the issuing a follow command to the mobile device comprises: transmitting a coordinate of the control device, and transmitting a command to cause the mobile device to move the parameter according to the setting Close to the coordinates of the control device and around the coordinates of the control device that align the image capture device in the mobile device.
The control device according to any one of claims 18 to 27, wherein the instructions, when executed by the processor, further cause the processor to prompt the follower to make a follow command after the slave device is issued The second preset action is to cause the mobile device to recognize and confirm the following object.
The control device according to claim 28, wherein said second preset action comprises at least one of the following:

When the subject is an animal, a mobile robot, an animal simulation robot or a person, swing the limb, torso or head according to the third setting;

When the following object is a moving object, the motion trajectory is made according to the fourth setting mode.
The control apparatus according to claim 29, wherein the swinging the limb, the trunk or the head according to the third setting manner comprises at least one of the following:

Swing the limb, torso or head at a set frequency;

Swing the limb, torso or head at a set angular velocity;

Swing the limb, torso or head in a set pattern;

Swing the limb, torso or head at a set angle range.
The control device according to claim 29, wherein the making of the motion trajectory according to the fourth setting manner comprises at least one of the following:

Swing yourself at the set frequency;

Swing yourself at the set angular velocity;

Move by the setting graphic;

Move by the set angle range.
The control device according to claim 28, wherein the prompting manner of prompting the following object to make the second preset action comprises at least one of the following:

A voice prompt is issued to the following object, a second preset motion graphic is displayed to the following object, the following object is prompted by the vibration mode, and a prompt signal is sent to the following object.
The control device according to claim 28, wherein said instructions, when executed by said processor, further cause said processor: after said mobile device recognizes and confirms following object, following a prompt to target: prompting target Follow the normal, prompt the target to follow the exception, prompt the target to find the follow object or prompt the target to follow the object is lost.
The control device according to claim 33, wherein the prompting manner of following the prompt to the target comprises at least one of the following:

A voice prompt is sent to the target to show the target a normal follow-up, follow an abnormality, or follow the lost object, and alert the target by vibration.
The control device according to any one of claims 18 to 34, further comprising at least one of the following action recognition units:

An acceleration sensor for identifying an amount of acceleration change of the action of the target;

An angular velocity sensor for identifying an angular velocity variation of the motion of the target;

A displacement sensor that is used to identify the displacement change of the target.
The control device according to any one of claims 18 to 35, further comprising a positioning unit for transmitting location information of the control device to the mobile device.
The control device according to claim 28, wherein said instructions, when executed by said processor, further cause said processor: after said mobile device recognizes and confirms following object, sends end-follow command to said mobile device .
A follow system that includes:

a control device, configured to recognize an action of the target, and issue a follow command to the movable device when the action of the target is the first preset action;

The mobile device is configured to receive the follow command and perform a follow operation according to the follow command.
The following system according to claim 38, wherein said first preset action comprises at least one of the following:

When the target is an animal, a mobile robot, an animal simulation robot or a human, swing the limb, torso or head according to the first setting.
The following system according to claim 38 or 39, wherein the movable device is further configured to recognize an action of the following object, such as confirming the following object for the second preset action.
The following system according to claim 38 or 39, wherein the following operation comprises the movable device approaching the control device with a preset movement parameter, the preset movement parameter comprising at least one of the following:

The relative position with the control device, the ground height, and the angle of alignment of the image recording device.
The following system according to any one of claims 38-41, wherein the movable device is further configured to: after receiving the end following command issued by the control device, ending the following.
The following system according to any one of claims 38-42, wherein the movable device is further configured to recognize an action of following the object, such as ending the following for the third setting action.
A computer readable storage medium storing instructions that, when executed by a processor, cause The processor performs the method according to any one of claims 1-17.