WO2017167239A1

WO2017167239A1 - Mobile control method, mobile electronic apparatus and mobile control system, and storage medium

Info

Publication number: WO2017167239A1
Application number: PCT/CN2017/078854
Authority: WO
Inventors: 陈子冲; 孙晓路; 蒲立; 王野
Original assignee: 纳恩博（北京）科技有限公司
Priority date: 2016-03-31
Filing date: 2017-03-30
Publication date: 2017-10-05
Also published as: CN105867433A

Abstract

A mobile control method, a mobile electronic apparatus and a mobile control system. The mobile control method is applied to a mobile electronic apparatus, in which a two-dimensional camera and a three-dimensional space detecting device are installed. The method comprises: obtaining, in response to a target position input operation performed on the basis of an image taken by the two-dimensional camera, the corresponding two-dimensional target position coordinates of the target position input operation on the image (S101); obtaining the depth value from the three-dimensional space detecting device aligned with the two-dimensional camera (S102); according to the two-dimensional target position coordinates, the depth value, the camera focal length and principal point of the two-dimensional camera, calculating and obtaining the three-dimensional target position coordinates of the two-dimensional target position coordinates relative to the movement center of the mobile electronic apparatus (S103); and controlling the movement center of the mobile electronic apparatus to move toward the three-dimensional target position coordinates. The technical solution described solves the technical problem of the simple robot being difficult to remotely control in the prior art, reducing the difficulty of controlling the simple robot.

Description

Mobile control method, mobile electronic device and mobile control system, storage medium

Cross-reference to related applications

The present invention is based on a Chinese patent application filed on Apr. 31, 2016, the entire disclosure of which is hereby incorporated by reference.

Technical field

The present invention relates to mobile control technologies, and in particular, to a mobile control method, a mobile electronic device, a mobile control system, and a storage medium.

Background technique

With the continuous development of science and technology, robots have developed rapidly. Robots are divided into simple robots and intelligent robots.

Among them, intelligent robots are the most complex robots, and they are also the machine friends that humans are most eager to make. However, it is not easy to create an intelligent robot. It is only for the machine to simulate the walking movement of human beings. Scientists have to pay dozens or even hundreds of years of hard work. For simple robots, the robot can be controlled by the operator constantly adjusting the remote control joystick.

Since the movement of the simple robot requires the operator to perform real-time tracking operation according to the actual motion situation, the control of the joystick also requires constant practice and exploration to accurately control, and the remote control is difficult. It can be seen that the simple robot in the related art has a technical problem that the remote control is difficult.

Summary of the invention

The embodiments of the present invention provide a mobile control method, a mobile electronic device, a mobile control system, and a storage medium, which are used to solve the technical problem that the remote control is difficult in the related art.

The embodiment of the invention provides a mobile control method, which is applied to a mobile electronic device, wherein the mobile electronic device is equipped with a two-dimensional camera and a three-dimensional space detecting device, and the method includes:

And obtaining a two-dimensional target position coordinate corresponding to the target position input operation on the image in response to a target position input operation based on an image captured by the two-dimensional camera;

Obtaining a depth value of the three-dimensional space detecting device aligned with the two-dimensional camera;

Calculating, according to the two-dimensional target position coordinate, the depth value, the camera focal length of the two-dimensional camera, and the camera main point, obtaining a three-dimensional target position coordinate of the two-dimensional target position coordinate relative to a motion center of the mobile electronic device ;

Controlling a motion center of the mobile electronic device to move to the three-dimensional target position coordinate.

In the above solution, the calculating, according to the two-dimensional target position coordinate, the depth value, the camera focal length of the two-dimensional camera, and the camera main point, obtaining the motion of the two-dimensional target position coordinate relative to the mobile electronic device The center's 3D target position coordinates, including:

Obtaining, according to the depth value, the two-dimensional target position coordinate, the camera focal length of the two-dimensional camera, and the camera main point, obtaining a three-dimensional relative position coordinate of the two-dimensional target position coordinate with respect to the optical center of the two-dimensional camera;

And multiplying a motion center of the mobile electronic device with respect to a spatial transformation matrix of the two-dimensional camera optical center and the three-dimensional relative position coordinates to obtain the three-dimensional target position coordinates.

In the above solution, the calculating, according to the depth value, the two-dimensional target position coordinate, the camera focal length of the two-dimensional camera, and the camera main point, obtaining the two-dimensional target position coordinate relative to the two-dimensional camera optical center The three-dimensional optical center position coordinates, including:

The three-dimensional relative position coordinate P is obtained by the following formula:

P=(z*(a-cx)/f,z*(b-cy)/f,z);

Where z denotes the depth value, (a, b) denotes the two-dimensional target position coordinate, (cx, cy) denotes the camera main point, and f denotes the camera focal length.

In the above solution, the obtaining the three-dimensional space detecting device is aligned with the two-dimensional camera Depth values, including:

When the three-dimensional space detecting device is a three-dimensional camera, the depth value is obtained by detecting the three-dimensional camera; or

When the three-dimensional space detecting device is an inertial detecting device, the ordinate in the three-dimensional target position coordinate is used as the height coordinate of the motion center of the mobile electronic device, and the depth value is constrained.

In the above solution, after controlling the motion center of the mobile electronic device to move to the three-dimensional target position coordinate, the method further includes:

Detecting whether there is an obstacle in the moving direction of the mobile electronic device;

If present, when the distance between the mobile electronic device and the obstacle is less than a first threshold distance, adjusting a moving direction of the mobile electronic device to a direction of the obstacle-free object;

After moving the second threshold distance in the adjusted direction, detecting whether there is an obstacle between the current position of the mobile electronic device and the three-dimensional target position coordinate;

If there is no obstacle between the current position and the three-dimensional target position coordinate, the mobile electronic device is pulled to move from the current position to the three-dimensional target position coordinate.

In the above solution, when the obtaining the target two-dimensional target position coordinates on the image, the method further includes:

Resetting a position coordinate of the mobile electronic device, and obtaining, by the three-dimensional space detecting device, a three-dimensional optical center position coordinate of a camera optical center of the two-dimensional camera;

Obtaining, according to the three-dimensional optical center position coordinates, a three-dimensional current coordinate of a motion center of the mobile electronic device with respect to a position coordinate after resetting;

The controlling the motion center of the mobile electronic device to move to the three-dimensional target position coordinate comprises:

And dragging, by the three-dimensional current coordinate and the three-dimensional target position coordinate, the mobile electronic device to move to the three-dimensional target position coordinate.

In the above solution, the response is based on a target position input operation performed by the image captured by the two-dimensional camera, and the coordinates of the two-dimensional target position corresponding to the target position input operation on the image are obtained, including:

Sending an image captured by the two-dimensional camera to a remote control device corresponding to the mobile electronic device, and displaying the image by the remote control device;

Receiving the target position input operation fed back by the remote control device;

And in response to the target position input operation, obtaining a two-dimensional target position coordinate corresponding to the target position input operation on the image.

In the above solution, the target position input operation includes: a touch operation on the image, or a voice or text input operation for calibrating the target position on the image.

The embodiment of the present invention further provides a mobile electronic device, wherein the mobile electronic device is equipped with a two-dimensional camera and a three-dimensional space detecting device, and the mobile electronic device further includes:

a response unit configured to obtain a two-dimensional target position coordinate corresponding to the target position input operation on the image in response to a target position input operation based on an image captured by the two-dimensional camera;

An acquiring unit configured to obtain a depth value of the three-dimensional space detecting device aligned with the two-dimensional camera;

a calculating unit configured to calculate, according to the two-dimensional target position coordinate, the depth value, a camera focal length of the two-dimensional camera, and a camera principal point, the two-dimensional target position coordinate is obtained relative to a motion center of the mobile electronic device Three-dimensional target position coordinates;

And a control unit configured to control a motion center of the mobile electronic device to move to the three-dimensional target position coordinate.

In the above solution, the calculating unit includes:

a first calculating subunit configured to calculate the two-dimensional target position coordinate according to the depth value, the two-dimensional target position coordinate, a camera focal length of the two-dimensional camera, and a camera principal point a three-dimensional relative position coordinate of the optical center of the two-dimensional camera;

And a second calculating subunit configured to multiply a motion transformation center of the mobile electronic device with respect to the spatial transformation matrix of the two-dimensional camera optical center and the three-dimensional relative position coordinates to obtain the three-dimensional target position coordinates.

In the above solution, the first calculating subunit is configured to obtain the three-dimensional relative position coordinate P by the following formula:

P=(z*(a-cx)/f,z*(b-cy)/f,z);

In the above solution, the obtaining unit is configured to:

When the three-dimensional space detecting device is a three-dimensional camera, the depth value is obtained by detecting the three-dimensional camera;

When the three-dimensional space detecting device is an inertial detecting device, it is assumed that the ordinate in the three-dimensional target position coordinate is equal to the height coordinate of the motion center of the mobile electronic device, and the constraint solves the depth value.

In the above solution, the mobile electronic device further includes:

a detecting unit configured to detect whether an obstacle exists in a moving direction of the mobile electronic device after controlling a motion center of the mobile electronic device to move to the three-dimensional target position coordinate;

The adjusting unit is configured to adjust a moving direction of the mobile electronic device to be when the distance between the mobile electronic device and the obstacle is less than a first threshold distance when the detection result of the detecting unit is present The direction of obstacles;

The detecting unit is further configured to: after moving the second threshold distance in the adjusted direction, detecting whether there is an obstacle between the current position of the mobile electronic device and the three-dimensional target position coordinate;

a traction unit configured to: when the detecting unit detects that there is no obstacle between the current position and the three-dimensional target position coordinate, to pull the mobile electronic device from the current position to the three-dimensional target position coordinate motion.

In the above solution, the mobile electronic device further includes:

a resetting unit configured to reset a position coordinate of the mobile electronic device when the corresponding two-dimensional target position coordinates on the image are input by the target position input operation, and obtain the location by the three-dimensional space detecting device The three-dimensional optical center position coordinate of the camera optical center of the two-dimensional camera;

The acquiring unit is further configured to: obtain, according to the three-dimensional optical center position coordinates, a three-dimensional current coordinate of a motion center of the mobile electronic device with respect to a position coordinate after resetting;

The control unit is further configured to: drag the mobile electronic device to move to the three-dimensional target position coordinate according to the three-dimensional current coordinate and the three-dimensional target position coordinate.

The embodiment of the invention further provides a mobile control system, comprising: a remote control device and a mobile electronic device, wherein the mobile electronic device is configured with a two-dimensional camera and a three-dimensional space detecting device;

The remote control device is configured to: receive an image of an environment in which the mobile electronic device is captured by the two-dimensional camera and display, obtain a target position input operation performed on the image, and input the target position Operation feedback to the mobile electronic device;

The mobile electronic device is configured to: receive the target position input operation fed back by the remote control device, obtain a corresponding two-dimensional target position coordinate of the target position input operation on the image; obtain the three-dimensional space detecting device and a depth value of the two-dimensional camera alignment; calculating, according to the two-dimensional target position coordinate, the depth value, a camera focal length of the two-dimensional camera, and a camera principal point, obtaining the two-dimensional target position coordinate relative to the movement a three-dimensional target position coordinate of a motion center of the electronic device; controlling a motion center of the mobile electronic device to move to the three-dimensional target position coordinate.

The embodiment of the invention further provides a mobile electronic device, including:

a two-dimensional camera, a three-dimensional space detecting device and a processor;

The processor is configured to obtain a two-dimensional target position coordinate corresponding to the target position input operation on the image in response to a target position input operation based on an image captured by the two-dimensional camera;

The processor is further configured to obtain a depth value of the three-dimensional space detecting device aligned with the two-dimensional camera;

The processor is further configured to calculate, according to the two-dimensional target position coordinates, the depth value, a camera focal length of the two-dimensional camera, and a camera principal point, the two-dimensional target position coordinates are obtained relative to the mobile electronic device The three-dimensional target position coordinates of the motion center;

The processor is further configured to control a motion center of the mobile electronic device to move to the three-dimensional target position coordinate.

The embodiment of the present invention further provides a storage medium, where the executable medium is stored with an executable instruction, and the executable instruction is used to execute the mobile control method provided by the embodiment of the present invention.

The above one or more technical solutions in the embodiments of the present invention have at least the following technical effects:

The embodiment of the present invention configures a two-dimensional camera and a three-dimensional space detecting device for the mobile electronic device, and the mobile electronic device obtains the two-dimensional target position coordinates of the target position in the image captured by the two-dimensional camera by inputting the response target position input operation, and obtains the three-dimensional space. Detecting a depth value of the device aligned with the two-dimensional camera to calculate a two-dimensional target position coordinate relative to a three-dimensional target of the motion center of the mobile electronic device according to the two-dimensional target position coordinate, the depth value, the camera focal length of the two-dimensional camera, and the camera main point Position coordinates, thereby controlling the motion center of the mobile electronic device to move to the three-dimensional target position coordinate, realizing the three-dimensional coordinate conversion of the two-dimensional coordinates in the image on the mobile electronic device to the actual environment, so that the user can perform the target position input on the image The operation controls the mobile electronic device to move to the corresponding spatial position, and does not require the user to manipulate the direction control lever of the mobile electronic device in real time, thereby solving the technical problem that the remote control of the simple robot exists in the related art, and reducing the control difficulty of the simple robot.

DRAWINGS

1 is a schematic flowchart of a mobile control method according to an embodiment of the present invention;

2 is a schematic flowchart of a mobile electronic device avoiding an obstacle according to an embodiment of the present invention;

3 is a schematic diagram of a mobile electronic device according to an embodiment of the present invention;

4 is a schematic diagram of a mobile control system according to an embodiment of the present invention;

FIG. 5 is a schematic structural diagram of hardware of a mobile electronic device according to an embodiment of the present invention.

detailed description

In the technical solution provided by the embodiment of the present invention, the user is provided with a function of converting the coordinates of the two-dimensional target position on the image into the coordinates of the three-dimensional target position, so that the user can perform the mobile control on the mobile electronic device. The image captured by the mobile electronic device performs a target position input operation to instruct the mobile electronic device to move to a corresponding spatial position, and the user does not need to perform real-time tracking operation according to the actual motion condition of the mobile electronic device, so as to solve the existence of the simple robot in the related art. The technical problem of remote control is difficult, and the control difficulty of the simple robot is reduced. The main implementation principles, specific implementation manners, and the corresponding beneficial effects that can be achieved by the technical solutions of the embodiments of the present invention are described in detail below with reference to the accompanying drawings.

Referring to FIG. 1 , an embodiment of the present invention provides a mobile control method, which is applied to a mobile electronic device, where a two-dimensional camera and a three-dimensional space detecting device are installed, and the method includes:

S101: Obtain a two-dimensional target position coordinate corresponding to the target position input operation on the image in response to a target position input operation performed based on an image captured by the two-dimensional camera;

S102: Obtain a depth value of the three-dimensional space detecting device aligned with the two-dimensional camera;

S103: Calculating, according to the two-dimensional target position coordinate, the depth value, a camera focal length of the two-dimensional camera, and a camera main point, obtaining the two-dimensional target position coordinate relative to the mobile electronic The three-dimensional target position coordinates of the motion center of the device;

S104: Control a motion center of the mobile electronic device to move to the three-dimensional target position coordinate.

The mobile electronic device provided by the embodiment of the present invention may be a mobile robot and can communicate with a remote control device. The remote control device may be a dedicated remote controller that matches the mobile robot, or a common electronic device that can communicate with the mobile robot, such as a smart phone, a tablet (PAD), a smart watch, and the like.

The mobile electronic device is equipped with a two-dimensional camera and a three-dimensional space detecting device. The two-dimensional camera may be a color mode RGB camera, and the three-dimensional space detecting device may be a 3D camera or an inertial detection unit (IMU). The inertial detection device IMU is a device for measuring the three-axis attitude angle (or angular velocity) and acceleration of an object, and is mostly used in devices that require motion control, such as automobiles and robots.

The mobile electronic device captures an image of the environment in which it is located (also referred to as a view image of the mobile electronic device) through its two-dimensional camera, and the image may be a real-time image, that is, a frame image in the video stream, and the image is transmitted to the remote control through the image transmission signal. The device, or display, is shared with the user on the display of the mobile electronic device.

When the mobile electronic device transmits the image captured by the two-dimensional camera to the remote control device corresponding to the mobile electronic device and displays the image through the remote control device, the user can perform a target position input operation on the image on the remote control device, and the remote control device detects When the target position input operation of the image is obtained, the target position input operation is fed back to the mobile electronic device, and the mobile electronic device receives the target position input operation fed back by the remote control device and responds to obtain the corresponding two-dimensional corresponding position input operation on the image. Target position coordinates.

For example, if the mobile electronic device is an electric balance car and the remote control device is a smart phone, the electric balance car takes a photo directly in front of its current location through the two-dimensional camera and transmits it to the user's mobile phone, and the user views the mobile phone through the mobile phone. After the image, click on the "elevator port" on the image. Performing the target position input operation, the mobile phone feeds back the operation to the electric balance car, and the electric balance car receives the feedback of the mobile phone and responds to the target position input operation to obtain the two-dimensional target position coordinates of the “elevator port” in the image, thereby “elevator port” The two-dimensional target position coordinates are converted into the coordinates of the three-dimensional target position where the "elevator port" is actually located, and then run to the "elevator port". Of course, the touch display provided on the mobile electronic device can also achieve the same function as the remote control, and the user can perform the target position input operation directly on the mobile electronic device, instructing the mobile electronic device to automatically move to the target without the user's own control. Rod.

After the user performs a target position input operation on the image captured by the two-dimensional camera, the mobile electronic device performs S101 to respond to the target position input operation, and obtains a corresponding two-dimensional target position coordinate of the target position input operation on the image. For example, when the user performs the target position input operation, the two-dimensional target position coordinates corresponding to the target position input operation can be obtained by the following three methods:

In the first step, when the user performs a touch operation on the target position in the image displayed on the touch display screen, the mobile electronic device obtains the coordinates of the two-dimensional touch position of the user on the touch display screen in response to the target position input operation. Then, according to the relative positional relationship between the image and the touch display screen and the obtained two-dimensional touch position coordinates, the two-dimensional target position coordinates corresponding to the target position input operation on the image are obtained. For example, if the image captured by the two-dimensional camera in the mobile electronic device is displayed in the touch display screen, the offset vector between the center point of the touch display screen and the center point of the image is (c, d), if The coordinates of the two-dimensional touch position obtained by the detection are (e, f), and then the coordinates of the two-dimensional target position are obtained as (a, b): a=e+c, b=d+f.

In the second mode, when the user performs a voice input operation on the target position in the image captured by the two-dimensional camera, the mobile electronic device obtains the voice data input by the user in response to the voice input operation, and converts the voice data into a mobile electronic device that can be recognized. The character data is further searched for the target position matching the character data from the image captured by the two-dimensional camera by image recognition, and the two-dimensional target position coordinates of the target position in the image are obtained. For example: suppose the user voice inputs "pot", then the mobile electronic device converts the voice data "pot" into the character "pot". And through the image recognition, the "flower pot" in the image and its two-dimensional target position coordinates (a, b) in the image.

In the third mode, when the user performs a text input operation on the target position in the image captured by the two-dimensional camera, the mobile electronic device obtains the text data in response to the text input operation, and finds and images from the image captured by the two-dimensional camera through image recognition. The text data matches the target position and obtains the coordinates of the two-dimensional target position of the target position in the image. For example: suppose the user text input target position is "flower pot", then the "flower pot" in the image searched by the image recognition and obtain the two-dimensional target position coordinates (a, b) of the "flower pot" in the image.

In order to make the positioning of the mobile electronic device more accurate and the movement of the mobile electronic device more accurate, the embodiment of the present invention resets the position coordinates of the mobile electronic device when obtaining the two-dimensional target position coordinates of the target position. The reset is (0, 0, 0), and the three-dimensional optical center position coordinates (x1, y1, z1) of the camera optical center of the two-dimensional camera of the mobile electronic device are obtained by the three-dimensional space detecting device of the mobile electronic device. Then, according to the three-dimensional optical center position coordinates of the camera optical center of the two-dimensional camera, the three-dimensional current coordinates (x, y, z) of the motion center of the mobile electronic device with respect to the position coordinates after the reset are obtained. For example, according to the positional relationship T of the camera optical center of the two-dimensional camera relative to the movement center of the mobile electronic device (considering the servo position of each joint of the mobile electronic device), the three-dimensional position of the mobile electronic device motion center relative to the reset position coordinate is obtained. The current coordinates (x, y, z) = (x1, y1, z1) * T.

After S101, S102 is performed to obtain a depth value z in which the three-dimensional space detecting device of the mobile electronic device is aligned with the two-dimensional camera. When the three-dimensional space detecting device is a three-dimensional camera, that is, a 3D camera, the depth value z=Depth(a, b) can be obtained directly by the 3D camera. When the three-dimensional space detecting device is an inertial detecting device, it is assumed that the ordinate in the three-dimensional target position coordinate is equal to the height coordinate of the moving center of the mobile electronic device, since the height coordinate of the mobile electronic device is a known number h when it leaves the factory, The ordinate of the three-dimensional target position coordinates y _t =F(z), F(z) is a function containing the depth value z. For this purpose, h=F(z) can be used as a constraint to solve the depth value z, ie, solved by the constraint Get the depth value z.

After obtaining the depth value z aligned by the three-dimensional space detecting device and the two-dimensional camera, proceed to perform S103 to calculate the two-dimensional target position coordinate relative moving electron according to the two-dimensional target position coordinate, the depth value, the camera focal length of the two-dimensional camera, and the camera main point. The 3D target position coordinates of the motion center of the device. The center of motion of the mobile electronic device can be the chassis motion center of the mobile electronic device. For example, the three-dimensional relative position coordinates of the two-dimensional target position coordinates relative to the optical center of the two-dimensional camera may be calculated according to the depth value, the two-dimensional target position coordinates, the camera focal length of the two-dimensional camera, and the camera main point; Then, the three-dimensional target position coordinates are obtained by multiplying the motion center of the mobile electronic device with the spatial transformation matrix of the two-dimensional camera optical center and the three-dimensional optical center position coordinates.

Wherein, the two-dimensional target position coordinate of the mobile electronic device is calculated relative to the three-dimensional relative position coordinate P of the two-dimensional camera optical center by the following formula (1):

P=(z*(a-cx)/f,z*(b-cy)/f,z);formula (a)

Where z is the depth value at which the three-dimensional space detecting device is aligned with the two-dimensional camera, (a, b) represents the two-dimensional target position coordinates, (cx, cy) represents the camera main point of the two-dimensional camera, and f represents the camera of the two-dimensional camera. focal length. If the spatial transformation matrix of the motion center of the mobile electronic device relative to the optical center of the two-dimensional camera is T, then the three-dimensional target position coordinates (x _t , y _t , z _t )=T*P.

After obtaining the three-dimensional target position coordinates, S104 is executed to control the motion center of the mobile electronic device to move to the three-dimensional target position coordinate. Before the motion, the position coordinates of the mobile electronic device are usually reset to obtain the three-dimensional current coordinates of the mobile electronic device motion center. When moving, according to the three-dimensional current coordinates (x, y, z) of the mobile electronic device motion center and the target position The three-dimensional target position coordinates (x _t , y _t , z _t ) pull the mobile electronic device to move to the three-dimensional target position coordinates (x _t , y _t , z _t ). In addition, when the mobile electronic device is in motion, the chassis of the mobile electronic device can be controlled to operate at a three-dimensional target position coordinate by a closed loop automatic control technology, that is, PID.

During the movement of the mobile electronic device, the following steps can also be performed, as shown in FIG. 2:

S201: Detect whether there is an obstacle in the moving direction of the mobile electronic device. As passed The infrared scan determines whether there is an obstacle in front, or captures an image of the moving direction through the camera, and determines whether there is an obstacle in front by image recognition. If there is no obstacle in the moving direction of the mobile electronic device, the process proceeds to S202; otherwise, if there is an obstacle in the moving direction of the mobile electronic device, the process proceeds to S203.

S202: Continue to move along the current motion direction until reaching a three-dimensional target position coordinate or a distance from the three-dimensional target position coordinate is less than a set distance, the mobile travel end ends, and a notification message that the target position has arrived is sent to the user. The set distance can be set by the user according to his actual needs, or the system default value, such as 50 cm (cm).

S203: When the distance between the mobile electronic device and the obstacle is less than the first threshold distance, adjust the moving direction of the mobile electronic device to the direction of the obstacle, and continue to move in the adjusted direction. The first threshold distance may be performed according to the size of the mobile electronic device, and at least ensure that the mobile electronic device can turn or turn within a first threshold distance. For example, if the size of the mobile electronic device is 50 cm*60 cm, the first threshold The distance may be 60 cm, 100 cm, 150 cm or the like.

In one embodiment, considering that the mobile electronic device has a certain speed during operation, it cannot be determined instantaneously, and a certain braking distance is required, so the value of the first threshold range may be determined according to the size of the mobile electronic device and the braking distance. For example, assuming that the mobile electronic device has a length of 60 cm and a braking distance of 40 to 100 cm, the first threshold distance can be set to 160 cm.

S204: After moving the second threshold distance in the adjusted direction, detecting whether there is an obstacle between the current position of the mobile electronic device and the three-dimensional target position coordinate. If there is no obstacle between the current position and the three-dimensional target position coordinate, the jump proceeds to S202. Conversely, if there is still an obstacle between the current position and the coordinates of the three-dimensional target position, the jump is performed S203-S204 until the distance to the three-dimensional target position coordinate or the distance between the three-dimensional target position coordinates is less than the set distance, the movement stroke End and send the user a notification message that the target location has arrived.

The following is a complete example of the mobile control method provided by the embodiment of the present invention through two specific examples. The implementation process is described in detail.

Example 1

Assume that the mobile electronic device is an electric balance car, and the electric balance car is equipped with a touch display screen, a red green blue (RGB, Red Green Blue) camera and a 3D camera. Among them, the viewing angle of the 3D camera is similar to that of the RGB camera and substantially coincides. The mobile control process is as follows:

Step 1. The touch screen displayed on the electric balance car displays the field of view image captured by the RGB camera.

Step 2: The user clicks on a target point of interest in the field of view image of the electric balance vehicle, and the electric balance vehicle obtains the RGB coordinates (a, b) of the point in the image coordinate system to start a round of movement control process.

Step 3: Reset the current position coordinate of the electric balance car to (0, 0, 0), open the visual mileage calculation method based on the 3D camera, calculate the three-dimensional optical center position coordinates of the RGB camera camera optical center, and according to the RGB camera camera The optical center is relatively balanced with the positional relationship of the motion center of the chassis (considering the servo position of each joint of the robot), and the three-dimensional current coordinates (x, y, z) of the chassis motion center with respect to the position coordinates after the reset are obtained.

Step 4: The electric balance vehicle calculates the position (x _t , y _t , z _t ) of the target point from the center of the chassis motion from (a, b), as follows:

Through the 3D camera, the depth value z=Depth(a,b) aligned with the RGB camera, the camera focal length of the RGB camera is f, and the camera main point of the RGB camera is (cx, cy). Then, the 3D position and the three-dimensional relative position coordinate of the target point with respect to the RGB camera optical center are p=(z*(a-cx)/f, z*(b-cy)/f, z). It is known that the spatial transformation matrix of the motion center of the robot chassis relative to the optical center of the RGB camera is T (considering the servo position of each joint of the robot), then (x _t , y _t , z _t )=T*p.

Step 5. According to the current coordinate (x, y, z) of the electric balance car and the target point coordinates (x _t , y _t , z _t ), the chassis of the robot is pulled by a proportional-integral-derivative (PID) controller. Target point movement.

Step 6. (Optional) If the obstacle-reducing sensor is on the electric balance car during the movement (3D photo Like the head, ultrasonic sensor, etc., if an obstacle appears in the direction of motion, the electric balance car starts the obstacle avoidance logic, brakes and tries to adjust the direction of motion to the direction without the obstacle until there is no obstacle in the direction of the traction to the target point.

Step 7. Repeat steps 5 and 6 until the distance difference between the current coordinates of the robot and the three-dimensional target position coordinates of the target point is less than the set threshold. Then the movement control is completed, the electric balance vehicle stops, and the user is notified.

Step 8. Give feedback to the user on the touch display interface of the electric balance vehicle, prompting to reach the target position, and allowing the user to click the target point again to start a new round of movement control. Of course, in the control process of steps 5 and 6, the user can also click the new target point position to adjust the target position at any time.

Example 2

Assuming that the mobile electronic device is a robot, the robot is equipped with an RGB camera and an IMU. The robot can communicate with a remote controller and is controlled by a remote controller, which is a touch screen mobile phone or a customized remote controller with a touch screen. The mobile control process is as follows:

Step 1. The robot captures its own field of view image through the RGB camera and transmits it to the remote controller for display by wireless transmission.

Step 2: The user clicks on a target point of interest in the field of view image of the robot, and the robot obtains the RGB coordinates (a, b) of the point in the image coordinate system to start a round of movement control process.

Step 3: Reset the current position coordinate of the robot to (0, 0, 0), open the visual mileage calculation method based on RGB camera + IMU, calculate the three-dimensional optical center position coordinates of the optical center of the RGB camera camera, and according to the RGB camera camera The optical center is relatively balanced with the positional relationship of the motion center of the chassis (considering the servo position of each joint of the robot), and the three-dimensional current coordinates (x, y, z) of the chassis motion center with respect to the position coordinates after the reset are obtained.

Step 4: The robot calculates the position (x _t , y _t , z _t ) of the target point from the center of the chassis motion from (a, b), as follows:

The depth value Depth(a,b) of the IMU aligned with the RGB camera is unknown, denoted as z; the camera focal length of the RGB camera is f, and the camera main point of the RGB camera is (cx, cy). Then, the 3D position and the three-dimensional relative position coordinate of the target point with respect to the RGB camera optical center are p=(z*(a-cx)/f, z*(b-cy)/f, z). It is known that the spatial transformation matrix of the motion center of the robot chassis relative to the optical center of the RGB camera is T (considering the servo position of each joint of the robot), then (x _t , y _t , z _t )=T*p. Assuming that the point clicked by the user is on the ground, y _t = chassis motion center height coordinate h can be considered, whereby the constraint solves the unknown z, and thus (x _t , y _t , z _t ).

Step 5. According to the current coordinates (x, y, z) of the robot and the coordinates of the target point (x _t , y _t , z _t ), the PID controller is used to pull the chassis of the robot to the target point.

Step 6. (Optional) During the movement, if the robot has obstacle avoidance sensors (IMU, ultrasonic sensors, etc.) and finds obstacles in the direction of motion, the robot starts the obstacle avoidance logic, brakes and tries to adjust to the direction without obstacles. The direction of motion until there is no obstacle in the direction of the traction toward the target point.

Step 7. Repeat steps 5 and 6 until the distance difference between the current coordinates of the robot and the three-dimensional target position coordinates of the target point is less than the set threshold. Then the movement control is completed, the robot stops, and the remote controller is notified.

Step 8. Give feedback to the user on the touch display interface of the remote controller, prompting to reach the target position, and allowing the user to click the target point again to start a new round of movement control. Of course, in the control process of steps 5 and 6, the user can also click the new target point position to adjust the target position at any time.

The three-dimensional coordinates of the two-dimensional coordinates in the image on the mobile electronic device to the actual environment are converted by the above embodiment, so that the user can control the mobile electronic device to move to the corresponding spatial position by performing a target position input operation on the image, The user needs to manipulate the direction joystick of the mobile electronic device in real time, and solves the technical problem of the remote control difficulty of the simple robot in the related art, and reduces the control difficulty of the simple robot.

Referring to FIG. 3, an embodiment of the present invention is directed to a mobile control method provided by the foregoing, and further provides a mobile electronic device. The mobile electronic device is equipped with a two-dimensional camera and three-dimensional space detection The device further includes:

The response unit 301 is configured to obtain a two-dimensional target position coordinate corresponding to the target position input operation on the image in response to a target position input operation based on an image captured by the two-dimensional camera;

The obtaining unit 302 is configured to obtain a depth value of the three-dimensional space detecting device aligned with the two-dimensional camera;

The calculating unit 303 is configured to calculate, according to the two-dimensional target position coordinate, the depth value, the camera focal length of the two-dimensional camera, and the camera main point, the motion of the two-dimensional target position coordinate relative to the mobile electronic device The coordinates of the three-dimensional target position of the center;

The control unit 304 is configured to control the motion center of the mobile electronic device to move to the three-dimensional target position coordinate.

In a specific implementation process, the calculating unit 303 includes: a first calculating subunit and a second calculating subunit. The first calculating subunit is configured to calculate the two-dimensional target position coordinate relative to the two-dimensional camera light according to the depth value, the two-dimensional target position coordinate, the camera focal length of the two-dimensional camera, and the camera main point The three-dimensional relative position coordinates of the heart. The second computing subunit is configured to multiply a motion center of the mobile electronic device with respect to a spatial transformation matrix of the two-dimensional camera optical center and the three-dimensional relative position coordinates to obtain the three-dimensional target position coordinates.

The first calculating subunit is configured to obtain the three-dimensional relative position coordinate P by the following formula: P=(z*(a-cx)/f, z*(b-cy)/f, z); Where z denotes the depth value, (a, b) denotes the two-dimensional target position coordinate, (cx, cy) denotes the camera main point, and f denotes the camera focal length.

In a specific implementation process, the acquiring unit 302 is configured to: when the three-dimensional space detecting device is a three-dimensional camera, obtain the depth value by detecting the three-dimensional camera; or when the three-dimensional space detecting device is inertial detecting The device uses the ordinate in the coordinate of the three-dimensional target position as the height coordinate of the motion center of the mobile electronic device, and the constraint solves the depth Degree value.

In an embodiment, the mobile electronic device may further include: a detecting unit, an adjusting unit, and a pulling unit. The detecting unit is configured to detect whether there is an obstacle in the moving direction of the mobile electronic device after controlling the motion center of the mobile electronic device to move to the three-dimensional target position coordinate; the adjusting unit is configured to be in the detecting unit When the detection result is present, when the distance between the mobile electronic device and the obstacle is less than the first threshold distance, the moving direction of the mobile electronic device is adjusted to the direction of the obstacle-free object; the detecting unit further After the second threshold distance is moved in the adjusted direction, detecting whether there is an obstacle between the current position of the mobile electronic device and the three-dimensional target position coordinate; the traction unit is configured to be detected by the detecting unit. When there is no obstacle between the current position and the three-dimensional target position coordinate, the mobile electronic device is pulled to move from the current position to the three-dimensional target position coordinate.

In one embodiment, the mobile electronic device further includes: a reset unit. The resetting unit is configured to reset the position coordinates of the mobile electronic device when the two-dimensional target position coordinates corresponding to the image input operation on the image is obtained, and obtain the location by the three-dimensional space detecting device The three-dimensional optical center position coordinate of the camera optical center of the two-dimensional camera; the acquiring unit 302 is further configured to: obtain the motion center of the mobile electronic device relative to the reset position coordinate according to the three-dimensional optical center position coordinate The three-dimensional current coordinate; the control unit 304 is further configured to: drag the mobile electronic device to move to the three-dimensional target position coordinate according to the three-dimensional current coordinate and the three-dimensional target position coordinate.

Regarding the mobile electronic device provided by the foregoing embodiments of the present invention, the specific manner in which the operations are performed by the respective units has been described in detail in the embodiments related to the method, and will not be described in detail herein.

Referring to FIG. 4, a mobile control system provided by the foregoing embodiment of the present invention further provides a mobile control system, including: a remote control device 401 and a mobile electronic device 402. The mobile electronic device 402 is configured with a two-dimensional camera and a three-dimensional space detecting device;

The remote control device 401 is configured to receive an image of an environment in which the mobile electronic device is captured by the two-dimensional camera and display, obtain a target position input operation performed on the image, and input the target position into an operation. Feedback to the mobile electronic device 402;

The mobile electronic device 402 is configured to: receive the target position input operation fed back by the remote control device, obtain two-dimensional target position coordinates corresponding to the target position input operation on the image; and obtain the three-dimensional space detecting device a depth value aligned with the two-dimensional camera; calculating, according to the two-dimensional target position coordinate, the depth value, a camera focal length of the two-dimensional camera, and a camera principal point, obtaining the two-dimensional target position coordinate relative to the The three-dimensional target position coordinates of the motion center of the mobile electronic device 402; controlling the motion center of the mobile electronic device 402 to move toward the three-dimensional target position coordinates.

Regarding the mobile control system in the above embodiment, the specific manner of the remote control device and the mobile electronic device has been described in detail in the embodiment related to the method, and will not be explained in detail herein.

Please refer to FIG. 5, which is a schematic diagram of an optional hardware structure of a mobile electronic device 500 according to an embodiment of the present invention.

The processor 501, the two-dimensional camera 502, the 3D camera 503, the inertial detection device 504, and the memory 505 may be connected by a bus between the components, and the component transmits data to the processor 501 via the bus and receives an instruction from the processor 501. The 3D camera 503 and the inertial detection device 504 can be implemented in one or all of the mobile electronic devices.

The processor 501 is configured to obtain a two-dimensional target position coordinate corresponding to the target position input operation on the image in response to a target position input operation based on an image captured by the two-dimensional camera 502;

The processor 501 is further configured to obtain the 3D camera 503 and the two-dimensional camera a depth value of 502 alignment, or a depth value at which the inertial detection device 504 is aligned with the two-dimensional camera 502;

The processor 501 is further configured to calculate, according to the two-dimensional target position coordinate, the depth value, a camera focal length of the two-dimensional camera 502, and a camera main point, the two-dimensional target position coordinate relative to the movement The three-dimensional target position coordinates of the motion center of the electronic device;

The processor 501 is further configured to control a motion center of the mobile electronic device to move to the three-dimensional target position coordinate.

The embodiment of the present invention further provides a storage medium storing executable instructions for executing the foregoing mobile control method, for example, the mobile control method shown in any of FIGS. 1 to 2 . The storage medium of the embodiment of the present invention may be a storage medium such as an optical disk, a hard disk, or a magnetic disk, and may be a non-transitory storage medium.

The following technical effects can be achieved by using one or more technical solutions in the embodiments of the present invention:

Those skilled in the art will appreciate that embodiments of the present invention can be provided as a method, system, Or a computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware. Moreover, the invention can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.

The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (system), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine for the execution of instructions for execution by a processor of a computer or other programmable data processing device. Means for implementing the functions specified in one or more of the flow or in a block or blocks of the flow chart.

The computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device. The apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.

These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device. The instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

While the preferred embodiment of the invention has been described, it will be understood that Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all modifications and modify.

It is apparent that those skilled in the art can make various modifications and variations to the invention without departing from the spirit and scope of the invention. Thus, it is intended that the present invention cover the modifications and modifications of the invention

Industrial applicability

The invention discloses a mobile control method, a mobile electronic device, a mobile control system and a storage medium. The mobile control method is applied to a mobile electronic device. The mobile electronic device is equipped with a two-dimensional camera and a three-dimensional space detecting device, and the method comprises: responding The target position input operation based on the image captured by the two-dimensional camera obtains the coordinate of the two-dimensional target position corresponding to the target position input operation on the image; obtains the depth value of the three-dimensional space detecting device aligned with the two-dimensional camera; according to the two-dimensional target position coordinate The depth value, the camera focal length of the two-dimensional camera, and the camera main point are calculated, and the two-dimensional target position coordinates are calculated relative to the three-dimensional target position coordinates of the motion center of the mobile electronic device; and the motion center of the mobile electronic device is controlled to move to the three-dimensional target position coordinate. Through the above technical solution, the technical problem that the remote control of the simple robot exists in the related art is solved, and the control difficulty of the simple robot is reduced.

Claims

A mobile control method is applied to a mobile electronic device, wherein the mobile electronic device is equipped with a two-dimensional camera and a three-dimensional space detecting device, and the method includes:

And obtaining a two-dimensional target position coordinate corresponding to the target position input operation on the image in response to a target position input operation based on an image captured by the two-dimensional camera;

Obtaining a depth value of the three-dimensional space detecting device aligned with the two-dimensional camera;

Calculating, according to the two-dimensional target position coordinate, the depth value, the camera focal length of the two-dimensional camera, and the camera main point, obtaining a three-dimensional target position coordinate of the two-dimensional target position coordinate relative to a motion center of the mobile electronic device ;

Controlling a motion center of the mobile electronic device to move to the three-dimensional target position coordinate.
The method of claim 1, wherein the calculating the two-dimensional target position coordinates relative to the two-dimensional target position coordinates, the depth value, the camera focal length of the two-dimensional camera, and the camera principal point The three-dimensional target position coordinates of the motion center of the mobile electronic device include:

Obtaining, according to the depth value, the two-dimensional target position coordinate, the camera focal length of the two-dimensional camera, and the camera main point, obtaining a three-dimensional relative position coordinate of the two-dimensional target position coordinate with respect to the optical center of the two-dimensional camera;

And multiplying a motion center of the mobile electronic device with respect to a spatial transformation matrix of the two-dimensional camera optical center and the three-dimensional relative position coordinates to obtain the three-dimensional target position coordinates.
The method of claim 2, wherein the calculating the two-dimensional target position coordinates relative to the depth value, the two-dimensional target position coordinates, the camera focal length of the two-dimensional camera, and the camera principal point The three-dimensional optical center position coordinates of the two-dimensional camera optical center include:

The three-dimensional relative position coordinate P is obtained by the following formula:

P=(z*(a-cx)/f,z*(b-cy)/f,z);

Where z represents the depth value, (a, b) represents the two-dimensional target position coordinate, (cx, cy) Indicates the camera's primary point, and f denotes the camera focal length.
The method of claim 1, wherein the obtaining the depth value of the three-dimensional space detecting device aligned with the two-dimensional camera comprises:

When the three-dimensional space detecting device is a three-dimensional camera, the depth value is obtained by detecting the three-dimensional camera;

When the three-dimensional space detecting device is the inertial detecting device IMU, the ordinate in the three-dimensional target position coordinate is used as the height coordinate of the motion center of the mobile electronic device, and the depth value is constrained.
The method of claim 1, wherein after controlling the motion center of the mobile electronic device to move to the three-dimensional target position coordinate, the method further comprises:

Detecting whether there is an obstacle in the moving direction of the mobile electronic device;

If present, when the distance between the mobile electronic device and the obstacle is less than a first threshold distance, adjusting a moving direction of the mobile electronic device to a direction of the obstacle-free object;

After moving the second threshold distance in the adjusted direction, detecting whether there is an obstacle between the current position of the mobile electronic device and the three-dimensional target position coordinate;

If there is no obstacle between the current position and the three-dimensional target position coordinate, the mobile electronic device is pulled to move from the current position to the three-dimensional target position coordinate.
The method of claim 1, wherein when the obtaining the target position input operation corresponds to the corresponding two-dimensional target position coordinates on the image, the method further comprises:

Resetting a position coordinate of the mobile electronic device, and obtaining, by the three-dimensional space detecting device, a three-dimensional optical center position coordinate of a camera optical center of the two-dimensional camera;

Obtaining, according to the three-dimensional optical center position coordinates, a three-dimensional current coordinate of a motion center of the mobile electronic device with respect to a position coordinate after resetting;

The controlling the motion center of the mobile electronic device to move to the three-dimensional target position coordinate comprises:

And dragging, by the three-dimensional current coordinate and the three-dimensional target position coordinate, the mobile electronic device to move to the three-dimensional target position coordinate.
The method according to any one of claims 1 to 6, wherein said responding is based on a target position input operation by an image captured by said two-dimensional camera, obtaining a two-dimensional corresponding to said target position input operation on said image Target location coordinates, including:

Sending an image captured by the two-dimensional camera to a remote control device corresponding to the mobile electronic device, and displaying the image by the remote control device;

Receiving the target position input operation fed back by the remote control device;

And in response to the target position input operation, obtaining a two-dimensional target position coordinate corresponding to the target position input operation on the image.
A mobile electronic device in which a two-dimensional camera and a three-dimensional space detecting device are installed, the mobile electronic device further comprising:

a response unit configured to obtain a two-dimensional target position coordinate corresponding to the target position input operation on the image in response to a target position input operation based on an image captured by the two-dimensional camera;

An acquiring unit configured to obtain a depth value of the three-dimensional space detecting device aligned with the two-dimensional camera;

a calculating unit configured to calculate, according to the two-dimensional target position coordinate, the depth value, a camera focal length of the two-dimensional camera, and a camera principal point, the two-dimensional target position coordinate is obtained relative to a motion center of the mobile electronic device Three-dimensional target position coordinates;

And a control unit configured to control a motion center of the mobile electronic device to move to the three-dimensional target position coordinate.
The mobile electronic device of claim 8 wherein said computing unit comprises:

a first calculating subunit configured to calculate the two-dimensional target position coordinate according to the depth value, the two-dimensional target position coordinate, a camera focal length of the two-dimensional camera, and a camera principal point a three-dimensional relative position coordinate of the optical center of the two-dimensional camera;

And a second calculating subunit configured to multiply a motion transformation center of the mobile electronic device with respect to the spatial transformation matrix of the two-dimensional camera optical center and the three-dimensional relative position coordinates to obtain the three-dimensional target position coordinates.
The mobile electronic device of claim 9, wherein the first computing subunit is configured to obtain the three-dimensional relative position coordinate P by the following formula:

P=(z*(a-cx)/f,z*(b-cy)/f,z);

Where z denotes the depth value, (a, b) denotes the two-dimensional target position coordinate, (cx, cy) denotes the camera main point, and f denotes the camera focal length.
The mobile electronic device of claim 8, wherein the acquisition unit is configured to:

When the three-dimensional space detecting device is a three-dimensional camera, the depth value is obtained by detecting the three-dimensional camera;

When the three-dimensional space detecting device is the inertial detecting device IMU, the ordinate in the three-dimensional target position coordinate is used as the height coordinate of the motion center of the mobile electronic device, and the depth value is constrained.
The mobile electronic device of claim 8, wherein the mobile electronic device further comprises:

a detecting unit configured to detect whether an obstacle exists in a moving direction of the mobile electronic device after controlling a motion center of the mobile electronic device to move to the three-dimensional target position coordinate;

The adjusting unit is configured to adjust a moving direction of the mobile electronic device to be when the distance between the mobile electronic device and the obstacle is less than a first threshold distance when the detection result of the detecting unit is present The direction of obstacles;

The detecting unit is further configured to: after moving the second threshold distance in the adjusted direction, detecting whether there is an obstacle between the current position of the mobile electronic device and the coordinate of the three-dimensional target position Object

a traction unit configured to: when the detecting unit detects that there is no obstacle between the current position and the three-dimensional target position coordinate, to pull the mobile electronic device from the current position to the three-dimensional target position coordinate motion.
The mobile electronic device of claim 8, wherein the mobile electronic device further comprises:

a resetting unit configured to reset a position coordinate of the mobile electronic device when the corresponding two-dimensional target position coordinates on the image are input by the target position input operation, and obtain the location by the three-dimensional space detecting device The three-dimensional optical center position coordinate of the camera optical center of the two-dimensional camera;

The acquiring unit is further configured to: obtain, according to the three-dimensional optical center position coordinates, a three-dimensional current coordinate of a motion center of the mobile electronic device with respect to a position coordinate after resetting;

The control unit is further configured to: drag the mobile electronic device to move to the three-dimensional target position coordinate according to the three-dimensional current coordinate and the three-dimensional target position coordinate.
A mobile control system includes: a remote control device and a mobile electronic device, wherein the mobile electronic device is configured with a two-dimensional camera and a three-dimensional space detecting device;

The remote control device is configured to: receive an image of an environment in which the mobile electronic device is captured by the two-dimensional camera and display, obtain a target position input operation performed on the image, and input the target position Operation feedback to the mobile electronic device;

The mobile electronic device is configured to: receive the target position input operation fed back by the remote control device, obtain a corresponding two-dimensional target position coordinate of the target position input operation on the image; obtain the three-dimensional space detecting device and a depth value of the two-dimensional camera alignment; calculating, according to the two-dimensional target position coordinate, the depth value, a camera focal length of the two-dimensional camera, and a camera principal point, obtaining the two-dimensional target position coordinate relative to the movement a three-dimensional target position coordinate of a motion center of the electronic device; controlling a motion center of the mobile electronic device to move to the three-dimensional target position coordinate.
A mobile electronic device comprising:

a two-dimensional camera, a three-dimensional space detecting device and a processor;

The processor is configured to obtain a two-dimensional target position coordinate corresponding to the target position input operation on the image in response to a target position input operation based on an image captured by the two-dimensional camera;

The processor is further configured to obtain a depth value of the three-dimensional space detecting device aligned with the two-dimensional camera;

The processor is further configured to calculate, according to the two-dimensional target position coordinates, the depth value, a camera focal length of the two-dimensional camera, and a camera principal point, the two-dimensional target position coordinates are obtained relative to the mobile electronic device The three-dimensional target position coordinates of the motion center;

The processor is further configured to control a motion center of the mobile electronic device to move to the three-dimensional target position coordinate.
A storage medium storing executable instructions for performing the movement control method according to any one of claims 1 to 7.