WO2021223611A1

WO2021223611A1 - Robot control method and apparatus, and robot and storage medium

Info

Publication number: WO2021223611A1
Application number: PCT/CN2021/089709
Authority: WO
Inventors: 陈辰; 胡文; 许春晖; 陶志东
Original assignee: 华为技术有限公司
Priority date: 2020-05-08
Filing date: 2021-04-25
Publication date: 2021-11-11
Also published as: CN111546338A

Abstract

A robot control method and apparatus, and a robot and a storage medium, which are applicable to the technical field of robots. The method comprises: acquiring position information of a target object; according to the position information of the target object, controlling a robot to rotate; and during the rotation process of the robot, dynamically adjusting an output picture of a display module, on the robot, used for simulating an eye feature. In the provided technical solution, the eye movement of a robot is simulated by means of an output interface of a display module, and deflection is realized without a motor drive during a moving process, such that the response time is relatively short; a smooth gaze tracking behavior is realized, such that the personification degree and the accuracy of the gaze tracking behavior are improved; and the robot can be better used for accompanying and educating children, and the human-machine interaction experience is enhanced.

Description

Robot control method, device, robot and storage medium

This application claims the priority of a Chinese patent application filed with the State Intellectual Property Office on May 8, 2020, with the application number 202010382291.0, and the application titled "A method, device, robot and storage medium for a robot", and its entire contents Incorporated in this application by reference.

Technical field

This application belongs to the field of robotics, and in particular relates to a control method, device, robot, and storage medium of a robot.

Background technique

In modern families, more and more parents cannot accompany their children at all times. When they cannot accompany their children, parents can use intelligent robots to accompany and educate their children. The existing intelligent robots can communicate with their children. , And based on the communication with the child, learn and update the way of communication with the child.

Anthropomorphism is an important indicator to measure the intelligence of robots. In the process of human-computer interaction, the accuracy of target following and gaze tracking behavior directly reflects the degree of anthropomorphism of the robot. By improving the anthropomorphism of the robot, the child's experience can be improved. . The existing robot control technology cannot achieve smooth gaze tracking behavior, thereby reducing the personification of the robot and the accuracy of the gaze tracking behavior.

Summary of the invention

The embodiments of the present application provide a robot control method, device, robot, and storage medium, which can solve the problem of the existing robot control technology, which cannot achieve smooth gaze tracking behavior. The degree of personification of the robot and the accuracy of the gaze tracking behavior are relatively high. Low question.

In the first aspect, an embodiment of the present application provides a method for controlling a robot, including:

Obtain the location information of the target object;

Controlling the rotation of the robot according to the position information of the target object;

During the rotation of the robot, the output screen of the display module for simulating eye features on the robot is dynamically adjusted, so that the output screen of the display module on the robot can simulate the eye features of the eye. The partial line of sight is the direction of looking at the target object during the rotation of the robot.

The above-mentioned controlling the rotation of the robot specifically refers to controlling the rotation of the rotating parts in the body of the robot. If the robot includes a head, the above-mentioned controlling the rotation of the robot specifically refers to controlling the rotation of the rotating parts connecting the head and the body of the robot to change the head of the robot. Frontal orientation; if the robot includes a head, a torso, and a bottom (such as a base or a leg), the above-mentioned controlling the rotation of the robot specifically refers to controlling the rotation of the rotating part connecting the head and the torso of the robot, and controlling the connection between the torso and the bottom of the robot The rotating parts in the middle rotate to change the orientation of the head and torso of the robot. Among them, the rotating part connecting any two parts of the robot can be one or two or more. When the rotating part connecting any two parts includes two or more, the rotation with multiple degrees of freedom can be realized. The above-mentioned multi-degree-of-freedom rotation includes: vertical rotation and horizontal rotation.

The above-mentioned simulated eye feature is specifically a picture that simulates the eyeball of a real person generated based on the video where the eye area of a real person is taken in various directions. It is also possible to construct eye pictures with eye characteristics through animation, cartoons, three-dimensional models, etc., including Used to simulate the dots or arbitrary shapes of the eyes. The output screen that simulates eye features may include simulated pupils, eyeballs, eyelids, eyebrows, eyelashes and other display objects, and the simulated eye screens are formed by the display objects.

In a possible implementation of the first aspect, the dynamic adjustment of the output screen of the display module for simulating eye features on the robot during the rotation of the robot is specifically:

During the rotation of the robot, the output screen of the display module for simulating eye features on the robot is dynamically adjusted according to the target deflection angle; the target deflection angle is the corresponding deflection angle when the robot is rotated from the initial angle to facing the target object .

The above-mentioned initial angle is specifically the direction angle corresponding to the front of the robot; if the robot includes a head, the direction angle corresponding to the front of the body is specifically the direction corresponding to the front of the robot's head, where the front of the robot's head Specifically, it refers to the surface that contains the simulated eyeball.

In a possible implementation of the first aspect, during the rotation of the robot, the output screen of the display module for simulating eye features on the robot is dynamically adjusted, so that the robot is The eye line of sight of the simulated eye feature of the output screen of the display module is the direction of looking at the target object during the rotation of the robot, including:

In the process of controlling the rotation of the robot, dynamically adjust the position of the simulated eye feature in the output screen to the target position of the eye feature according to the real-time rotation angle fed back by the robot in real time and the target deflection angle; The eye feature target position is the position in the direction in which the line of sight of the eye simulated by the output screen looks toward the target object for the first time;

After the line of sight of the eye looks in the direction of the target object for the first time, dynamically adjust the position of the simulated eye feature according to the real-time rotation angle to maintain the direction of the line of sight of the eye to the target object .

In a possible implementation of the first aspect, in the process of controlling the rotation of the robot, according to the real-time rotation angle fed back by the robot in real time and the target deflection angle, the output screen is dynamically adjusted Simulate the position of the eye feature to the target location of the eye feature, including:

Determining the amount of horizontal deflection and the amount of vertical deflection based on the real-time rotation angle;

According to the target deflection angle, the horizontal deflection amount, and the vertical deflection amount, the eye characteristic target position is determined; the real-time offset is specifically:

Wherein, eye_yaw is the horizontal component of the eye characteristic target position; eye_pitch is the vertical component of the eye characteristic target position; target_yaw is the horizontal component of the target deflection angle; target_pitch is the vertical component of the target deflection angle; motor_yaw is the horizontal deflection amount; motor_pitch is the horizontal deflection amount.

In a possible implementation manner of the first aspect, the controlling the rotation of the robot according to the position information of the target object includes:

Control the torso rotation of the robot based on the target deflection angle; the target deflection angle is the corresponding deflection angle when the torso of the robot rotates from the initial angle to the target object;

During the rotation of the torso of the robot, according to the real-time torso angle and the target deflection angle fed back in real time, the head of the robot is dynamically controlled to rotate to the target head position, and the target head position is the robot The position of the head facing the target object for the first time;

After the front of the robot head faces the direction facing the target object for the first time, according to the real-time torso angle, dynamically control the rotation of the robot head to keep the direction of the robot head facing the target object .

In a possible implementation manner, the controlling the torso rotation of the robot based on the target deflection angle includes:

Acquiring the first reference rotation angle of the torso of the robot in each control period in a preset control period;

Generate a first cycle rotation instruction corresponding to the current control cycle according to the first reference rotation angle of the current control cycle, the first tracking error angle of the previous control cycle, and the trunk rotation angle;

The torso of the robot is controlled to rotate according to each of the first cycle rotation instructions.

Wherein, the rotation angle in the first cycle rotation instruction is specifically:

Wherein, motor ₀ _yaw is the rotation angle in the first cycle rotation command; current_motor ₀ _yaw is the first reference rotation angle of the current control cycle; target_yaw is the target deflection angle; motor ₀ _yaw_diff is the current control The first tracking error angle of the cycle; motor ₀ _yaw_last_diff is the first tracking error angle of the previous control cycle; motor ₀ _Kp and motor ₀ _Kd are preset adjustment parameters of the trunk rotating part.

In a possible implementation of the first aspect, in the process of rotating the torso of the robot, according to the real-time torso angle fed back in real time and the target deflection angle, the head of the robot is dynamically controlled to rotate to The head target position, where the head target position is the position of the robot head facing the target object for the first time, includes:

Determining a horizontal deflection angle and a vertical deflection angle based on the target deflection angle;

Controlling the head of the robot to rotate in a vertical direction according to the vertical deflection angle;

And at the same time, during the rotation of the torso of the robot, according to the real-time torso angle and the horizontal deflection angle fed back in real time, the head of the robot is dynamically controlled to rotate in the horizontal direction to a horizontal target position; the horizontal target position Is the position of the robot head facing the target object in the horizontal direction for the first time.

In a possible implementation of the first aspect, the controlling the head of the robot to rotate in a vertical direction according to the vertical deflection angle includes:

Acquiring the second reference rotation angle of the head of the robot in the vertical direction in each control period in a preset control period;

Generate a second cycle rotation command corresponding to the current control cycle according to the second reference rotation angle of the current control cycle, the second tracking error angle of the previous control cycle, and the vertical deflection angle;

The head of the robot is controlled to rotate in the vertical direction according to each of the second cycle rotation instructions.

Wherein, the rotation angle in the second cycle rotation instruction is specifically:

Wherein, motor ₂ _pitch is the rotation angle in the second cycle rotation command; current_motor ₂ _pitch is the second reference rotation angle of the current control cycle; target_pitch is the vertical deflection angle; motor ₂ _pitch_diff is the current control The second tracking error angle of the cycle; motor ₂ _pitch_last_diff is the second tracking error angle of the previous control cycle; motor ₂ _Kp and motor ₂ _Kd are preset adjustment parameters of the second head rotating part.

In a possible implementation of the first aspect, in the process of rotating the torso of the robot, according to the real-time torso angle fed back in real time and the horizontal deflection angle, the head of the robot is dynamically controlled to be horizontal. Rotate in the direction to the horizontal target position, including:

Acquiring the third reference rotation angle of the head of the robot in each control period in the horizontal direction in a preset control period;

According to the third reference rotation angle of the current control period, the third tracking error angle of the previous control period, the real-time torso angle, and the horizontal deflection angle, the third reference rotation angle corresponding to the current control period is generated. Cycle rotation instruction;

The head of the robot is dynamically controlled to rotate in the horizontal direction according to each of the three-cycle rotation instructions.

Wherein, the rotation angle in the third cycle rotation instruction is specifically:

Wherein, motor ₁ _yaw is the rotation angle in the third cycle rotation command; current_motor ₁ _yaw is the third reference rotation angle of the current control cycle; target_yaw is the horizontal deflection angle; motor ₁ _yaw_diff is the current control The third tracking error angle of the cycle; motor ₁ _yaw_last_diff is the third tracking error angle of the previous control cycle; motor ₁ _Kp and motor ₁ _Kd are the preset adjustment parameters of the first head rotating part; current_motor ₀ _yaw Is the real-time torso angle.

In a possible implementation manner of the first aspect, the acquiring location information of the target object includes:

Acquiring a scene image containing the target object through a camera module built into the robot;

Mark the boundary coordinates of the target object from the scene image, and determine the position information of the target object of the target object according to the boundary coordinates; the position information of the target object includes the object center coordinates of the target object ；

Determine the target deflection angle according to the object center coordinates and the image center coordinates of the scene image;

Wherein, the target deflection angle is specifically:

Wherein, target_yaw is the horizontal component of the target deflection angle; target_pitch is the vertical component of the target deflection angle; OC is the focal length of the camera module; CD is the horizontal deviation between the object center coordinates and the image center coordinates; AC is The vertical deviation between the object center coordinate and the image center coordinate.

In a possible implementation of the first aspect, the acquiring a scene image containing the target object through a camera module built into the robot includes:

Determine the image correction parameter according to the offset between the position of the simulated eye feature of the output screen of the camera module and the display module;

The original image collected by the camera module is adjusted based on the image correction parameter to generate the scene image.

In the second aspect, an embodiment of the present application provides a robot control device, including:

The location information acquiring unit is used to acquire the location information of the target object;

A rotation control unit for controlling the rotation of the robot according to the position information of the target object;

The screen adjustment unit is used to dynamically adjust the output screen of the display module for simulating eye features on the robot during the rotation of the robot, and look in the direction of the target object so that the robot is on the The line of sight of the eye that simulates the eye feature of the output screen of the display module is the direction of looking at the target object during the rotation of the robot.

In a third aspect, an embodiment of the present application provides a robot, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the The computer program realizes the control method of the robot according to any one of the above-mentioned first aspects.

In a fourth aspect, an embodiment of the present application provides a robot, including a processor, a display, and a transmission component for controlling the rotation of the robot, wherein the processor executes a computer program to implement any one of the first aspects In the control method of the robot, the transmission component controls the rotation of the robot according to the control instruction output by the computer program, and the display dynamically adjusts the output for simulating eye features according to the output instruction of the computer program Picture.

In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium that stores a computer program, and is characterized in that, when the computer program is executed by a processor, any of the above-mentioned aspects of the first aspect is implemented. A method of controlling the robot.

In a sixth aspect, the embodiments of the present application provide a computer program product, which when the computer program product runs on a robot, causes the robot to execute the robot control method described in any one of the above-mentioned first aspects.

It is understandable that, for the beneficial effects of the second aspect to the fifth aspect described above, reference may be made to the relevant description in the first aspect described above, and details are not repeated here.

The embodiment of the present application determines the target deflection angle according to the relative position between the target object and the robot, and adjusts the output screen and rotating parts of the display module based on the target deflection angle to realize the gaze tracking behavior, because the eye movement of the robot is displayed The output interface of the module is simulated, no motor drive is needed to achieve deflection during movement, the response time is short, smooth gaze tracking behavior is realized, and the degree of personification and the accuracy of gaze tracking behavior are improved.

Description of the drawings

FIG. 1 is a block diagram of a part of the structure of a robot provided by an embodiment of the present application;

FIG. 2 is a schematic diagram of the software structure of the robot according to an embodiment of the present application;

Fig. 3 is a schematic diagram of a robot provided by an embodiment of the present application;

FIG. 4 is a schematic structural diagram of a robot provided by an embodiment of the present application;

FIG. 5 is an implementation flowchart of the robot control method provided by the first embodiment of the present application;

Fig. 6 is a schematic diagram of a key center of a target object provided by an embodiment of the present application;

FIG. 7 is a schematic diagram of obtaining a target deflection angle provided by an embodiment of the present application;

FIG. 8 is a specific implementation flowchart of a robot control method S501 and S502 according to another embodiment of the present application;

FIG. 9 is a schematic diagram of determining the center coordinates of an object provided by an embodiment of the present application;

FIG. 10 is a schematic diagram of obtaining a target offset angle provided by an embodiment of the present application;

FIG. 11 is a schematic diagram of the principle of shooting offset provided by an embodiment of the present application;

FIG. 12 is a specific implementation flowchart of a robot control method S5011 provided by another embodiment of the present application;

FIG. 13 is a schematic diagram of a motor-driven multi-degree-of-freedom robot provided by an embodiment of the present application;

Fig. 14 is a schematic diagram of a robot with 2 degrees of freedom provided by an embodiment of the present application;

15 is a specific implementation flowchart of a robot control method S503 provided by another embodiment of the present application;

FIG. 16 is a schematic diagram of a robot's eye sight aligning with a target object for the first time according to an embodiment of the present application;

FIG. 17 is a specific implementation flowchart of a robot control method S5031 according to another embodiment of the present application;

18 is a schematic diagram of a robot with one degree of freedom in a rotating part provided by an embodiment of the present application;

19 is a schematic diagram of a robot with two degrees of freedom in a rotating part provided by an embodiment of the present application;

20 is a specific implementation flowchart of a robot control method S502 provided by another embodiment of the present application;

FIG. 21 is a specific implementation flowchart of a robot control method S2001 provided by another embodiment of the present application;

FIG. 22 is a specific implementation flowchart of a robot control method S2002 provided by another embodiment of the present application;

FIG. 23 is a schematic diagram of a robot with three degrees of freedom in a rotating part provided by an embodiment of the present application;

FIG. 24 is a specific implementation flowchart of a robot control method S2202 provided by another embodiment of the present application;

25 is a specific implementation flowchart of a robot control method S2203 provided by another embodiment of the present application;

FIG. 26 is a structural block diagram of a robot control device provided by an embodiment of the present application;

FIG. 27 is a schematic diagram of a robot provided by another embodiment of the present application.

Detailed ways

In the following description, for the purpose of illustration rather than limitation, specific details such as a specific system structure and technology are proposed for a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from obstructing the description of this application.

It should be understood that when used in the specification and appended claims of this application, the term "comprising" indicates the existence of the described features, wholes, steps, operations, elements and/or components, but does not exclude one or more other The existence or addition of features, wholes, steps, operations, elements, components, and/or collections thereof.

It should also be understood that the term "and/or" used in the specification and appended claims of this application refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

As used in the description of this application and the appended claims, the term "if" can be construed as "when" or "once" or "in response to determination" or "in response to detecting ". Similarly, the phrase "if determined" or "if detected [described condition or event]" can be interpreted as meaning "once determined" or "in response to determination" or "once detected [described condition or event]" depending on the context ]" or "in response to detection of [condition or event described]".

In addition, in the description of the specification of this application and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

The reference to "one embodiment" or "some embodiments" described in the specification of this application means that one or more embodiments of this application include a specific feature, structure, or characteristic described in combination with the embodiment. Therefore, the sentences "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification are not necessarily All refer to the same embodiment, but mean "one or more but not all embodiments" unless it is specifically emphasized otherwise. The terms "including", "including", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

Gaze tracking behavior refers to the robot's active tracking of the target object through the coordinated actions of the robot's torso, head, and eyes. It is the most important function of human-computer non-verbal interaction and an important step in human-computer interaction. For anthropomorphic robots, gaze tracking is particularly important. Only when the system design realizes the tracking gaze and natural confrontation between the robot and the interactive object, can further deep-level interaction be carried out. Gaze tracking behavior specifically includes aspects, which are the robot's tracking of the target and the eye contact between the robot and the target object. In view of the current robot control technology that cannot achieve smooth gaze tracking behavior, thereby reducing the degree of personification of the robot and the accuracy of gaze tracking behavior, embodiments of the present application provide a robot control method, device, robot, and storage medium According to the relative position between the target object and the robot, determine the target deflection angle, and adjust the output screen and rotating parts of the display module based on the target deflection angle to realize the gaze tracking behavior, because the eye movement of the robot is output through the display module The interface is simulated, and no motor is required to achieve deflection during movement. The response time is short, and smooth gaze tracking behavior is realized, and the degree of personification and the accuracy of gaze tracking behavior are improved.

The technical solution of the present application will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

The robot control method provided in the embodiments of the present application can be applied to an intelligent robot that can realize human-computer interaction. The above-mentioned human-computer interaction includes, but is not limited to, target tracking, gaze tracking behavior, intelligent question and answer, intelligent navigation, music on demand, and intelligent For interactive operations such as escort, intelligent robots can also automatically perform task operations associated with instructions according to pre-configured instructions. For example, when the robot detects the arrival of the preset time node, it can give voice prompts to the user, or meet the preset When a trigger condition is triggered, a response operation associated with the trigger condition is executed, for example, when it is detected that the current indoor temperature is greater than a preset temperature threshold, the indoor air-conditioning device is turned on. In the process of interaction between the robot and the user, in order to improve the anthropomorphic process of the robot, the eyes of the robot can move with the movement of the user. The above process is the gaze tracking behavior.

In this embodiment, the above-mentioned robot may be a robot 100 having a hardware structure as shown in FIG. 1. As shown in FIG. 1, the robot 100 may specifically include: a communication module 110, a memory 120, a camera module 130, a display unit 140, The sensor 150, the audio circuit 160, the rotating component 170, the processor 180, and the power supply 190 and other components. Those skilled in the art can understand that the structure of the robot 100 shown in FIG. 1 does not constitute a limitation on the robot. The robot may include more or fewer components than shown, or combine certain components, or different component arrangements. .

The following describes the components of the robot in detail with reference to Figure 1:

The communication module 110 can be used to establish a communication connection with other devices, so as to receive control instructions and firmware update data packets sent by other devices, and can also send operation records of the robot to other devices. Optionally, the robot can also use the communication module 110 Establish communication connections with other robots in the scene, so as to cooperate with other robots. In particular, the communication module 110 may send the downlink information received from other devices to the processor 180 for processing; in addition, it may send the designed uplink data to other devices connected to it. Generally, the communication module 110 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, a wireless communication module, a Bluetooth communication module, and so on. In addition, the communication module 110 may also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile Communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (Code Division) Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE)), Email, Short Messaging Service (SMS), etc.

The memory 120 may be used to store software programs and modules. The processor 180 executes various functional applications and data processing of the robot by running the software programs and modules stored in the memory 120. The memory 120 may mainly include a program storage area and a data storage area, where the program storage area may store an operating system, at least one application program required by at least one function (such as a gaze tracking behavior application, an intelligent escort application, an intelligent education application, etc.), etc.; The data area may store data created according to the use of the robot (for example, image data collected by the camera module 130, rotation angle fed back by the rotating component 170, etc.) and the like. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

The camera module 130 can be used to collect images of the environment where the robot is located. The shooting direction of the camera module 130 can be consistent with the direction of the front of the robot, so that the robot can simulate the environment where the robot is "seeing". Optionally, the robot has a plurality of camera modules 130 built in, and different camera modules 130 are used to collect environmental images of the robot in different directions; optionally, the robot has a built-in camera module 130, which can preset a trajectory Move or rotate around an axis to obtain environmental images of different angles and directions. The camera module 130 can store the collected images in the memory 120, and can also directly transmit the collected images to the processor 180.

The display unit 140 may be used to display information input by the user or information provided to the user and various menus of the robot. The display unit 140 may include a display panel 141. Optionally, the display panel 141 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel can cover the display panel 141, and when the touch panel detects a touch operation on or near it, it transmits it to the processor 180 to determine the type of the touch event, and then the processor 180 determines the type of the touch event according to the type of the touch event. Corresponding visual output is provided on the display panel 141. In particular, the head area of the robot contains a display module for simulating binoculars. The output screen of the display module is specifically a screen that simulates the eyes. The eyes can be generated based on the video corresponding to the movement of the eye area of a real person in various directions. The screen that simulates the eyeballs of real people can also be constructed through animation, comics, three-dimensional models and other methods. The output screen of the simulated eye may include simulated pupils, eyeballs, eyelids, eyebrows, eyelashes, and other display objects, and the simulated eye screens are formed by the above-mentioned display objects. It should be noted that the head of the robot can include a display module, which can be used to output an output screen containing two eyes; the head of the robot can also include two display modules, namely a left-eye display module and a right-eye display module. The display module, the left-eye display module is used to output an output image that simulates the left eye, and the right-eye display module is used to output an output image that simulates the right eye.

The robot 100 may also include at least one sensor 150, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display unit 140 according to the brightness of the ambient light. The proximity sensor is used to determine the distance between the robot and the user. When the preset distance threshold is used, the transmission components of the robot are used to control the robot to be far away from the user, so as to avoid collision with the user. As a kind of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in various directions (usually three-axis), and can detect the magnitude and direction of gravity when it is stationary, which can be used to recognize robot posture applications, vibration recognition related functions, etc.; As for other sensors, such as gyroscopes, thermometers, infrared sensors, etc., that the robot can also configure, I won't repeat them here.

The audio circuit 160, the speaker 161, and the microphone 162 can provide an audio interface between the user and the robot. The audio circuit 160 can transmit the electrical signal converted from the received audio data to the speaker 161, which is converted into a sound signal for output by the speaker 161; on the other hand, the microphone 162 converts the collected sound signal into an electrical signal, and the audio circuit 160 After being received, it is converted into audio data, processed by the audio data output processor 180, and sent to, for example, another robot via the communication module 110, or the audio data is output to the memory 120 for further processing. Optionally, in the process of the user interacting with the robot, the audio circuit 160 can collect the voice command initiated by the user, and respond to the response operation based on the voice command; the audio circuit 160 can also be used in the process of human-computer interaction. In, the intelligent response voice signal is output, and the simulation interaction process with the user is realized.

In the process of movement, target tracking, and gaze tracking behavior of the robot, the rotating part 170 can be used to control the robot to perform corresponding movements. The rotating part 170 can be driven by a motor, and the motor provides a driving force to control the rotating part 170 to rotate, so as to realize movement operations such as robot movement, posture adjustment, and orientation adjustment. In a possible implementation, in order to improve the personification of the gaze tracking behavior, multiple rotating parts 170 can be installed on the head, torso, etc. of the robot. Different rotating parts can control the robot to rotate in one direction. That is, corresponding to one degree of freedom, according to the installation of multiple rotating parts 170 in the robot, a gaze tracking behavior with multiple degrees of freedom can be realized. The rotating component 170 can also feed back the real-time rotation angle to the processor 180 in real time, and the processor 180 can control the movement of the robot according to the feedback real-time rotation angle, so as to implement smooth gaze tracking behavior on the target object.

The processor 180 is the control center of the robot. It uses various interfaces and lines to connect various parts of the entire robot. Various functions and processing data of the robot, so as to monitor the robot as a whole. Optionally, the processor 180 may include one or more processing units; preferably, the processor 180 may integrate an application processor and a modem processor, where the application processor mainly processes the operating system, user interface, application programs, etc. , The modem processor mainly deals with wireless communication. It can be understood that the foregoing modem processor may not be integrated into the processor 180.

The robot 100 also includes a power source 190 (such as a battery) for supplying power to various components. Preferably, the power source may be logically connected to the processor 180 through a power management system, so that functions such as charging, discharging, and power consumption management can be managed through the power management system.

The software system of the robot 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiment of the present application takes an Android system with a layered architecture as an example to illustrate the software structure of the robot 100 by way of example.

FIG. 2 is a block diagram of the software structure of the robot 100 according to an embodiment of the present application. The Android system is divided into four layers, which are application layer, application framework layer (framework, FWK), system layer, and hardware abstraction layer. The layers communicate through software interfaces.

The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Communication between layers through software interface. In some embodiments, the Android system is divided into four layers, from top to bottom, the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.

The application layer can include a series of application packages.

As shown in Figure 2, the application package may include applications such as camera, smart question and answer, smart escort, multimedia on-demand, smart learning and education.

The application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

As shown in Figure 2, the application framework layer can include a window manager, a content provider, a view system, a resource manager, a notification manager, and so on.

The window manager is used to manage window programs. The window manager can obtain the size of the display module, determine the display status of the current display module, and so on.

The content provider is used to store and retrieve data and make these data accessible to applications. The data may include video, image, audio, the rotation angle fed back by the rotating component, and so on.

The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. In particular, the visual controls include controls for simulating eyeballs.

The resource manager provides various resources for the application, such as localized strings, icons, pictures, graphics, layout files, video files, and so on.

The notification manager enables the application to display notification information in the status bar, which can be used to convey notification-type messages, and it can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, and so on. The notification manager can also be a notification that appears in the status bar at the top of the system in the form of a chart or a scroll bar text, such as a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, text messages are prompted in the status bar, prompt sounds, electronic devices vibrate, and indicator lights flash.

Android Runtime includes core libraries and virtual machines. Android runtime is responsible for the scheduling and management of the Android system.

The core library consists of two parts: one part is the function functions that the java language needs to call, and the other part is the core library of Android.

The application layer and application framework layer run in a virtual machine. The virtual machine executes the java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

The system library can include multiple functional modules. For example: surface manager (surface manager), media library (Media Libraries), three-dimensional graphics processing library (for example: OpenGL ES), 2D graphics engine (for example: SGL), etc.

The surface manager is used to manage the display subsystem and provides a combination of 2D and 3D layers for multiple applications.

The media library supports a variety of commonly used audio, video format recording, and still image files. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing. Specifically, the output screen used to simulate the eyeballs can be rendered and synthesized through a three-dimensional graphics processing library.

The 2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver. In some embodiments, the above-mentioned kernel layer further includes a PCIE driver.

In the embodiment of the present application, the execution subject of the process is a robot. As an example and not a limitation, the robot includes a display module and a rotating part, an output screen for simulating eyeballs is output through the display module, and the robot is controlled to move through the rotating part. As an example and not a limitation, FIG. 3 shows a schematic diagram of a robot provided by an embodiment of the present application. As shown in Fig. 3, the robot provided by the present application can be a robot with a simulated human form (a-shaped robot in Fig. 3), or a non-human-shaped robot, such as a robot with a simulated animal form (b in Fig. 3). Type robots) or non-biological robots (such as type c robots in Figure 3). The above-mentioned robots are any equipment with motion functions, and motions include movement, rotation, and so on.

As an example and not a limitation, FIG. 4 shows a schematic structural diagram of a robot provided by an embodiment of the present application. As shown in Figure 4, the display module of the robot is installed on the head of the robot. The display module includes a left-eye display module for simulating the left eye and a right-eye display module for simulating the right eye. The output of the two display modules is The screen simulates eye movement and realizes gaze tracking. The rotating part of the robot includes a trunk rotating part for controlling the left and right rotation of the robot's torso, a first head rotating part for controlling the left and right rotation of the robot head, and a second head rotating part for controlling the up and down rotation of the robot head. The robot realizes smooth gaze tracking behavior by controlling the output screen of the display module and the rotation of the rotating parts.

FIG. 5 shows an implementation flowchart of the robot control method provided by the first embodiment of the present application, and the details are as follows:

In S501, the location information of the target object is acquired.

In S502, the robot is controlled to rotate according to the position information of the target object.

In this embodiment, the robot can determine the target object through automatic recognition or manual setting, and determine the location information of the target object. The position information may specifically be the relative position between the robot and the target object. For example, the relative position may be a relative direction, such as left, right, front, rear, etc.; it may also be a specific angle, such as +60°, -120°, where the above angle may specifically be a vector Angle, the positive and negative values of the angle represent the corresponding direction. For example, the left side can be defined as the positive direction, and the right side can be defined as the negative direction. The robot can control the rotation of the robot according to the above-mentioned position information.

In a possible implementation manner, S502 may specifically be: determining the initial angle of the robot to rotate to the target deflection angle corresponding to the target object according to the position information exclusive to the target; controlling the target deflection angle according to the target deflection angle; The robot rotates.

In this embodiment, the target deflection angle is determined according to the initial angle of the robot itself and the pose of the target object. The initial angle of the above-mentioned robot is specifically the direction angle corresponding to the front of the robot. If the robot includes a head, the direction angle corresponding to the front of the body is specifically the direction corresponding to the front of the head of the robot. The front of specifically refers to the surface that contains the simulated eyeball. In order to achieve the purpose of gaze tracking, the above-mentioned target deflection angle is specifically used to view the current line of sight of the robot in the direction of the target object. If the target object is a physical person, the target deflection angle is specifically used to set the current robot Align the line of sight to the key center of the target object.

As an example and not a limitation, FIG. 6 shows a schematic diagram of the key center of the target object provided by an embodiment of the present application. As shown in Figure 6, the key center of the target object can be specifically: the geometric center of the target object, the geometric center of the head of the target object. If the target object is a physical person, the above-mentioned key center can also be the eyes of the target object. The middle area.

Illustratively, FIG. 7 shows a schematic diagram of acquiring a target deflection angle provided by an embodiment of the present application. As shown in Figure 7, the target object is a physical person. Since the physical person has a certain volume of space, any point in the space belongs to the target object. For gaze tracking, the most important thing is to match the line of sight of the robot with the target object. The line of sight meets, that is, in the same plane. Therefore, the robot needs to determine its current line of sight direction and the line of sight direction of the target object, and determine the deflection angle based on the two line of sight directions.

In a possible implementation, the robot can automatically recognize the target object in the following three ways:

Method 1: Determine the target object based on the distance sensor. The specific implementation method is as follows: the robot can be equipped with a distance sensor, which can contain multiple distance detection units, and build the scene where the robot is based on the distance values collected by the multiple distance detection units Based on the depth map, determine the scene objects contained in the current scene, and select the scene object with the smallest distance as the target object. Optionally, if the target object of the gaze tracking behavior is a physical person, the contour line of each scene image can be determined according to the constructed depth map, and the contour line can be matched with the standard contour line of the physical person based on the matching result from the scene object Identify the target audience. Preferably, the robot also includes an infrared thermal imaging module, which can collect the temperature value of the outer surface of each scene object in the scene, so as to construct a depth map containing temperature information, according to the contour information and temperature value in the depth map , The object type of the scene object contained in the scene where the robot is located can be determined, and the scene object whose object type is human is selected as the target object. If there are multiple scene objects whose object type is human, the closest scene object whose object type is human may be selected as the target object.

Method 2: Determine the target object based on the captured image. The implementation method is as follows: the robot can obtain the scene image of the current scene through the camera module. The contour information contained in the scene image is extracted by the contour recognition algorithm, and the scene image is divided based on the contour information to obtain multiple subject areas. The subject type is identified for each subject area, and the target object is determined based on the subject type. For example, if the target object of the robot's gaze tracking behavior is a physical person, a physical person-type subject can be selected from the subject type as the target object; if multiple scene types included in the shooting scene are physical human subjects, then The subject with the largest subject area can be selected as the target object. Optionally, if the target object is a physical person, after acquiring the scene image, the robot can locate the face area through a face recognition algorithm, and determine the target object based on the face area.

Method 3: Determine the target object based on the voice signal. The implementation method is as follows: the robot can be equipped with a microphone module, and the sound signal in the current scene can be obtained through the microphone module. If it is detected that the signal strength of the sound signal in the current scene is greater than the preset decibel threshold, voice analysis is performed on the sound signal, and the target object is determined based on the voice analysis result. Among them, the above-mentioned voice analysis may specifically be the conversion of a sound signal into text information. The robot may be equipped with an activation password. If the above-mentioned text information matches the activation password, it is recognized that the user needs to interact with the robot. At this time, the corresponding voice signal can be obtained. The sounding direction, and the object corresponding to the sounding direction in the current scene is identified as the target object. The aforementioned voice analysis may also be to extract the voiceprint feature parameters of the voice signal, match the voiceprint feature parameters with the standard biological parameters of each registered user, and determine the registered user corresponding to the voice signal based on the matching result , And identify the registered user corresponding to the sound signal as the target user. In this case, the robot can obtain the environment image in the current scene through the camera module, and determine the location information of the target user based on the pre-stored standard face image of the target object and the above-mentioned environment image.

In a possible implementation, the method for the robot to determine the target object can be a method set by the user, and the method set by the user can include the following two:

Method 1: Determine the target object according to the user's selection instruction. The realization method is as follows: the robot can display the candidate objects in the scene on the interactive interface (for example, the display module configured on the body part of the robot outputs the scene image captured by the current scene, Face recognition is performed on the scene image, and the recognized face is used as a candidate object. The user can send a selection instruction to the robot through interactive means such as touch or keypress, and select one of multiple candidate objects as the target object.

Method 2: Determine the target object according to the positioning device. The implementation is as follows: The user who needs to follow the gaze can wear a positioning device, which can be a smart watch, smart necklace, smart phone, smart glasses, etc., equipped with a positioning module Wearable device. The positioning module can send positioning information to the robot in a preset feedback cycle, and the robot can recognize the target object according to the positioning information.

In a possible implementation, the robot can trigger the gaze and follow behavior in the following three ways:

Method 1, the robot can trigger the gaze and follow behavior when it detects a change in the pose of the target object. The robot can determine the position of the target object in the above-mentioned manner, and determine the posture of the target object according to the contour information of the target object. If the position movement and/or posture of the target object is detected to change, it is determined that the posture of the target object has changed. At this time, the gaze tracking behavior is triggered, and the operations of S501 and S502 are executed.

Method 2, the robot can trigger the gaze and follow behavior when it detects a change in the robot's pose. The robot may have a built-in motion sensor. The motion sensor includes but is not limited to: gyroscope, vibration sensor, acceleration sensor, gravity sensor, etc. The robot can obtain the sensing value fed back by each motion sensor to determine whether the pose of the robot has changed, for example, if the gyro If the reading of the meter changes or the value of the acceleration sensor is not 0, then the pose of the robot can be changed. At this time, the gaze tracking behavior is triggered, and the operations of S501 and S502 are executed.

Method 3, the robot can trigger the gaze tracking behavior when detecting the change of the target object. The aforementioned change of the target object includes the change of the target object from a non-target object to a target object, and also includes a change from object A to object B. The above-mentioned target change can be based on the user's selection instruction to determine the object that needs to be changed, or the target can be changed through automatic recognition by the robot (for example, the target object appears in the collection screen of the robot, and the target object B is close to the robot to recognize it. The distance between the object B and the robot is smaller than the distance between the object A and the robot, and then the robot switches the target object from the object A to the object B).

FIG. 8 shows a specific implementation flowchart of a robot control method S501 and S502 provided by another embodiment of the present application. Referring to FIG. 8, compared with the embodiment described in FIG. 5, S501 in a robot control method provided in this embodiment includes: S5011 to S5012, and S502 includes S5021, and the details are as follows:

In S5011, a scene image containing the target object is acquired through a camera module built into the robot.

In this embodiment, the robot has a built-in camera module, and the scene image in the current scene is collected through the camera module.

In a possible implementation, the camera module can obtain the scene video in the current scene in the form of a video format, and the robot can use the latest video image frame captured in the scene video as the above scene image and recognize it from the scene image target.

In a possible implementation, the robot may send a shooting instruction to the camera module. After the camera module receives the shooting instruction, it obtains the image corresponding to the moment the shooting instruction is received, and recognizes the currently shot image as the aforementioned scene image.

In this embodiment, the robot may be equipped with a target object recognition algorithm. The robot analyzes the image data fed back from the camera image through the above-mentioned target object recognition algorithm to determine whether the image data contains the target object. The image data is a scene image containing the target object, and the operation of S5012 is performed; on the contrary, if the target object is not included in the image data, there is no need to perform gaze tracking.

In a possible implementation manner, if the target object is a person, the recognition algorithm of the target object may be a face recognition algorithm. In this case, the robot can determine whether a human face is included in the scene image fed back by the camera module through a face recognition algorithm, and if it does, perform the operation of S5012. On the contrary, if the scene image does not contain a human face, it is recognized that there is no target object for gaze tracking in the current scene, and the standby state is entered or the original posture is maintained.

In S5012, the boundary coordinates of the target object are marked from the scene image, and the position information of the target object of the target object is determined according to the boundary coordinates; the position information of the target object includes the target object The center coordinates of the object.

In this embodiment, the robot recognizes the image of the area where the target object is located in the scene object, and determines the above-mentioned boundary coordinates from the image of the area where the target object is located.

In a possible implementation, the robot is equipped with a contour recognition algorithm, the scene image can be imported into the above contour recognition algorithm, the contour line contained in the scene image is extracted, and at least one pixel point is selected from the contour line as the aforementioned contour recognition algorithm. The boundary coordinates.

In a possible implementation, the robot can select the closest boundary point between the target object and the origin coordinates of the scene image (for example, the scene image uses the upper left corner of the image as the origin coordinates) as the aforementioned boundary coordinates, and according to the target object in the scene The area and boundary coordinates in the image determine the center coordinates of the object. The image coordinate system is constructed with the horizontal and vertical directions of the scene image as the coordinate axis. According to the relative position between the boundary coordinates and the origin coordinates of the scene image, the value of the boundary coordinates can be (left, top), where left is the origin The horizontal distance between the coordinates; top is the vertical distance between the coordinates of the origin. If the target object is a rectangle, the area of the corresponding area can be expressed as (width, height), where width is the width of the rectangular area, and height is the length of the rectangular area. Based on the above multiple parameters, the position information of the target object can be determined, and the position information includes the object center coordinates corresponding to the target object in the scene image.

Exemplarily, FIG. 9 shows a schematic diagram of determining the center coordinates of an object provided by an embodiment of the present application. As shown in Fig. 9, the target object T can be expressed as [left, top, width, height]. Through the above four parameters, the object center coordinate A=[left+width/2,top+height/2] of the target object can be determined.

In a possible implementation manner, if the target object is a human face, the boundary coordinates of the target object may be the eye coordinates of the target object, and the midpoint of the line segment formed by the two human eyes of the target object is taken as the object of the target object Center, use the coordinates of the midpoint as the object center coordinates of the target object.

In S5021, determine the target deflection angle according to the object center coordinates and the image center coordinates of the scene image;

In this embodiment, the shooting angle of the camera module is consistent with the eye sight direction of the robot. Therefore, if the eye sight is to be aligned with the target object, the object center coordinates of the target object need to be moved to the center coordinates of the scene image. Therefore, an offset vector can be generated according to the center coordinates of the object and the image center coordinates of the scene image, and the above-mentioned target deflection angle can be determined according to the offset vector and the initial angle of the robot's eye line of sight.

Exemplarily, FIG. 10 shows a schematic diagram of obtaining a target offset angle provided by an embodiment of the present application. As shown in FIG. 10, the object center of the target object is point A, the focal point of the scene image is point C, and the center point of the robot's eyes is point O. Therefore, the line of sight of the robot's eyes is a straight line OD, the focal length of the camera module is OC, and the distance from the image center coordinate offset to the object center coordinate is AC. Through the above four coordinate points, the deflection angle of the robot in the horizontal direction can be determined target_yaw and the deflection angle target_pitch in the vertical direction.

In the embodiment of the present application, the scene image containing the target object is acquired through the camera module, and the target offset angle is calculated according to the deviation between the center coordinates of the target object in the scene image and the coordinates of the eye line of sight in the scene image , Calculate the amount of deflection in the three-dimensional direction through the two-dimensional image, the calculation method is simple, and the calculation amount is small, which can reduce the calculation pressure of the robot.

Further, if there is a certain offset between the camera module and the line of sight of the eye, in order to improve the accuracy of the target deflection angle, the robot can calibrate the scene image based on the offset when acquiring the scene image. Exemplarily, FIG. 11 shows a schematic diagram of the principle of shooting offset provided by an embodiment of the present application. As shown in FIG. 11, the robot includes a camera module, which is located above the display module and has a certain offset from the eyeball in the output image of the display module. Based on this, if a camera module is also configured at the position of the simulated eye feature, the position of the target object in the image observed by the eyeball and the position of the target object captured by the camera will have a certain offset, so that in the subsequent When the gaze follows, the line of sight of the robot's eyes will shift from the center of the target object. Therefore, it is necessary to calibrate the image collected by the aforementioned camera module.

In this case, S5011 in the previous embodiment may include a calibration operation. FIG. 12 shows a specific implementation flowchart of a robot control method S5011 provided by another embodiment of the present application. Referring to FIG. 12, with respect to the embodiment described in FIG. 8, S5011 in a robot control method provided in this embodiment includes: S1201 to S1202, which are detailed as follows:

In S1201, the image correction parameter is determined according to the offset between the position of the simulated eye feature of the output screen of the camera module and the display module.

In this embodiment, the robot can determine the position of the eyeball relative to the robot body according to the display position of the eyeball in the output screen, and according to the local position of the camera module in the robot and the above-determined position of the eyeball relative to the robot, Determine the above offset. The robot can be configured with a conversion algorithm between the offset and the correction amount, and the offset is imported into the above conversion algorithm to calculate the image correction parameters.

In S1202, the original image collected by the camera module is adjusted based on the image correction parameter to generate the scene image.

In this embodiment, after the robot receives the original image collected by the camera module, it can calibrate the original image using the image correction parameters. For example, the original image can be rotated, stretched, stretched horizontally, and stretched according to the correction parameters. Correction operations such as vertical stretching and translation of the target object are performed, and the corrected original image is used as the above-mentioned scene image, and the subsequent determination of the target offset angle is performed.

In the embodiment of the present application, the original image captured by the camera module is corrected according to the offset between the camera module and the line of sight of the eye, so that the accuracy of the subsequent target deflection angle can be improved, and the deviation between the robot modules can be eliminated. The deviation of the line of sight caused by the movement further improves the accuracy of the gaze tracking behavior and improves the degree of personification of the robot.

In S503, during the rotation of the robot, dynamically adjust the output screen of the display module for simulating the eyeball feature on the robot, and control the rotation of the built-in rotating part of the robot, so that the robot The line of sight of the eye that simulates the eye feature of the eyeball of the output picture of the display module looks in the direction of the target object during the rotation of the robot.

In this embodiment, after the robot has determined the relative position between its frontal pose and the frontal facing position of the target object, that is, after the position information of the target object, it can simulate eye features through the output screen of the display module While moving, control the rotation of the built-in rotating parts of the robot, so that the simulated eye features in the output screen of the robot display module, such as the simulated eyeball or the simulated eye aligning with the target object.

Compared with mechanical movement, outputting an output image for simulating the movement of eye features through the display module will make the movement of eye features smoother. Since the output picture through the display module is not limited and the machine movement, the position of the simulated eye feature can be changed only by refreshing the picture content, which reduces the startup time required by mechanical driving methods such as motor drive, and improves the response rate. The eye movement simulated by the display module is closer to the response speed of the human eye movement, and the degree of personification of the robot is improved.

In a possible implementation manner, the dynamic adjustment of the output screen of the display module for simulating eye features on the robot during the rotation of the robot is specifically: during the rotation of the robot, according to The target deflection angle dynamically adjusts the output screen of the display module used for simulating eye features on the robot; the target deflection angle is the corresponding deflection angle when the robot rotates from the initial angle to face the target object.

In a possible implementation, the way to adjust the sight of the robot's eyes to the target object may specifically be: determining the offset of the simulated eye feature in the output screen and the rotation angle of the rotating part of the robot according to the target deflection angle , While adjusting the position of the simulated eye features in the output screen, control the rotation of the robot's built-in rotating parts.

Among them, because the display module of the robot is attached to the surface of the robot, the movement of the position of the simulated eye feature needs to consider the movement of the robot body. In this case, the method of determining the rotation angle and the offset of the simulated eyeball can be: Use the target offset angle as the rotation angle of the rotating part, and calculate the eyeball offset according to the following calculation method:

Among them, eye_angle is the offset of the eyeball; target_angle is the target deflection angle; motor_angle is the rotation angle of the rotating part during the movement of the eyeball; motor_rate is the rotation speed of the rotating part; time is the time required for the above-mentioned offset for the eyeball movement ;Eye_rate is the speed of eyeball movement.

In a possible implementation manner, due to the different target deflection angles, the starting acceleration of the rotating part will also be different, resulting in that the speed of the rotating part is not a constant value. In this case, the robot can collect the historical speed of the rotating part under different target deflection angles through big data analysis, so as to construct the correspondence between the target deflection angle and the rotation angle. The robot can determine the rotation speed of the rotating part according to the target deflection angle obtained this time, and then import it into the above calculation equation to calculate the eyeball offset.

As can be seen from the above, the robot control method provided by the embodiment of the present application determines the target deflection angle according to the relative position between the target object and the robot, and adjusts the output screen and rotating parts of the display module based on the target deflection angle to achieve this. Eye tracking, because the robot's eye movement is simulated through the output interface of the display module, no motor drive is needed to achieve deflection during the movement, and the response time is short, achieving smooth eye tracking, improving the degree of personification and the accuracy of eye tracking .

In the existing robot control technology, the following two types of methods are mainly used for gaze tracking:

First, gaze tracking is achieved through visual fusion, dual-mode motion, and control algorithms. Since the speed of visual offset is different from the speed of body movement such as the head and torso, in the calculation process of dual-mode motion, The speed deviation between different components needs to be considered, which greatly increases the amount of calculations for gaze tracking, and when the robot's automaticity in the gaze tracking process increases, the amount of calculations mentioned above also increases at a geometric level, further increasing the robot's Computing pressure. FIG. 13 shows a schematic diagram of a motor-driven multi-degree-of-freedom robot provided by an embodiment of the present application. As shown in Figure 13, the robot head can rotate in three directions during the tracking and gaze process, namely based on left and right rotation, vertical rotation, and internal and external rotation. The robot’s head has a built-in motor-driven The eye part of, the eye part can be rotated based on two degrees of freedom, namely, the up and down direction rotation and the left and right direction rotation. The head of the robot is also equipped with an inertial measurement unit IMU for motion control compensation; for the gaze tracking control algorithm, forward and inverse kinematics calculation and IMU compensation can be used, and P (proportional) I (integral) D (derivative) control is integrated. algorithm. Because the motor in the robot has too much freedom, in order to ensure the stability of the system, the IMU feedback angle is required for motion compensation. Especially for eye movement, complex forward and inverse kinematics are required to solve the target position, and gaze tracking cannot be taken into account at the same time. Accuracy, real-time and smoothness, thereby reducing the degree of personification. In addition, due to the multi-motor drive, the start-up time is longer, which reduces the sensitivity of gaze tracking, and has a stronger mechanical sense, which further reduces the effect of the gaze.

Second, adopt a low-degree-of-freedom gaze-following method, such as two degrees of freedom (ie, up-down direction and left-right direction), and mainly use head rotation to achieve gaze tracking of the target. Fig. 14 shows a schematic diagram of a robot with 2 degrees of freedom provided by an embodiment of the present application. As shown in Figure 14, the eye component of the robot is fixedly installed on the head, that is, it cannot move, and the head is equipped with two rotating parts, which can control the head of the robot to rotate in the left and right directions and rotate in the up and down directions. . In the process of gaze tracking, it mainly relies on the deflection of the head to achieve the alignment of the target object. However, the above-mentioned method has a low degree of freedom and cannot move the eyes, so that the degree of personification is low, and the user experience is low.

Compared with the existing robot control technology, the embodiment of the present application uses the output screen of the display module to simulate eye movement. Compared with the realization of eye movement based on a motor-driven rotating part, in addition to having a faster response speed, it also has a faster response speed for motor-driven The calculation amount of the algorithm is also reduced, and the requirements for forward and inverse kinematics calculation are also reduced. It can improve the smoothness of gaze tracking behavior and reduce the computing pressure of the robot, achieve smooth and stable gaze tracking, and effectively improve the robot’s performance. Athletic performance and degree of personification. At the same time, through the display module to simulate eye movement, only the display module needs to be installed on the head of the robot, without the need to configure motor parts, rotating parts and physical eye parts, thereby reducing the cost of the robot.

FIG. 15 shows a method for controlling a robot provided by another embodiment of the present application. In S503, during the rotation of the robot, the specific output screen of the display module for simulating eyeball features on the robot is dynamically adjusted. Realize the flowchart. Referring to FIG. 15, with respect to the embodiment described in FIG. 5, in the robot control method provided in this embodiment, during the rotation of the robot, the display on the robot for simulating eye features is dynamically adjusted according to the target deflection angle The output screen of the module includes: S5031～S5033, the details are as follows:

In S5031, in the process of controlling the rotation of the robot, dynamically adjust the position of the simulated eye feature in the output screen to the eye feature target according to the real-time rotation angle fed back by the robot in real time and the target deflection angle Position; the eye feature target position is the position in the direction in which the line of sight of the eye simulated by the output screen looks toward the target object for the first time.

In this embodiment, the process of robot gaze tracking can be specifically divided into two aspects, one of which is to align the front of the robot with the front of the target object, and the other is to adjust the direction of the robot’s eye line of sight Aim at the key center of the target object. The above two different aspects can be accomplished by different parts of the robot. For the process of aligning the front face of the robot with the front face of the target object, the pose of the robot can be adjusted by controlling the rotation of the robot to realize the alignment of the front face; and the alignment of the eye sight can be completed by the operation of S5031. That is, the position of the simulated eye feature in the output screen of the display module is dynamically adjusted, so that the line of sight of the robot's eye is aligned with the key center of the target object.

It should be noted that controlling the rotation of the robot and adjusting the output screen are performed at the same time. While the robot is controlling the rotating part of the robot to rotate, it can dynamically refresh the position of the simulated eye feature in the output screen of the display module. Among them, the position of the simulated eye feature in the output screen of the display model does not directly jump to the above-mentioned target object position, but can be rotated at the above-mentioned eyeball rotation speed from the initial position before adjustment based on the preset eyeball rotation speed A certain amount of rotation time is required to reach the target position of the eye feature, that is, from the initial position to the position where the target simulates the eye feature. For example, if the angular difference between the target position of the eye feature and the initial position of the eyeball is 30°, and the robot's preset eyeball rotation speed is 120°/s, the initial position of the eyeball will rotate to the target position of the eye feature. The required time is 30/120s=0.25s. That is, it takes 0.25s for the eye sight to align with the target object for the first time.

In this embodiment, since the display module of the robot is on the surface of the main body of the robot, the eye movement in the output screen of the display module will be coupled with the movement of the main body of the robot. The eyeball is equivalent to the rotation angle between the target object, including the relative The rotation angle of the robot (that is, the angle at which the robot controls the rotation of the eyeball), and the rotation of the robot itself (that is, the rotation of the robot through the rotating part). Therefore, the rotation angle of the eyeball relative to the target object is the same as the target offset angle. But in the end, the angle at which the eyeball is equivalent to the eyeball rotation of the robot needs to be adjusted according to the actual angle of the rotating part. Based on this, during the rotation of the robot, the rotating part will feed back the real-time rotation angle to the robot in real time. The robot can superimpose the current rotation angle of the robot's eyeball and the real-time rotation angle of the rotating part to determine whether the above-mentioned target offset angle has been reached. , So as to determine whether the robot's eye sight is aligned with the target object; if the current rotation angle of the eyeball and the real-time rotation angle of the rotating part are superimposed less than the target deflection angle, the robot's eye sight is not aligned with the target object, and continue to control the eyeball and The rotating part rotates; conversely, if the current rotation angle of the eyeball and the real-time rotation angle of the rotating part are superimposed to be equal to the target deflection angle, it is recognized that the eye line of sight of the robot is aligned with the target object, and the operation of S5032 is performed.

It should be noted that the real-time rotation angle fed back by the above-mentioned rotating component is the rotation angle during the entire rotation process from the start of rotation to the time of feedback. The rotating component can feed back the real-time rotation angle to the robot at a preset time interval. For example, the time interval can be 30 μs or 30 ms. When the time interval is short, the real-time rotation angle can be fed back approximately in real time. Among them, the determination process of the above-mentioned rotation angle can be completed by calling the built-in rotation angle control module of the robot. The rotation angle control module can be a P (proportional) D (differential) controller, a P (proportional) I (integral) controller, and a P (proportional) D (differential) controller. (Proportional) I (integral) D (derivative) controller and other control modules. The angle calculation process of the rotation angle control module can be calculated based on the principle of proportional integral derivative, predictive control, sliding mode control and other principles. Make a limit.

In this embodiment, when the robot controls the rotation of the rotating part, it needs to start the motor drive, and drive the rotating part to rotate through the motor drive, so as to realize the mechanical rotation of the robot, but the motor-driven start, motor-driven traction and other operations are time-consuming. Long, and for the display module, the eyeball movement can be realized by only refreshing the output image. Since the output image of the display module can reach a refresh rate of 60Hz, 120Hz or even higher, the eyeball movement is equivalent to an instant response, which is much faster than mechanical rotation. On the other hand, from the perspective of anthropomorphism, in the process of following objects, the eyes often rotate faster than the body. Therefore, the rotation of the robot's body is controlled by rotating parts, and the eye movement is simulated by the display module, which makes the robot The gaze tracking process is more personified, which further improves the user experience. Based on the above reasons, the time it takes for the robot's eyes to align with the key center of the target object is shorter than the front face of the robot aligning with the front face of the target object.

As an example and not a limitation, FIG. 16 shows a schematic diagram of a robot's eye sight aligned with a target object for the first time according to an embodiment of the present application. As shown in FIG. 16, before the gaze tracking behavior, the direction of the robot's eye line of sight is the PA direction, and the front face of the robot is also facing the PA direction at this time. At this time, the target offset angle between the robot's eye direction and the target object is ∠APD, which is θ. At this time, the robot can control the eyeballs in the output screen of the display module to rotate and the robot's body to move. Since the eyeball rotates fast and the rotation angle of the robot's body is superimposed, when the robot's eye line of sight is aligned with the target object, the front face of the robot is still in the PB direction and is not aligned with the target object. At this time, the eyeballs are equivalent. The rotation angle of the robot is α, and the rotation angle of the robot's body equivalent to the target object is β, so the actual rotation angle of the eyeball relative to the target object is α+β=θ. Therefore, the eye line of sight will prioritize the front of the robot to align with the target object.

Further, FIG. 17 shows a specific implementation flowchart of a robot control method S5031 provided by another embodiment of the present application. Referring to FIG. 17, with respect to the embodiment described in FIG. 15, S5031 in a robot control method provided in this embodiment includes: S1701 to S1702, which are detailed as follows:

In S1701, the horizontal deflection amount and the vertical deflection amount are determined based on the real-time rotation angle.

In this embodiment, since the simulated eyeball in the output image of the display module contains two degrees of freedom, it can move in various directions of the plane where the display screen is located, and all moving directions can be decomposed into horizontal and vertical components. Therefore, after the robot receives the real-time rotation angle fed back by the rotating component, the rotation angle can be decomposed in the horizontal direction and the vertical direction to obtain the horizontal deflection amount and the vertical deflection amount.

In a possible implementation, if the body rotation of the robot is controlled by a number of different rotating parts, and different rotating parts correspond to a rotation direction, the above-mentioned real-time rotation angle may be composed of the rotation components of each rotating part , The robot can determine the rotation direction corresponding to each rotating component, superimpose all the rotation components in the horizontal direction to obtain the above-mentioned horizontal deflection amount, and superimpose all the rotation components in the vertical direction to obtain the above-mentioned vertical deflection amount.

In a possible implementation manner, if the posture rotation of the robot is controlled by a multi-degree-of-freedom rotating part, the rotating part can rotate in multiple directions. In this case, the robot can decompose according to the three-dimensional rotation angle fed back by the rotating part to obtain the above-mentioned horizontal deflection amount and vertical deflection amount.

In S1702, determine the eye feature target position according to the target deflection angle, the horizontal deflection amount, and the vertical deflection amount; the real-time offset is specifically:

In this embodiment, the robot can decompose the target deflection angle to obtain the horizontal and vertical components of the target deflection angle. Since the relative movement between the eyeball of the robot and the target object includes the movement of the eyeball relative to the robot and the robot relative to the The movement of the target object, therefore, the actual movement of the eyeball relative to the robot needs to subtract the contribution of the robot's motion to the target object, that is, the horizontal component is calculated by eye_yaw=target_yaw-motor_yaw, and the vertical component is calculated by eye_pitch=target_pitch-motor_pitch. The above-mentioned eye feature target position refers to the offset position of the robot when the eyeball is aligned with the target object in the display screen of the robot.

In the embodiment of this application, by decomposing the target deflection angle in the horizontal and vertical directions, the horizontal component of the position of the target simulated eye feature and the vertical component of the position of the target simulated eye feature can be obtained, thereby uniquely determining The above-mentioned calculation process of the eye feature target position is simple, which can reduce the calculation pressure of the robot.

In S5032, after the line of sight of the eye looks in the direction of the target object for the first time, the position of the simulated eye feature is dynamically adjusted according to the real-time rotation angle to keep the line of sight of the eye looking towards the The direction of the target object.

In this embodiment, after the robot is aligned with the target object for the first time, since the robot is still rotating, that is, the rotation angle of the robot does not reach the preset target deflection angle, in order to keep the eye sight continuously aligned with the target object. At this point, the robot can obtain the real-time rotation angle, and continue to adjust the position of the simulated eye feature of the simulated eye in the output screen of the display module, that is, the position of the simulated eye feature will gradually return to the right, and finally after the rotating part is rotated to the target deflection angle, At this time, the front face of the robot is also facing the target object, and the line of sight of the robot's eyes is also consistent with the front face of the robot, that is, it rotates to the center area of the eyes of the robot (that is, the eyeball returns to the center).

In this embodiment, the above-mentioned method of adjusting the position of the simulated eye feature according to the real-time rotation angle may specifically be: based on the real-time rotation angle fed back by the rotation component at each feedback moment, determine the rotation speed of the rotation component, and control the display module The simulated eyeball moves in the opposite direction of the rotation direction of the rotating member and moves at the above-mentioned rotation speed, so as to offset the offset caused by the rotation of the rotating member, thereby keeping the eye sight aligned with the target object.

In a possible implementation manner, the rotating part of the robot includes a degree of freedom, which is used to control the head to rotate in the left and right directions. FIG. 18 shows a schematic diagram of a robot with one degree of freedom in a rotating part provided by an embodiment of the present application. As shown in Figure 18, when the robot controls the head to rotate left and right, the line of sight of the simulated eyeball on the display module will first be aimed at the target object. At this time, the head of the robot can continue to rotate, and the eyeball of the robot can be adjusted according to the head position. The amount of rotation in the horizontal direction, moving in the opposite direction in the horizontal direction, while keeping the eyeball still in the vertical direction.

In a possible implementation, the robot rotating part contains two degrees of freedom, that is, it is used to control the head to rotate in the left-right direction and the up-down direction. FIG. 19 shows a schematic diagram of a robot with two degrees of freedom in a rotating part provided by an embodiment of the present application. As shown in Fig. 19, the head of the robot can be rotated in two directions. The rotating part 1 controls the head to rotate left and right, and the rotating part 2 controls the head to rotate up and down. Since the rotation of the above two rotating modules takes a long time, the line of sight of the simulated eyeball on the display module will first be aimed at the target object. At this time, the robot can continue to control the head to rotate through the rotating part 1 and the rotating part 2. The eyeball can move in the opposite direction in the horizontal direction according to the amount of rotation of the rotating part 1 in the horizontal direction, and move in the opposite direction in the vertical direction according to the amount of rotation of the rotating part 2 in the vertical direction, so as to realize the line of sight of the robot Keep track of the target object.

In the embodiment of the present application, the robot is coupled with the real-time rotation of the rotating part of the robot during the process of controlling the movement of the eyeball, so that the accuracy of gaze tracking can be improved. Real-time rotation, dynamically adjust the position of the simulated eye features, so as to keep the line of sight always aligned with the target object, thereby improving the degree of personification.

Further, when the torso and head of the robot are separated, the movement of the torso and the head can be controlled by different rotating parts (that is, the head rotating part and the torso rotating part), then in S502 according to the position of the target object Information to control the rotation of the robot may include operations from S2001 to S2003. FIG. 20 shows a specific implementation flow chart of controlling the rotation of the robot according to the position information of the target object in a method S502 for controlling a robot according to another embodiment of the present application. Referring to FIG. 20, with respect to the embodiment described in FIG. 5, S502 in a robot control method provided in this embodiment includes: S2001 to S2003, which are detailed as follows:

In S2001, the torso rotation of the robot is controlled based on the target deflection angle; the target deflection angle is the corresponding deflection angle when the robot's torso rotates from an initial angle to a target object.

In this embodiment, since the movement of the torso of the robot is not affected by the movement of the head and eyeballs, the rotation angle required for the initial angle of the robot to align with the target object is the aforementioned target deflection angle, which can be based on the target deflection angle Control the torso of the robot to rotate. Wherein, the initial angle is the direction angle to which the front of the torso of the robot faces.

Further, FIG. 21 shows a specific implementation flowchart of a robot control method S2001 provided by another embodiment of the present application. Referring to FIG. 21, with respect to the embodiment described in FIG. 20, S2001 in a robot control method provided in this embodiment includes: S2101 to S2103, which are detailed as follows:

In S2101, the first reference rotation angle of the torso of the robot in each control period is acquired in a preset control period.

In this embodiment, the torso rotation of the robot can be controlled by the robot deployed on the torso rotating part of the robot torso. The robot will send a control command to the torso rotating part in a preset control cycle, and each control command sent may include It is related to the rotation angle of the trunk rotating component during the entire rotation process, and the rotation angle is dynamically adjusted according to the actual rotation amount fed back in each control cycle, so as to ensure the accuracy of the rotation operation. Therefore, when the robot sends the first cycle rotation instruction to the trunk rotation component, it needs to determine the current robot's posture, that is, the rotation angle of the robot's torso at the start time corresponding to the control cycle, that is, the aforementioned first reference rotation angle.

In S2102, according to the first reference rotation angle of the current control period, the first tracking error angle of the previous control period, and the trunk rotation angle, the first period rotation corresponding to the current control period is generated Instruction; the rotation angle in the first cycle rotation instruction is specifically:

In this embodiment, since the movement of the trunk rotating component is not uniform and stable, there will be a certain rotation error during the actual rotation. In order to eliminate the errors caused by mechanical rotation of the robot, the rotation angle can be adjusted in real time _{through the motor 0} _Kp and motor _{0 _Kd corresponding to the torso rotating parts.} The history rotation record in the history control operation is determined by the way of big data learning. Of course, it is also possible to update the above adjustment parameters in real time by communicating with the cloud server. The methods for obtaining the aforementioned preset parameters are not limited one by one here. Among them, motor ₀ _Kp is the calibration parameter of the current cycle; motor ₀ _Kd is the calibration parameter of the cycle iteration.

In this embodiment, the first tracking error angle is the angle that represents the phase difference with the target deviation angle, that is, the angle at which the body of the robot still needs to be rotated. Since current_motor ₀ _yaw is the deflection angle of the current robot's torso, that is, the first A reference angle of rotation. When the angle is the same as the target deflection angle, the above value is 0, and there is no deviation. The front of the robot's torso is aligned with the target object, and there is no need to continue to rotate; on the contrary, if the above value is non-zero, That is, the robot still needs to continue to rotate and generate the first cycle control command.

In S2103, the torso of the robot is controlled to rotate according to each of the first cycle rotation instructions.

In this embodiment, the robot generates a first-cycle rotation command in each control cycle. The first-cycle rotation command includes the above-mentioned determining the required rotation angle of the rotating component, and sends the first-cycle control command to The torso rotating part is used to control the torso rotating part to rotate until the rotation angle of the target rotating part reaches the target deflection angle. At this time, it is recognized that the current frontal pose of the robot is aligned with the target object.

In the embodiment of the present application, by generating a first-cycle rotation command for controlling the body rotating part in each cycle, the cycle precision control of the rotating part is realized, and when the first-cycle rotation command is generated, the motor ₀ _Kp And motor ₀ _Kd to eliminate mechanical rotation deviation and improve the accuracy of robot control.

In S2002, during the rotation of the torso of the robot, the head of the robot is dynamically controlled to rotate to the target head position according to the real-time torso angle fed back in real time and the target deflection angle. Is the position of the robot head facing the target object for the first time.

In this embodiment, the process of the robot aligning the front face of the head with the front face of the target object can be specifically divided into two aspects, one of which is aligning the front face of the robot head with the front face of the target object, and the other On the one hand, the front face of the robot torso is aligned with the front face of the target object. When the above two parts are aligned with the front face of the target object, it is recognized that the robot has been aligned with the target object. The above two different aspects can be accomplished by different parts of the robot. For the process of aligning the front face of the robot's head with the front face of the target object, the head position of the robot can be adjusted through the head rotation part of the robot, so as to realize the alignment of the front face of the head with the front face of the target object; and For the process of aligning the front face of the robot torso with the front face of the target object, the head position of the robot can be adjusted through the torso rotating part of the robot, so as to realize the alignment of the front face of the head with the front face of the target object.

It should be noted that S2001 and S2002 are executed at the same time, so while the robot controls the torso rotating part of the robot to perform a rotating operation, it can control the head rotating part of the robot to perform a rotating operation. Wherein, the rotation speed of the head rotating part relative to the target object will be higher than the rotation speed of the body rotating part relative to the target object, that is, the head part will be aligned with the front face of the target object faster than the body part. The reasons are as follows:

Since the head of the robot is mounted on the torso of the robot, the rotation of the head part will be coupled with the motion of the torso of the robot. The rotation angle of the head part relative to the target object includes the rotation of the head part relative to the robot. The angle (that is, the angle at which the robot controls the head part), and the rotation of the robot's torso (that is, the rotation of the robot through the torso rotating part), therefore, the actual rotation angle of the head part needs to be adjusted according to the actual rotation angle of the torso rotating part. Based on this, during the rotation of the torso rotating part, the torso rotating part will feed back the real-time torso angle to the robot in real time. The robot can superimpose the current head rotation angle of the robot with the real-time torso angle of the torso rotating part to determine whether it has reached the above The target offset angle, so as to determine whether the front face of the robot's head is aligned with the target object; if the current head rotation angle and the real-time torso angle of the torso rotating part are superimposed less than the target deflection angle, the front face of the robot's head faces If the target object is not aligned, continue to control the head rotating part and the torso rotating part to rotate; conversely, if the current head rotation angle and the real-time torso angle of the torso rotating part are superimposed equal to the target deflection angle, the head of the robot is recognized The front face has been aligned with the target object.

In terms of anthropomorphism, in the process of following objects, the head rotates faster than the torso. Therefore, the torso rotation of the robot is controlled by the torso rotation component, and the head rotation is controlled by coupling the torso rotation. The component controls the rotation of the head of the robot, so that the head is aligned with the target object prior to the torso. The gaze tracking process of the robot is more personified, which further improves the user experience.

In S2003, after the head of the robot faces the direction of the target object for the first time, the head of the robot is dynamically controlled to rotate according to the real-time torso angle to keep the head of the robot facing the target object. direction.

In this embodiment, after the front face of the robot head is aligned with the target object for the first time, because the torso rotating part is still rotating, that is, the rotation angle of the torso rotating part does not reach the preset target deflection angle, in order to be able to hold the robot head The front face of the department continuously aligns the target object. At this point, the robot can obtain the real-time torso angle of the torso rotating part, and continue to control the head of the robot to rotate through the head rotating part, that is, gradually return the head to the center, and finally after the torso rotating part rotates to the target deflection angle, the robot torso The front face of is also facing the target object, and the front face of the robot's head is also consistent with the front face of the robot's torso, that is, the head is returned to the center.

In this embodiment, the above-mentioned method of adjusting the position of the simulated eye feature according to the real-time rotation angle may specifically be: based on the real-time torso angle fed back by the torso rotation component at each feedback moment, determine the torso rotation speed of the torso rotation component, and control the head The part rotation member moves in the opposite direction of the rotation direction of the trunk rotation member and moves at the above-mentioned trunk rotation speed, so as to offset the offset caused by the rotation of the trunk rotation member, thereby keeping the front face of the head facing the target object.

In the embodiments of the present application, the robot is coupled with the real-time rotation of the rotating parts of the robot's torso during the process of controlling the rotation of the head, so that the accuracy of gaze tracking can be improved, and the front of the head is aligned with the target object for the first time Later, the rotation of the head rotating part can be dynamically adjusted according to the real-time rotation amount of the torso rotating part, so that the front face of the head can always be aligned with the target object, so that the degree of personification can be improved.

Further, when the head rotation of the robot includes two degrees of freedom, for example, two different head rotation parts (ie the first head rotation part for left and right rotation and the second head rotation part for up and down rotation can be used). In the case where the rotating part) controls the head to rotate left and right and rotate up and down, S2002 may include the operations of S2201 to S2203. FIG. 22 shows a specific implementation flowchart of a robot control method S2002 provided by another embodiment of the present application. Referring to FIG. 22, with respect to the embodiment described in FIG. 20, S2202 in a robot control method provided in this embodiment includes: S2201 to S2203, which are detailed as follows:

In S2201, the horizontal deflection angle and the vertical deflection angle are determined based on the target deflection angle.

In this embodiment, since the robot head includes two degrees of freedom, the first head rotating part is used to control the head to rotate in the horizontal direction, and the second head part is used to control the rotation in the vertical direction. Therefore, the robot can decompose the target deflection angle in the horizontal direction and the vertical direction to obtain the horizontal deflection angle and the vertical deflection angle.

FIG. 23 shows a schematic diagram of a robot with three degrees of freedom in a rotating part provided by an embodiment of the present application. As shown in Figure 23, the head of the robot can be rotated in two directions. Rotating part 1 controls the head to rotate left and right, and rotating part 2 controls the head to rotate up and down. The torso of the robot can rotate in one direction. That is, the body is controlled to rotate in the left-right direction by the rotating part 0. Since only the second head rotating part is controlled in the vertical direction, the second head rotating part assumes all the vertical rotation; while in the horizontal direction, there is a first head rotating part used to control the left and right rotation of the head (ie, rotating Part 1) and the torso rotating part (ie, rotating part 0) used to control the left and right rotation of the torso, so the real-time torso angle of the torso needs to be considered in the process of head rotation.

In S2202, the head of the robot is controlled to rotate in the vertical direction according to the vertical deflection angle.

In this embodiment, since only the second head rotation component is controlled in the vertical direction, the rotation of the torso does not affect the vertical rotation angle, so the second head rotation component can be controlled according to the vertical deflection angle determined above. Turn to move the head to the target position in the vertical direction.

Further, FIG. 24 shows a specific implementation flowchart of a robot control method S2202 provided by another embodiment of the present application. Referring to FIG. 24, with respect to the embodiment described in FIG. 22, S2202 in a robot control method provided in this embodiment includes: S2401 to S2403, which are detailed as follows:

In S2401, the second reference rotation angle of the head of the robot in each control period in the vertical direction is acquired in a preset control period.

In this embodiment, the robot will send a control command to the second head rotating part in a preset control cycle. Each control command sent may include information about the vertical direction of the second head rotating part during the entire rotation. The rotation angle of the vertical direction is dynamically adjusted according to the actual vertical rotation amount fed back in each control cycle. Therefore, when the robot sends the second cycle rotation instruction to the second head rotation component, it needs to determine the current posture of the robot head in the vertical direction, that is, the rotation angle of the robot head in the vertical direction at the beginning of the control cycle, That is, the above-mentioned second reference rotation angle.

In S2402, according to the second reference rotation angle of the current control period, the second tracking error angle of the previous control period, and the vertical deflection angle, a second period rotation corresponding to the current control period is generated Instruction; the rotation angle in the second cycle rotation instruction is specifically:

Wherein, motor ₂ _pitch is the rotation angle in the second cycle rotation command; current_motor ₂ _pitch is the second reference rotation angle of the current control cycle; target_pitch is the vertical deflection angle; motor ₁ _yaw_diff is the current control The third tracking error angle of the cycle; motor ₁ _yaw_last_diff is the third tracking error angle of the previous control cycle; motor ₂ _Kp and motor ₂ _Kd are preset adjustment parameters of the second head rotating part.

In this embodiment, since the operation of the second head rotating part is not uniform and stable, there will be a certain rotation error during the actual rotation. In order to eliminate the error caused by mechanical rotation of the robot _{, the rotation angle can be adjusted in real time through the motor 2} _Kp and motor ₂ _Kd corresponding to the second head rotating part. The above two parameters can be configured by default when the robot is shipped from the factory, or Based on the historical rotation record of the robot in the historical control operation, it is determined by way of big data learning. Of course, it is also possible to update the above-mentioned adjustment parameters in real time by communicating with the cloud server. The methods for obtaining the aforementioned preset parameters are not limited one by one here. Among them, motor ₂ _Kp is the calibration parameter of the current cycle; motor ₂ _Kd is the calibration parameter of the cycle iteration.

In this embodiment, the second tracking error angle is the angle of the vertical deviation from the target deviation angle, that is, the angle at which the robot's head still needs to rotate in the vertical direction. Because current_motor ₂ _pitch is the current robot's head The deflection angle in the vertical direction, that is, the second reference rotation angle. When the angle is the same as the deflection angle of the vertical line of defense of the target deflection angle, and the above value is 0, there is no deviation. The head of the robot is in the vertical direction. The target position has been reached, and there is no need to continue to rotate; on the contrary, if the above-mentioned value is non-zero, that is, the second head rotating part still needs to continue to rotate and generate a second cycle control command.

In S2403, the head of the robot is controlled to rotate in the vertical direction according to each of the second cycle rotation instructions.

In this embodiment, the robot generates a second-cycle rotation instruction in each control cycle. The second-cycle rotation instruction includes the above-mentioned determination of the required rotation angle of the second head rotating part, and the second cycle is changed in each control cycle. The control instruction is sent to the second head rotating part to control the second head rotating part to rotate until the vertical deflection angle is reached.

In the embodiment of the present application, by generating a second cycle rotation command for controlling the second head rotation component in each cycle, precise cycle control of the second head rotation component is achieved, and the second cycle rotation is generated. When instructing, the motor ₂ _Kp and motor ₂ _Kd are used to eliminate the mechanical rotation deviation and improve the accuracy of robot control.

In S2203, during the rotation of the torso of the robot, the head of the robot is dynamically controlled to rotate in the horizontal direction to a horizontal target position according to the real-time torso angle and the horizontal deflection angle fed back in real time; The target position is the position of the robot head facing the target object in the horizontal direction for the first time.

In this embodiment, since the head of the robot is mounted on the torso of the robot, the horizontal rotation of the head part will be coupled with the horizontal movement of the torso of the robot. The rotation angle includes the rotation angle of the head part relative to the robot in the horizontal direction (that is, the angle at which the robot controls the first head part), and the rotation of the robot torso in the horizontal direction (that is, the rotation of the robot through the torso rotating part) Therefore, the actual rotation angle of the head part in the horizontal direction needs to be adjusted according to the actual rotation angle of the trunk rotation part in the horizontal direction.

Further, FIG. 25 shows a specific implementation flowchart of a robot control method S2203 provided by another embodiment of the present application. Referring to FIG. 25, with respect to the embodiment described in FIG. 22, S2203 in a robot control method provided in this embodiment includes: S2501 to S2503, which are detailed as follows:

In S2501, the third reference rotation angle of the head of the robot in each control period in the horizontal direction is acquired in a preset control period.

In this embodiment, similar to S2401, the robot obtains the third reference rotation angle from the second head rotating part in a preset control cycle. For specific parameters, please refer to the relevant description of S2401, which will not be repeated here.

In S2502, the current control period is generated according to the third reference rotation angle of the current control period, the third tracking error angle of the previous control period, the real-time torso angle, and the horizontal deflection angle The corresponding third cycle rotation command; the rotation angle in the third cycle rotation command is specifically:

In this embodiment, similarly, in order to eliminate the error of mechanical motion, when determining the rotation angle in the third cycle rotation command, motor ₁ _Kp and motor ₁ _Kd can be introduced to realize parameter correction.

_{In this embodiment, since the current_motor 1} _yaw is coupled with the torso left and right rotation current_motor ₀ _yaw in the process of performing the left and right rotation of the first head rotation component, the aforementioned coupling influence needs to be considered when calculating the third tracking error angle.

In S2503, the head of the robot is dynamically controlled to rotate in the horizontal direction according to each of the three-cycle rotation instructions.

In this embodiment, the robot generates a third-cycle rotation instruction in each control cycle. The third-cycle rotation instruction includes the above-mentioned determining the required rotation angle of the first head rotating part, and the third cycle is changed in each control cycle. The control instruction is sent to the first head rotating part to control the first head rotating part to rotate until the horizontal deflection angle is reached.

In the embodiment of the present application, the horizontal rotation amount of the robot torso is considered in the process of controlling the horizontal rotation of the head, so that the accuracy of the rotation control can be improved.

It should be understood that the size of the sequence number of each step in the foregoing embodiment does not mean the order of execution. The execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

Corresponding to the robot control method described in the above embodiment, FIG. 26 shows a structural block diagram of a robot control device provided in an embodiment of the present application. For ease of description, only parts related to the embodiment of the present application are shown.

Referring to Figure 26, the robot control device includes:

The location information acquiring unit 261 is configured to acquire location information of the target object;

The rotation control unit 262 is configured to control the rotation of the robot according to the position information of the target object;

The screen adjustment unit 263 is configured to dynamically adjust the output screen of the display module for simulating eye features on the robot during the rotation of the robot, so that the output screen of the display module on the robot The eye line of sight that simulates the eye features is the direction of looking at the target object during the rotation of the robot.

The image adjustment unit 263 is specifically configured to: during the rotation of the robot, dynamically adjust the output image of the display module for simulating eye features on the robot according to the target deflection angle; the target deflection angle is the initial angle of the robot. Rotate to the corresponding deflection angle when facing the target object.

Optionally, the picture adjustment unit 263 includes:

The eye movement control unit is used to dynamically adjust the position of the simulated eye feature in the output screen according to the real-time rotation angle fed back by the robot in real time and the target deflection angle in the process of controlling the rotation of the robot Eye feature target position; the eye feature target position is the position where the eye's line of sight simulated by the output screen looks in the direction of the target object for the first time;

The eye correction control unit is further configured to dynamically adjust the position of the simulated eye feature according to the real-time rotation angle after the eye line of sight is in the direction of the target object for the first time, so as to maintain the eye The line of sight looks in the direction of the target object.

Optionally, the eye movement control unit includes:

A deflection component determining unit, configured to determine a horizontal deflection amount and a vertical deflection amount based on the real-time rotation angle;

The eye feature target position determining unit is configured to determine the eye feature target position according to the target deflection angle, the horizontal deflection amount, and the vertical deflection amount; the real-time offset is specifically:

Optionally, the rotation control unit 262 includes:

The torso rotation control unit is configured to control the torso rotation of the robot based on the target deflection angle; the target deflection angle is the corresponding deflection angle when the torso of the robot rotates from an initial angle to a target object;

The head rotation control unit is used to dynamically control the head of the robot to rotate to the target head position according to the real-time torso angle fed back in real time and the target deflection angle during the rotation of the torso of the robot. The head target position is the position of the robot head facing the target object for the first time;

The head rotation return unit is used to dynamically control the head rotation of the robot according to the real-time torso angle after the robot head faces the direction of the target object to keep the robot head facing The direction of the target object.

Optionally, the trunk rotation control unit includes:

A first reference rotation angle determining unit, configured to obtain a first reference rotation angle of the robot's torso in each control period in a preset control period;

The first cycle rotation instruction generating unit is configured to generate the current control cycle according to the first reference rotation angle of the current control cycle, the first tracking error angle of the previous control cycle, and the trunk rotation angle The corresponding first cycle rotation command; the rotation angle in the first cycle rotation command is specifically:

Wherein, motor ₀ _yaw is the rotation angle in the first cycle rotation command; current_motor ₀ _yaw is the first reference rotation angle of the current control cycle; target_yaw is the target deflection angle; motor ₀ _yaw_diff is the current control The first tracking error angle of the cycle; motor ₀ _yaw_last_diff is the first tracking error angle of the previous control cycle; motor ₀ _Kp and motor ₀ _Kd are the preset adjustment parameters of the trunk rotation part;

The first cycle rotation instruction execution unit is configured to control the torso of the robot to rotate according to each of the first cycle rotation instructions.

Optionally, the head rotation control unit includes:

A rotation angle component determining unit, configured to determine a horizontal deflection angle and a vertical deflection angle based on the target deflection angle;

The first head rotation control unit is configured to control the head of the robot to rotate in the vertical direction according to the vertical deflection angle; and at the same time,

The second head rotation control unit is used to dynamically control the head of the robot to rotate to the horizontal in the horizontal direction according to the real-time torso angle and the horizontal deflection angle fed back in real time during the rotation of the torso of the robot Target position; the horizontal target position is the position of the robot head facing the target object in the horizontal direction for the first time.

Optionally, the first head rotation control unit includes:

The second reference rotation angle determination unit is configured to obtain the second reference rotation angle of the head of the robot in the vertical direction in each control period in a preset control period;

The second cycle rotation command generating unit is configured to generate the current control cycle according to the second reference rotation angle of the current control cycle, the second tracking error angle of the previous control cycle, and the vertical deflection angle The corresponding second cycle rotation command; the rotation angle in the second cycle rotation command is specifically:

Wherein, motor ₂ _pitch is the rotation angle in the second cycle rotation command; current_motor ₂ _pitch is the second reference rotation angle of the current control cycle; target_pitch is the vertical deflection angle; motor ₂ _pitch_diff is the current control Cycle second tracking error angle; motor ₂ _pitch_last_diff is the second tracking error angle of the previous control cycle; motor ₂ _Kp and motor ₂ _Kd are preset adjustment parameters of the second head rotating part;

The second cycle rotation instruction execution unit is configured to control the head of the robot to rotate in the vertical direction according to each of the second cycle rotation instructions.

Optionally, the second head rotation control unit includes:

The third reference rotation angle determination unit is configured to obtain the third reference rotation angle of the head of the robot in each control period in the horizontal direction in a preset control period;

The third cycle rotation instruction generating unit is used to generate the third reference rotation angle of the current control cycle, the third tracking error angle of the previous control cycle, the real-time torso angle, and the horizontal deflection angle, The third cycle rotation command corresponding to the current control cycle is generated; the rotation angle in the third cycle rotation command is specifically:

Wherein, motor ₁ _yaw is the rotation angle in the third cycle rotation command; current_motor ₁ _yaw is the third reference rotation angle of the current control cycle; target_yaw is the horizontal deflection angle; motor ₁ _yaw_diff is the current control The third tracking error angle of the cycle; motor ₁ _yaw_last_diff is the third tracking error angle of the previous control cycle; motor ₁ _Kp and motor ₁ _Kd are the preset adjustment parameters of the first head rotating part; current_motor ₀ _yaw Is the real-time torso angle;

The third cycle rotation instruction execution unit is used to dynamically control the head of the robot to rotate in the horizontal direction according to each of the three cycle rotation instructions.

Optionally, the location information acquiring unit 261 includes:

A scene image acquisition unit, configured to acquire a scene image containing the target object through a camera module built into the robot;

The object center coordinate determining unit is configured to mark the boundary coordinates of the target object from the scene image, and determine the position information of the target object of the target object according to the boundary coordinates; the position information of the target object includes The object center coordinates of the target object;

A target deflection angle calculation unit, configured to determine the target deflection angle according to the object center coordinates and the image center coordinates of the scene image;

Optionally, the scene image acquisition unit includes:

The image correction parameter acquisition unit is configured to determine the image correction parameter according to the offset between the position of the simulated eye feature of the output screen of the camera module and the display module;

The image correction execution unit is configured to adjust the original image collected by the camera module based on the image correction parameter to generate the scene image.

Therefore, the robot control device provided by the embodiments of the present application can also determine the target deflection angle according to the relative position between the target object and the robot, and adjust the output screen and rotating parts of the display module based on the target deflection angle to achieve eye tracking. Since the eye movement of the robot is simulated through the output interface of the display module, no motor drive is required to achieve deflection during the movement, the response time is short, smooth eye tracking is realized, and the degree of personification and accuracy of eye tracking are improved.

FIG. 27 is a schematic structural diagram of a robot provided by an embodiment of the application. As shown in FIG. 27, the robot 27 of this embodiment includes: at least one processor 270 (only one is shown in FIG. 27), a processor, a memory 271, and a memory 271 that is stored in the memory 271 and can be used in the at least one processor. A computer program 272 running on the 270. The processor 270 implements the steps in any of the foregoing embodiments of the control method for each robot when the computer program 272 is executed.

The robot 27 may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server. The robot may include, but is not limited to, a processor 270 and a memory 271. Those skilled in the art can understand that FIG. 27 is only an example of the robot 27, and does not constitute a limitation on the robot 27. It may include more or less parts than shown, or a combination of some parts, or different parts, such as It can also include input and output devices, network access devices, and so on.

The so-called processor 270 may be a central processing unit (Central Processing Unit, CPU), and the processor 270 may also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSPs), and application specific integrated circuits (Application Specific Integrated Circuits). , ASIC), ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.

In some embodiments, the memory 271 may be an internal storage unit of the robot 27, such as a hard disk or a memory of the robot 27. In other embodiments, the memory 271 may also be an external storage device of the robot 27, such as a plug-in hard disk equipped on the robot 27, a smart media card (SMC), and a secure digital (Secure Digital). Digital, SD) card, flash card, etc. Further, the memory 271 may also include both an internal storage unit of the robot 27 and an external storage device. The memory 271 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 271 can also be used to temporarily store data that has been output or will be output.

It should be noted that the information interaction and execution process between the above-mentioned devices/units are based on the same concept as the method embodiment of this application, and its specific functions and technical effects can be found in the method embodiment section. I won't repeat it here.

Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the division of the above functional units and modules is used as an example. In practical applications, the above functions can be allocated to different functional units and modules as needed. Module completion, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist alone physically, or two or more units can be integrated into one unit. The above-mentioned integrated units can be hardware-based Formal realization can also be realized in the form of a software functional unit. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the present application. For the specific working process of the units and modules in the foregoing system, reference may be made to the corresponding process in the foregoing method embodiment, which will not be repeated here.

An embodiment of the present application also provides a network device, which includes: at least one processor, a memory, and a computer program stored in the memory and running on the at least one processor, and the processor executes The computer program implements the steps in any of the foregoing method embodiments.

The embodiments of the present application also provide a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in each of the foregoing method embodiments can be realized.

The embodiments of the present application provide a computer program product. When the computer program product runs on a mobile terminal, the steps in the foregoing method embodiments can be realized when the mobile terminal is executed.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the implementation of all or part of the processes in the above-mentioned embodiment methods in the present application can be accomplished by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed by the processor, the steps of the foregoing method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms. The computer-readable medium may at least include: any entity or device capable of carrying computer program code to the camera/robot, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory ( RAM, Random Access Memory), electric carrier signal, telecommunications signal and software distribution medium. For example, U disk, mobile hard disk, floppy disk or CD-ROM, etc. In some jurisdictions, according to legislation and patent practices, computer-readable media cannot be electrical carrier signals and telecommunication signals.

In the above-mentioned embodiments, the description of each embodiment has its own focus. For parts that are not described in detail or recorded in an embodiment, reference may be made to related descriptions of other embodiments.

A person of ordinary skill in the art may realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.

In the embodiments provided in this application, it should be understood that the disclosed apparatus/network equipment and method may be implemented in other ways. For example, the device/network device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units. Or components can be combined or integrated into another system, or some features can be omitted or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, a person of ordinary skill in the art should understand that it can still implement the foregoing The technical solutions recorded in the examples are modified, or some of the technical features are equivalently replaced; these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the application, and should be included in Within the scope of protection of this application.

Claims

A method for controlling a robot, which is characterized in that it comprises:

Obtain the location information of the target object;

Controlling the rotation of the robot according to the position information of the target object;

During the rotation of the robot, the output screen of the display module for simulating eye features on the robot is dynamically adjusted, so that the output screen of the display module on the robot can simulate the eye features of the eye. The partial line of sight is the direction of looking at the target object during the rotation of the robot.
The control method according to claim 1, wherein the dynamic adjustment of the output screen of the display module for simulating eye features on the robot during the rotation of the robot is specifically:

During the rotation of the robot, the output screen of the display module for simulating eye features on the robot is dynamically adjusted according to the target deflection angle; the target deflection angle is the corresponding deflection angle when the robot is rotated from the initial angle to facing the target object .
The control method according to claim 2, characterized in that, during the rotation of the robot, dynamically adjusting the output screen of the display module for simulating eye features on the robot according to the target deflection angle, comprising:

In the process of controlling the rotation of the robot, dynamically adjust the position of the simulated eye feature in the output screen to the target position of the eye feature according to the real-time rotation angle fed back by the robot in real time and the target deflection angle; The eye feature target position is the corresponding position when the eye sight simulated by the output screen looks in the direction of the target object for the first time;

After the line of sight of the eye looks in the direction of the target object for the first time, dynamically adjust the position of the simulated eye feature according to the real-time rotation angle to maintain the direction of the line of sight of the eye to the target object .
The control method according to claim 3, wherein, in the process of controlling the rotation of the robot, the output screen is dynamically adjusted according to the real-time rotation angle fed back by the robot in real time and the target deflection angle. The simulated eye feature position to the eye feature target position includes:

Determining the amount of horizontal deflection and the amount of vertical deflection based on the real-time rotation angle;

According to the target deflection angle, the horizontal deflection amount, and the vertical deflection amount, the eye characteristic target position is determined; the real-time offset is specifically:

Wherein, eye_yaw is the horizontal component of the eye characteristic target position; eye_pitch is the vertical component of the eye characteristic target position; target_yaw is the horizontal component of the target deflection angle; target_pitch is the vertical component of the target deflection angle; motor_yaw is the horizontal deflection amount; motor_pitch is the horizontal deflection amount.
The control method according to claim 1, wherein the controlling the rotation of the robot according to the position information of the target object comprises:

Control the torso rotation of the robot based on the target deflection angle; the target deflection angle is the corresponding deflection angle when the torso of the robot rotates from the initial angle to the target object;

During the rotation of the torso of the robot, according to the real-time torso angle and the target deflection angle fed back in real time, the head of the robot is dynamically controlled to rotate to the target head position, and the target head position is the robot The corresponding position when the head faces the direction of the target object for the first time;

After the head of the robot faces the direction of the target object for the first time, according to the real-time torso angle, the head of the robot is dynamically controlled to rotate to keep the head of the robot facing the direction of the target object.
The control method according to claim 5, wherein the controlling the torso rotation of the robot based on the target deflection angle comprises:

Acquiring the first reference rotation angle of the torso of the robot in each control period in a preset control period;

Generate a first cycle rotation instruction corresponding to the current control cycle according to the first reference rotation angle of the current control cycle, the first tracking error angle of the previous control cycle, and the trunk rotation angle;

The torso of the robot is controlled to rotate according to each of the first cycle rotation instructions.
The control method according to claim 5, wherein during the rotation of the torso of the robot, the head of the robot is dynamically controlled to rotate to the head target according to the real-time torso angle fed back in real time and the target deflection angle Position, the head target position is the position of the robot head facing the target object for the first time, including:

Determining a horizontal deflection angle and a vertical deflection angle based on the target deflection angle;

Controlling the head of the robot to rotate in a vertical direction according to the vertical deflection angle;

And at the same time, during the rotation of the torso of the robot, according to the real-time torso angle and the horizontal deflection angle fed back in real time, the head of the robot is dynamically controlled to rotate in the horizontal direction to a horizontal target position; the horizontal target position Is the position of the robot head facing the target object in the horizontal direction for the first time.
The control method according to claim 7, wherein the controlling the head of the robot to rotate in the vertical direction according to the vertical deflection angle comprises:

Acquiring the second reference rotation angle of the head of the robot in the vertical direction in each control period in a preset control period;

Generate a second cycle rotation command corresponding to the current control cycle according to the second reference rotation angle of the current control cycle, the second tracking error angle of the previous control cycle, and the vertical deflection angle;

The head of the robot is controlled to rotate in the vertical direction according to each of the second cycle rotation instructions.
The control method according to claim 7, characterized in that, during the rotation of the torso of the robot, according to the real-time torso angle fed back in real time and the horizontal deflection angle, the head of the robot is dynamically controlled to Rotate in the horizontal direction to the horizontal target position, including:

Acquiring the third reference rotation angle of the head of the robot in each control period in the horizontal direction in a preset control period;

According to the third reference rotation angle of the current control period, the third tracking error angle of the previous control period, the real-time torso angle, and the horizontal deflection angle, the third reference rotation angle corresponding to the current control period is generated. Cycle rotation instruction;

The head of the robot is dynamically controlled to rotate in the horizontal direction according to each of the three-cycle rotation instructions.
The control method according to any one of claims 1-9, wherein said acquiring location information of the target object comprises:

Acquiring a scene image containing the target object through a camera module built into the robot;

Mark the boundary coordinates of the target object from the scene image, and determine the position information of the target object of the target object according to the boundary coordinates; the position information of the target object includes the object center coordinates of the target object .
The control method according to claim 10, wherein the acquiring a scene image containing the target object through a camera module built into the robot comprises:

Determine the image correction parameter according to the offset between the position of the simulated eye feature of the output screen of the camera module and the display module;

The original image collected by the camera module is adjusted based on the image correction parameter to generate the scene image.
A robot control device is characterized in that it comprises:

The location information acquiring unit is used to acquire the location information of the target object;

A rotation control unit for controlling the rotation of the robot according to the position information of the target object;

The screen adjustment unit is used to dynamically adjust the output screen of the display module for simulating eye features on the robot during the rotation of the robot, so that the output screen of the display module on the robot is The line of sight of the eye that simulates the eye feature is the direction of looking at the target object during the rotation of the robot.
The control device according to claim 12, wherein the picture adjustment unit is specifically configured to dynamically adjust the display module of the robot for simulating eye characteristics according to the target deflection angle during the rotation of the robot. Output screen; the target deflection angle is the corresponding deflection angle when the robot rotates from the initial angle to face the target object.
The control device according to claim 13, wherein the picture adjustment unit comprises:

The eye movement control unit is used to dynamically adjust the position of the simulated eye feature in the output screen according to the real-time rotation angle fed back by the robot in real time and the target deflection angle in the process of controlling the rotation of the robot Eye feature target position; the eye feature target position is the position where the eye's line of sight simulated by the output screen looks in the direction of the target object for the first time;

The eye correction control unit is further configured to dynamically adjust the position of the simulated eye feature according to the real-time rotation angle after the eye line of sight is in the direction of the target object for the first time, so as to maintain the eye The line of sight looks in the direction of the target object.
The control device according to claim 12, wherein the rotation control unit comprises:

The torso rotation control unit is configured to control the torso rotation of the robot based on the target deflection angle; the target deflection angle is the corresponding deflection angle when the torso of the robot rotates from an initial angle to a target object;

The head rotation control unit is used to dynamically control the head of the robot to rotate to the target head position according to the real-time torso angle fed back in real time and the target deflection angle during the rotation of the torso of the robot. The head target position is the position of the robot head facing the target object for the first time;

The head rotation return unit is used to dynamically control the head rotation of the robot according to the real-time torso angle after the robot head faces the direction of the target object for the first time to keep the robot head facing The direction of the target object.
The control device according to claim 15, wherein the head rotation control unit comprises:

A rotation angle component determining unit, configured to determine a horizontal deflection angle and a vertical deflection angle based on the target deflection angle;

The first head rotation control unit is configured to control the head of the robot to rotate in the vertical direction according to the vertical deflection angle; and at the same time,

The second head rotation control unit is used to dynamically control the head of the robot to rotate to the horizontal in the horizontal direction according to the real-time torso angle and the horizontal deflection angle fed back in real time during the rotation of the torso of the robot Target position; the horizontal target position is the position of the robot head facing the target object in the horizontal direction for the first time.
A robot comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program as claimed in claims 1 to 11 Any of the methods.
A robot comprising a processor, a display, and a transmission component for controlling the rotation of the robot, wherein the processor implements the method according to any one of claims 1 to 11 when the processor executes a computer program, and the transmission component The robot is controlled to rotate according to the control instructions output by the computer program, and the display dynamically adjusts the output screen used to simulate eye features according to the output instructions of the computer program.
A computer-readable storage medium storing a computer program, wherein the computer program implements the method according to any one of claims 1 to 11 when the computer program is executed by a processor.