WO2021102855A1

WO2021102855A1 - Mobile platform, terminal device and control method therefor, and storage medium

Info

Publication number: WO2021102855A1
Application number: PCT/CN2019/121766
Authority: WO
Inventors: 舒路; 苏冠华
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2019-11-28
Filing date: 2019-11-28
Publication date: 2021-06-03
Also published as: CN112292867A

Abstract

A mobile platform, a terminal device and a control method therefor, and a storage medium. The method comprises: a terminal device acquires a terminal sound file according to environmental sound and sends the terminal sound file to a mobile platform (S110); the mobile platform decodes the terminal sound file and then performs playback (S120); the mobile platform generates a platform sound file according to environmental sound (S130); and after receiving the platform sound file, the terminal device plays back platform audio information obtained by decoding the platform sound file (S140).

Description

Movable platform, terminal equipment and its control method and storage medium

Technical field

This specification relates to the field of movable platforms, and in particular to a movable platform, a terminal device, a control method thereof, and a storage medium.

Background technique

With the development of technology, mobile platforms gradually have more and more applications. Movable platforms such as mobile robots and unmanned aerial vehicles can usually communicate with terminal devices such as mobile phones and remote controls to realize functions such as command transmission and image transmission. But these functions are only limited to realize the interactive mode between the terminal equipment and the mobile platform, which is relatively simple.

Summary of the invention

Based on this, this specification provides a mobile platform, a terminal device and its control method, and a storage medium, which can realize the interactive mode of voice transmission between the terminal device and the mobile platform, such as realizing processes such as voice intercom.

In the first aspect, this specification provides a control method applied to a system composed of a terminal device and a movable platform, and both the terminal device and the movable platform are provided with audio sensors and speakers;

The method includes:

The terminal device obtains a terminal sound file according to the environmental sound of the terminal device, and sends the terminal sound file to the mobile platform;

The mobile platform receives the terminal sound file sent by the terminal device, decodes the terminal sound file, and plays it;

The movable platform generates a platform sound file according to the environmental sound of the movable platform;

After receiving the platform sound file sent by the mobile platform, the terminal device decodes the platform sound file to obtain platform audio information, and plays the platform audio information.

In the second aspect, this specification provides a control method for a terminal device, the terminal device is used to communicate with a movable platform, and both the terminal device and the movable platform are provided with audio sensors and speakers;

The method includes:

Acquiring a terminal sound file according to the environmental sound of the terminal device;

Sending the terminal sound file to the mobile platform, so that the mobile platform decodes the terminal sound file and plays it;

Acquiring a platform sound file sent by the mobile platform, the platform sound file being generated by the mobile platform after collecting the environmental sound of the mobile platform;

Decoding the platform sound file to obtain platform audio information, and playing the platform audio information.

In the third aspect, this specification provides a control method, which is applied to a movable platform, the movable platform is used to communicate with a terminal device, and both the terminal device and the movable platform are provided with audio sensors and speakers ；

The method includes:

Acquire the terminal sound file sent by the terminal device, decode the terminal sound file to generate terminal audio information, and play the terminal audio information; wherein the terminal sound file is collected by the terminal device from the environment of the terminal device Generated after the sound;

Generate a platform sound file according to the environmental sound of the movable platform;

Send the platform sound file to the terminal device, so that the terminal device decodes the platform sound file and plays it.

In the fourth aspect, this specification provides a terminal device, including an audio sensor, a speaker, a memory, and a processor;

The audio sensor is used to collect environmental sounds of the terminal device, and the speaker is used to play audio information;

The memory is used to store a computer program;

The processor is configured to execute the computer program and, when executing the computer program, implement the following steps:

In the fifth aspect, this specification provides a movable platform, including an audio sensor, a speaker, a memory, and a processor;

The audio sensor is used to collect environmental sounds of the movable platform, and the speaker is used to play audio information;

The memory is used to store a computer program;

In a sixth aspect, this specification provides a computer-readable storage medium that stores a computer program, and when the computer program is executed by a processor, the processor implements the above-mentioned control method.

The embodiments of this specification provide a movable platform, a terminal device and a control method thereof, and a storage medium. The terminal device collects audio data of its surrounding environment and sends the audio data to the corresponding movable platform, so that the user can A sound is made through the movable platform at a place far away from the movable platform, for example, a person near the movable platform or other movable platforms are called. In addition, collecting the audio data of the surrounding environment through the mobile platform and sending the audio data to the terminal device of the mobile platform can enable the user of the terminal device to listen to the mobile platform even if it is not in the vicinity of the mobile platform. The sound scene of the environment where the platform is located can facilitate the user to interact with the surrounding environment more conveniently and intuitively, facilitate the user's control of the movable platform, and meet the user's purpose of transmitting voice information.

It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the disclosure of this specification.

Description of the drawings

In order to explain the technical solutions of the embodiments of this specification more clearly, the following will briefly introduce the drawings used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of this specification. Ordinary technicians can obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic flowchart of a control method provided by an embodiment of this specification;

FIG. 2 is a schematic diagram of an embodiment of a system composed of a terminal device and a movable platform;

FIG. 3 is a schematic diagram of another embodiment of a system composed of a terminal device and a movable platform;

4 is a schematic flowchart of a control method provided by another embodiment of this specification;

Fig. 5 is a schematic diagram of an embodiment of a platform control interface of a terminal device;

Figure 6 is a schematic diagram of the display mode of the intercom button in the platform control interface;

FIG. 7 is a schematic diagram of the processing state of the environmental sound displayed on the terminal device;

FIG. 8 is a schematic diagram of another implementation manner of a platform control interface of a terminal device;

FIG. 9 is a schematic diagram of an embodiment of a recording record list;

FIG. 10 is a schematic flowchart of a control method provided by another embodiment of this specification;

FIG. 11 is a schematic block diagram of a terminal device according to an embodiment of the present specification;

Fig. 12 is a schematic block diagram of a movable platform provided by an embodiment of the present specification.

Detailed ways

The technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by a person of ordinary skill in the art without creative work shall fall within the protection scope of this specification.

The flowchart shown in the drawings is only an example, and does not necessarily include all contents and operations/steps, nor does it have to be executed in the described order. For example, some operations/steps can also be decomposed, combined or partially combined, so the actual execution order may be changed according to actual conditions.

Hereinafter, some embodiments of this specification will be described in detail with reference to the accompanying drawings. In the case of no conflict, the following embodiments and features in the embodiments can be combined with each other.

Please refer to FIG. 1, which is a schematic flowchart of a control method provided by an embodiment of this specification. The control method can be applied to a system composed of a terminal device and a movable platform, and is used to implement processes such as voice intercom between the terminal device and the movable platform. Specifically, both the terminal device and the movable platform are provided with audio sensors and speakers.

Among them, terminal devices may include, for example, remote controls, mobile phones, tablet computers, notebook computers, desktop computers, personal digital assistants, and wearable devices, such as virtual reality (Virtual Reality, VR) glasses, FPV (First Person View, first-person view) At least one item of glasses, etc.; the movable platform may be, for example, a movable robot, a robotic vehicle, an unmanned aerial vehicle, etc., and a movable robot is used as an example for schematic illustration.

In some embodiments, as shown in FIG. 2, the mobile robot 11 and the terminal device 13 may communicate, and the communication method may be wired communication or wireless communication. This embodiment takes wireless communication as an example.

Exemplarily, the mobile platform and the terminal device may be directly connected in communication, or may be connected to each other through communication connections such as routers, servers, base stations, etc.

In some embodiments, as shown in Figure 2, the mobile robot includes:

The robot body 110 includes a chassis main body 111 and a pan/tilt main body 112 provided on the chassis main body 111. The pan/tilt main body 112 is used to carry the camera 101;

The power device 120 is arranged on the chassis body 111 and is used to provide moving power to the robot body 110;

The audio sensor and the speaker are arranged on the robot body 110, the audio sensor is used to collect environmental sound, and the speaker is used to play audio;

The communication device is provided on the robot body 110 and is used to communicate with the terminal device.

In some embodiments, a launching device is provided on the mobile robot, and the launching device can be used to launch projectiles, and the size and shape of the projectiles are not specifically limited.

Optionally, as shown in FIG. 3, multiple mobile robots, such as the mobile robot 11 and the mobile robot 12, launch projectiles or light beams through their respective launching devices to compete. In addition, each mobile robot may also correspond to one terminal device, or multiple mobile robots correspond to one terminal device. For example, the mobile robot 11 corresponds to the terminal device 13 and the mobile robot 12 corresponds to the terminal device 14.

As shown in FIG. 1, the control method applied to a system composed of a terminal device and a movable platform may include step S110 to step S140.

S110: The terminal device obtains a terminal sound file according to the environmental sound of the terminal device, and sends the terminal sound file to the movable platform.

S120. The mobile platform receives the terminal sound file sent by the terminal device, decodes the terminal sound file, and plays it.

S130. The movable platform generates a platform sound file according to the environmental sound of the movable platform.

S140: After receiving the platform sound file sent by the movable platform, the terminal device decodes the platform sound file to obtain platform audio information, and plays the platform audio information.

In a possible manner, as shown in Figures 2 and 3, the mobile robot can be provided with an audio sensor, which can be used to collect audio data in the surrounding environment of the mobile robot. For example, the audio sensor can be a microphone. After the mobile robot collects audio data in the surrounding environment through a microphone, the audio data can be sent to the terminal device that communicates with the mobile robot.

For example, as shown in FIG. 3, the mobile robot 11 is in communication connection with the terminal device 13, and the mobile robot 11 collects audio data in the surrounding environment through a microphone, and then sends the audio data to the terminal device 13. Optionally, the audio data in the surrounding environment collected by the mobile robot 11 through a microphone may be derived from other mobile robots, for example, the mobile robot 12. Alternatively, the audio data in the surrounding environment collected by the microphone on the mobile robot 11 may come from users of other mobile robot terminal devices. For example, the mobile robot 12 is in communication connection with the terminal device 14, and the mobile robot 11 The audio data collected by the microphone is the audio data of the terminal device 14 or the user who controls the terminal device 14. In other words, the mobile robot 11 may send the collected audio data emitted by the mobile robot 12 and/or the audio data of the terminal device 14 corresponding to the mobile robot 12 or the user of the terminal device 14 to the terminal device 13.

Similarly, the mobile robot 12 may also send the collected audio data from the mobile robot 11 and/or the terminal device 13 corresponding to the mobile robot 11 or the user's audio data of the terminal device 13 to the terminal device 14.

In another possible manner, as shown in FIG. 1, the terminal device may be provided with an audio sensor, and the audio sensor may be used to collect audio data in the surrounding environment of the terminal device, for example, audio data of a user of the terminal device. Further, the terminal device sends the user's audio data to the mobile robot that communicates with the terminal device.

For example, the mobile robot 11 is in communication connection with the terminal device 13, and the terminal device 13 is provided with a microphone. The microphone can be used to collect the audio data of the user of the terminal device 13. Further, the terminal device 13 sends the user's audio data to the user Mobile robot 11. Similarly, the terminal device 14 may also send the audio data of the user of the terminal device 14 to the mobile robot 12.

In another possible manner, as shown in Fig. 2, both the terminal device and the mobile robot are provided with audio sensors. For example, the mobile robot 11 is in communication connection with the terminal device 13, and the mobile robot 11 and the terminal device 13 are respectively provided with microphones, and the mobile robot 11 collects the audio data around the mobile robot 11 in real time through the microphone on the mobile robot 11, and The audio data is sent to the terminal device 13, and at the same time, the terminal device 13 collects the audio data of the user of the terminal device 13 in real time through the microphone on the terminal device 13 and sends the audio data to the mobile robot 11.

For another example, as shown in FIG. 3, the mobile robot 11 is in communication connection with the terminal device 13, and the mobile robot 12 is in communication connection with the terminal device 14. The mobile robot 11, the mobile robot 12, the terminal device 13 and the terminal device 14 are respectively A microphone is provided, and the terminal device 13 collects the audio data of the user of the terminal device 13 through the microphone on the terminal device 13, and sends the audio data to the mobile robot 11. The mobile robot 11 may also be provided with a speaker. The speaker can be used to play audio data of the user of the terminal device 13. At the same time, the microphone on the mobile robot 12 collects the audio data of the user of the terminal device 13 and moves to the position of the terminal device 14 to play the audio data of the user of the terminal device 13 to the user of the terminal device 14. In the same way, the mobile robot 12 can also receive the audio data of the user of the terminal device 14, and play the audio data of the user of the terminal device 14 through the speaker on the mobile robot 12. At the same time, the mobile robot 11 collects the terminal device 14 14 of the user’s audio data, and sent to the terminal device 13.

The control method provided by the above-mentioned embodiments of this specification collects the audio data of its surrounding environment through the terminal device and sends the audio data to the corresponding movable platform, so that the user can pass through the movable platform far away from the movable platform. The platform emits sounds, for example, shouting to people near the movable platform or other movable platforms. In addition, collecting the audio data of the surrounding environment through the mobile platform and sending the audio data to the terminal device of the mobile platform can enable the user of the terminal device to listen to the mobile platform even if it is not in the vicinity of the mobile platform. The sound scene of the environment where the platform is located. For example, even if the user of the terminal device is not on the mobile platform, they can still feel the atmosphere of the battle scene, and the user of the terminal device of the mobile platform can send it back according to the mobile platform. The audio accurately controls the movable platform. It is convenient for users to interact with the surrounding environment more conveniently and intuitively, which is conducive to the user's control of the movable platform and meets the user's purpose of transmitting voice information. In the networked competition of multiple mobile robots, the real-time audio-visual effects are increased, and the voice interaction fun of the players is improved.

Please refer to FIG. 4 in conjunction with the foregoing embodiment. FIG. 4 is a schematic flowchart of a control method provided by an embodiment of this specification. The control method can be applied to a terminal device that is used to communicate with a movable platform, and both the terminal device and the movable platform are provided with audio sensors and speakers.

As shown in FIG. 4, the control method of the embodiment of this specification includes step S210 to step S240.

S210: Acquire a terminal sound file according to the environmental sound of the terminal device.

In some embodiments, the terminal device collects sound in the environment through an audio sensor, and encodes the collected sound data to obtain a terminal sound file. Reduce the amount of data sent by encoding, and improve the real-time performance of sound file sending.

Exemplarily, the terminal device encodes the currently collected environmental sound in real time, and generates a terminal sound file in the corresponding encoding format after the recording ends. This improves the real-time performance of the terminal sound file sending to the mobile platform.

The terminal device encodes the currently collected sound data into OPUS data in real time, and generates a terminal sound file in OPUS format after the recording is finished.

In some embodiments, the control method further includes: displaying a platform control interface, and the platform control interface includes an intercom button.

In some other embodiments, the intercom button may also be a physical button set on a terminal device, such as a remote control, VR glasses, or FPV glasses.

In some embodiments, the camera device mounted on the movable platform can transmit the collected images to the terminal device, and the terminal device can display the image taken by the movable platform on the platform control interface as shown in FIG. 5, so that the user can understand the movable platform. The environment.

As shown in Figure 5, an intercom button named megaphone is displayed on the upper right of the terminal equipment platform control interface, and the environmental sound of the terminal device can be controlled and collected through the intercom button.

Exemplarily, according to the setting operation of the intercom button by the user, the intercom control operation of the intercom button by the user can be enabled or disabled, and the display mode of the intercom button can be adjusted.

For example, as shown in Figure 6, display mode A is the default display mode of the intercom button. In this display mode, the user can press and hold the intercom button to trigger the terminal device to collect ambient sound. The button can be displayed in display mode C. When the user clicks the intercom button in display mode A, the intercom button can be displayed in display mode B to prompt the user; after the user clicks the intercom button and releases it, adjust the display mode of the intercom button to display mode D, And close the user's intercom control operation on the intercom button; in this display mode D, even if the user presses the intercom button for a long time, the terminal device will not be triggered to collect environmental sounds. The user can also click on the intercom button of display mode C to trigger the terminal device to display the intercom button in display mode A, so as to enable the user to control the intercom operation of the intercom button, such as allowing triggering by long pressing the intercom button The terminal equipment collects environmental sounds.

It is understandable that the obtaining the terminal sound file according to the environmental sound of the terminal device includes: obtaining the environmental sound of the terminal device according to the intercom control operation of the intercom button by the user, and encoding the terminal sound file.

Specifically, when the user's intercom control operation on the intercom button is enabled, if it is detected that the duration of the user's pressing of the intercom button exceeds a preset threshold, such as 0.5 seconds, then start to obtain the environment of the terminal device The sound is encoded to obtain the terminal sound file.

Exemplarily, the environmental sound of the terminal device may be acquired during the time period after the intercom button is pressed to before the intercom button is released, and the terminal sound file may be obtained by encoding.

It is understandable that the terminal device can collect the environmental sound when the intercom button is pressed, and the user releases the intercom button to make the terminal device end the collection of the environmental sound.

Exemplarily, if the duration of the intercom button being pressed reaches a preset duration, for example, 60 seconds, the acquisition of the ambient sound of the terminal device may be stopped, and the acquired sound may be encoded to obtain the terminal sound file. It can limit the data volume of the terminal sound file, and prevent the transmission to the mobile platform from occupying too much time and affecting other operations, such as controlling the movement of the mobile robot.

For example, if the time for the user to press the intercom button reaches 30 seconds, for example, the terminal device ends collecting environmental sounds.

In some embodiments, the user can cancel the collection, encoding, and transmission of ambient sound by operating the intercom button. For example, when the user long presses the intercom button to record, if the terminal device detects the user's drag operation of the intercom button, for example, when the intercom button is pressed, the finger touches the position and slides away from the intercom button, then After collecting environmental sounds, you can also clear the encoded environmental sounds.

Exemplarily, as shown in FIG. 5, when acquiring the environmental sound of the terminal device, the terminal device may display the sound spectrogram of the environmental sound on the platform control interface. So that the user can intuitively understand the current recording status.

Specifically, when it is detected that the duration of the user pressing the intercom button exceeds a preset threshold, such as 0.5 seconds, the sound spectrogram of the environmental sound may be displayed on the platform control interface, and the sound may not be displayed when the environmental sound is not collected. The area of the spectrogram, so that more information can be displayed on the platform control interface.

S220. Send the terminal sound file to the mobile platform, so that the mobile platform decodes the terminal sound file and plays it.

Specifically, as shown in FIG. 2, the user can control the mobile robot 11 to move in front of other people or animals through the terminal device 13, and the mobile robot 11 can send the terminal sound file sent by the terminal device 13 after receiving and decoding the terminal sound file. The mobile robot 11 plays to other people or animals, and realizes the function of transmitting sounds to other people or animals.

Specifically, as shown in FIG. 3, the mobile robot 11 is located within the sound transmission range of the mobile robot 12. After the mobile robot 11 receives and decodes the terminal sound file sent by the terminal device 13, the mobile robot 11 can play it to the mobile robot 12. Realize the function of transmitting sound to other mobile robots. Exemplarily, the mobile robot 12 may collect and send the sound played by the mobile robot 11, and then send the collected sound to the terminal device 14 communicatively connected with the mobile robot 12, and the terminal device 14 broadcasts the sound to the terminal device 14 user.

In some embodiments, the control method further includes: displaying the processing status of the environmental sound on the platform control interface, the processing status including at least one of silent recording, recording, transmission, and transmission completed . It can prompt the user of the processing status of the environment sound.

Exemplarily, as shown in FIG. 5 and FIG. 7, an image or animation of the processing state may be displayed in a certain area of the platform control interface.

As shown in Fig. 5, the terminal device displays the sound spectrogram of the environmental sound on the platform control interface, prompting the user that the processing state of the environmental sound is recording.

As shown in Figure 7, from top to bottom, the processing status of the environmental sound is not detected, the processing status of the environmental sound being collected, the processing status of the terminal sound file of the environmental sound being sent to the mobile platform, and the terminal sound file The processing status of the sending.

In some embodiments, as shown in FIG. 5, the platform control interface further includes a platform control button. The platform control buttons may include, for example, a launch button, a camera button, a video button, a pedestrian follow button, a custom skill button, and the like.

The control method may further include: generating and sending a corresponding platform control instruction to the movable platform according to a button trigger operation of the platform control button by the user, so that the movable platform executes according to the platform control instruction Preset tasks.

Exemplarily, a launching device is provided on the mobile robot, and the launching device can be used to launch projectiles. If the terminal device detects that the user triggers an operation on the launch button, it sends a launch instruction to the movable platform, and the movable robot can launch the projectile according to the launch instruction.

Exemplarily, the user can operate the camera button and the video button to enable the terminal device to load the mobile platform with buttons and video instructions, so that the mobile platform can perform tasks such as photography and video recording through the mounted camera device.

Exemplarily, when the terminal device detects that the user triggers the operation of the pedestrian follow button, it sends a pedestrian follow instruction to the movable platform, and the movable platform follows the pedestrian target in the captured image according to the pedestrian follow instruction, and then shoots The image is sent to the terminal device.

Exemplarily, the user can define tasks of the movable platform corresponding to the custom skill button, such as drift. When the terminal device detects that the user triggers an operation on the button of the custom skill button, it sends a drift instruction to the movable platform, and the movable platform implements the drift task according to the drift instruction.

Exemplarily, the custom skill button can also be defined as a stun skill. The tasks of the corresponding movable platform include releasing the skill to a certain movable platform and hitting the movable platform, and can control the hitting movable platform in the original The ground rotates and lasts for 1.5 seconds.

Exemplarily, the custom skill button can also be defined as a blinding skill, the task of the corresponding movable platform is included within a preset time threshold, and the display interface of the remote control terminal corresponding to the movable platform being hit is adjusted to The animation effect corresponding to the blinding skill. Animation effects, such as blurred, black or snowflake screens, block the image transmission screen, making it impossible for users to view the image transmission screen normally. In practical applications, the preset time threshold may specifically be 1.5 seconds.

Exemplarily, the custom skill button can also be defined as an electromagnetic interference skill, which can be launched by an infrared transmitter, and the image transmission of the movable platform that is hit is interfered for 2.5 seconds, and it can also be expressed as the FPV interface displayed as a flower screen effect.

Exemplarily, the custom skill button can also be defined as a speed skill, and the tasks of the corresponding movable platform include obtaining a faster moving speed and lasting for 3 seconds.

Exemplarily, the custom skill button can also be defined as an invincible skill. The tasks of the corresponding movable platform include automatically canceling the skill effect released by the opponent, and obtaining a shield for 3 seconds so that the opponent cannot cause damage to it.

Exemplarily, the platform control button may include a focusing button, such as the button at the bottom right of the interface in FIG. 5, which indicates that the focal length of the camera device of the current movable platform is 4 times. The user can switch the focal length of the camera device of the movable platform to, for example, 1x, 2x, etc. by operating the focus button.

Exemplarily, as shown in FIG. 5, the platform control button may include a sound return button. When the terminal device detects that the user triggers an operation on the sound return button, it sends a sound return instruction to the movable platform, which can move The platform obtains the sound around the movable platform according to the sound return instruction and transmits the collected sound back to the terminal device. The terminal device can, for example, play the sound around the movable platform.

In some embodiments, the control method may further include: acquiring a user's control voice, generating and sending a corresponding platform control instruction to the movable platform according to the control voice, so that the movable platform can be controlled according to the The platform control commands execute preset tasks.

Exemplarily, the terminal device stores the mapping relationship data between the control voice and the platform control instruction, so that the user can send the corresponding platform control instruction to the movable platform through the voice control terminal device, so that the movable platform can perform the preset task. For example, if the terminal device detects the user's "launch" control instruction, it sends a launch instruction to the movable platform, and the movable platform can launch projectiles according to the launch instruction.

Exemplarily, the obtaining the control voice of the user may include: obtaining the environmental sound of the terminal device, and detecting the control voice in the environmental sound.

Specifically, the terminal device continuously monitors the sound of the environment in which it is located, and detects whether there is a control voice in the environment sound. Users can quickly input control commands by voice.

Exemplarily, the obtaining the control voice of the user may include: obtaining the control voice uttered by the user when the user triggers the voice control function.

Exemplarily, the display interface of the terminal device also displays a voice control button, and the user can make a control voice when pressing the voice control button. It can prevent the wrong detection of the control voice when the user does not need the voice control.

S230. Obtain a platform sound file sent by the movable platform, where the platform sound file is generated by the movable platform after collecting the environmental sound of the movable platform.

S240. Decode the platform sound file to obtain platform audio information, and play the platform audio information.

In some embodiments, the mobile platform can collect the environmental sound of the mobile platform independently or according to the control of the terminal device, generate the platform sound file and send it to the terminal device, so that the terminal device can play the environmental sound of the mobile platform, which is convenient for The user understands the environment in which the mobile platform is located. For example, the user of the terminal device can feel the atmosphere of the battle scene even if the user of the terminal device is not on the battle scene of the mobile robot; it can also make the user of the terminal device of the mobile robot accurately control the mobile robot according to the audio data. robot.

Exemplarily, as shown in Figure 5, the terminal device is provided with a sound return button. When the terminal device detects that the user triggers an operation on the sound return button, it sends a sound return instruction to the movable platform, which can move The platform collects the environmental sound of the movable platform according to the sound return instruction, generates the platform sound file and sends it to the terminal device. Therefore, the user can control to listen to or not listen to the ambient sound of the mobile platform through the terminal device.

In some embodiments, the platform sound file is generated by the movable platform according to the voice of a person near the movable platform.

Exemplarily, as shown in FIG. 2, the user can control the mobile robot 11 to move in front of other people or animals through the terminal device 13, then the mobile robot 11 can collect the messages sent by the other people or animals that the mobile robot 11 faces. The sound can be sent to the terminal device 13 after the platform sound file is generated, and the terminal device will be played to other people or animals that the mobile robot 11 faces.

In some embodiments, the platform sound file is generated by the movable platform according to a sound originating from at least another movable platform.

Exemplarily, another movable platform can make a sound such as a roaring sound autonomously or according to the control of a corresponding terminal device, and the movable platform can generate a platform sound file according to the sound made by the another movable platform.

In some embodiments, the platform sound file is generated by the mobile platform according to a user's voice from a terminal device of at least another mobile platform.

Specifically, as shown in FIG. 3, the mobile robot 11 is located within the sound transmission range of the mobile robot 12, and the mobile robot 11 can play to the mobile robot 12 after receiving and decoding the terminal sound file sent by the terminal device 13; The mobile robot 12 can collect and send the sound played by the mobile robot 11, and then send the collected sound to the terminal device 14 communicatively connected with the mobile robot 12, and the terminal device 14 will play the sound to the user of the terminal device 14.

In some embodiments, the control method may further include: the terminal device determines the playback target of the terminal sound file according to the user's object setting operation.

Exemplarily, the sending the terminal sound file to the mobile platform so that the mobile platform decodes the terminal sound file and then plays it includes: sending the terminal sound file and the playback target The information is sent to the movable platform, so that the movable platform plays the terminal sound file when the playback object is recognized. In this way, the user can control the movable platform to play sound to the specified object.

Exemplarily, the user can specify the playback object of the terminal sound file on the terminal device. The playback object includes, for example, other movable platforms with designated marks, such as a QR code pattern or a team pattern, people or animals with designated facial features, etc. .

For example, when the terminal sound file is sent to the movable platform in step S220, the designated mark and the like are also sent to the movable platform. The movable platform can detect whether there is a playback object in the captured image, and if there is a playback object, it will play the corresponding terminal sound file to the playback object.

Exemplarily, the terminal device may obtain an image taken by the movable platform from the movable platform, display the image, and then determine the playback object according to a user's selection operation of the playback object in the image.

For example, the mobile platform sends the captured image to the terminal device for display, and the user can perform a selection operation in the image. For example, by clicking or box selecting a certain area in the image, the terminal device and the mobile platform can determine the playback object With some information, the movable platform can detect whether there is a playback object in the field of view of the camera device according to the selected area.

Exemplarily, the terminal device may display the local image selected by the user according to the user's selection operation on the local image of the terminal device, and then determine the playback object according to the user's determining operation on the playback object in the local image.

For example, if the terminal device locally stores the team pattern of its own team or an image containing the team pattern, the team pattern can be selected so that the terminal device determines the pattern that the playback object needs to have; and the team pattern is sent to the mobile platform. The mobile platform can detect whether there is a playback object in the field of view of the camera device based on the team pattern.

In some embodiments, the control method may further include: displaying a recording record list, and acquiring a terminal sound file corresponding to the recording record according to a user's playback control operation of the recording record in the recording record list. Then, the terminal sound file can be sent to the mobile platform, so that the mobile platform decodes the terminal sound file and plays it. In this way, users can send terminal sound files to the mobile platform more quickly.

As shown in Figure 8, the terminal device can display a list of recording records on the platform control interface. For example, the user can control the terminal device to display the recording record list by clicking the corresponding button on the platform control interface. Or the terminal device may also display the recording record list on other interfaces, which is not limited in this embodiment. Exemplarily, the corresponding close button is displayed in the upper right corner of the recording record list. When the user clicks the close button, the recording record list is closed.

Specifically, the recording record list may include one or more recording records, and each recording record corresponds to a corresponding terminal sound file. For example, the terminal device stores a corresponding terminal sound file corresponding to each recording record in the recording record list.

Exemplarily, if there are many recording records in the recording record list, several of them can be displayed, and the user can scroll up and down to view or select the remaining recording records.

As shown in Figure 8, the recording duration can be displayed at the corresponding position of each recording record, such as 10", etc., and the playback button can also be displayed on the left side of the recording duration. If the terminal device detects that the user has selected the playback button corresponding to a certain recording record The playback control operation is to obtain the terminal sound file corresponding to the recording record.

Exemplarily, as shown in FIG. 9, the length of the icon of each recording record in the recording record list can be adjusted according to the recording duration, for example, the longer the recording duration, the longer the icon.

Exemplarily, the playback status and/or playback progress of the recording record may also be displayed in the recording record list. As shown in Figure 9, the playback status and/or playback progress of the first recording can be displayed in the form of a progress bar.

Exemplarily, the user can add a new recording record to the recording record list. Specifically, the terminal device may obtain and process the newly recorded sound according to the user's newly added recording operation to obtain a new terminal sound file, and update the corresponding recording record in the recording record list.

For example, as shown in FIG. 8, the user can press the "press and hold recording" button to trigger the terminal device to acquire and process the newly recorded sound to obtain a new terminal sound file, and update the corresponding recording record in the recording record list.

Exemplarily, the control method further includes: acquiring a terminal sound file corresponding to the recording record according to a user's loop playback operation of the recording record in the recording record list.

As shown in Figure 8 and Figure 9, the loop button can be displayed on the right side of each recording. If the terminal device detects the user's playback control operation on the loop button corresponding to a certain recording record, it acquires the terminal sound file corresponding to the recording record. Then, the terminal sound file and the loop instruction may be sent to the movable platform, so that the movable platform decodes the terminal sound file and plays it in a loop. Therefore, the user can control the movable platform to repeatedly play the specified recording through the loop playback operation.

In some other implementation manners, the control method may further include: displaying a list of recording records; sending the information of the recording records to the mobile platform according to a user's playback control operation on the recording records in the recording record list, So that the mobile platform can play the terminal audio information corresponding to the recording record.

Exemplarily, the mobile platform may pre-store a terminal sound file corresponding to each recording record in the recording record list, or terminal audio information obtained by decoding the terminal sound file. Therefore, the mobile platform can determine the terminal sound file or terminal audio information to be played according to the information recorded in the recording selected by the user on the terminal device.

Exemplarily, the control method may further include: according to a user's loop playback operation of the recording records in the recording record list, sending the recording record information and loop instructions to the movable platform, so that the The mobile platform cyclically plays the terminal audio information corresponding to the recording record. Therefore, the user can control the movable platform to repeatedly play the specified recording through the loop playback operation.

Exemplarily, the control method may further include: acquiring and processing the newly recorded sound to obtain a new terminal sound file according to the new recording operation of the user, and updating the corresponding recording record in the recording record list; The terminal sound file is sent to the mobile platform, so that the mobile platform stores terminal audio information obtained by decoding the terminal sound file.

For example, the terminal device may immediately send the terminal sound file to the mobile platform after obtaining the new terminal sound file.

For example, the terminal device may send the terminal sound file to the mobile platform according to the user's playback control operation of the recording record corresponding to the new terminal sound file. For example, when the user performs a playback control operation or a loop playback operation on a new recording record for the first time, the terminal device sends the terminal sound file corresponding to the new recording record to the mobile platform.

Therefore, the user can edit the recording records in the recording record list, and can also send the terminal sound file corresponding to the newly added recording record to the mobile platform so that the mobile platform can store the terminal sound file or terminal audio information corresponding to the newly added recording record.

Please refer to FIG. 10 in conjunction with the foregoing embodiment. FIG. 10 is a schematic flowchart of a control method provided by an embodiment of this specification. The control method can be applied to a movable platform, the movable platform is used to communicate with a terminal device, and both the terminal device and the movable platform are provided with audio sensors and speakers.

As shown in FIG. 10, the control method of the embodiment of this specification includes step S310 to step S330.

S310. Obtain a terminal sound file sent by the terminal device, and decode the terminal sound file to generate terminal audio information, and play the terminal audio information.

Exemplarily, the terminal sound file is generated by the terminal device after collecting the environmental sound of the terminal device. For example, the terminal device collects the sound in the environment through the audio sensor, encodes the collected sound data to obtain the terminal sound file, and then sends the terminal sound file to the mobile platform.

Exemplarily, the terminal device may display a list of recording records, and then obtain the terminal sound file corresponding to the recording record according to the user's playback control operation on the recording record in the recording record list, so as to send the terminal sound file to the mobile platform .

In some embodiments, the control method further includes: if acquiring the information of the recording record sent by the terminal device, determining the terminal audio information corresponding to the recording record, and playing the terminal audio information corresponding to the recording record.

Exemplarily, the mobile platform may pre-store the terminal sound file sent by the terminal device or the terminal audio information generated by decoding the terminal sound file. After receiving the playback instruction or the loop instruction sent by the terminal device according to the user's control operation of the recording record in the recording record list, the terminal audio information corresponding to the recording record is played.

In some embodiments, the movable platform is equipped with a camera device, and the control method further includes: if the information of the playback object sent by the terminal device is acquired, identifying in the image taken by the camera device according to the information The play object; if the play object is recognized, the corresponding terminal audio information is played to the play object.

Exemplarily, the movable platform sends the image taken by the camera device to the terminal device, so that the terminal device determines the playback object according to the user's selection operation of the playback object in the image.

Exemplarily, the playing the terminal audio information in step S310 includes: if a loop instruction corresponding to the terminal audio information is obtained, playing the terminal audio information in a loop.

For example, if the terminal device detects that the user performs a playback control operation on a loop button corresponding to a certain recording record, it acquires the terminal sound file corresponding to the recording record. Then, the terminal sound file and the loop instruction may be sent to the movable platform, so that the movable platform decodes the terminal sound file and plays it in a loop. Therefore, the user can control the movable platform to repeatedly play the specified recording through the loop playback operation.

For example, the terminal device sends the information of the recording record and the loop instruction to the mobile platform according to the user's circular playback operation of the recording record in the recording record list, so that the mobile platform can play the recording record in a loop Corresponding terminal audio information.

In some implementation manners, the control method further includes: sending the playback status and/or playback progress of the terminal audio information to the terminal device.

Exemplarily, the terminal device may display the playback status and/or playback progress of the terminal audio information on the interface, for example, display the playback status and/or playback progress of the corresponding recording record in the recording record list.

In some embodiments, the control method further includes: when playing the terminal audio information, adjusting the display parameters of the display device of the movable platform according to the terminal audio information.

For example, the movable platform, such as the movable robot, includes a display device, and the display device may include, for example, a ring-shaped light bar.

Exemplarily, the mobile platform may adjust the display brightness of the display device according to the sound intensity of the terminal audio information; and/or adjust the flicker frequency of the display device according to the sound frequency of the terminal audio information. For example, when the sound intensity of the terminal audio information becomes stronger at a certain moment, the display brightness of the display device is brightened; the higher the sound frequency of the terminal audio information in a certain period of time, the higher the flicker frequency of the display device. A display device, such as a light bar, intuitively reminds the user that the mobile platform is playing terminal audio information.

S320: Generate a platform sound file according to the environmental sound of the movable platform.

S330. Send the platform sound file to the terminal device, so that the terminal device decodes the platform sound file and plays it.

In some embodiments, the movable platform acquires the environmental sound of the movable platform when the terminal audio information is not played. For example, when the mobile platform is playing the terminal audio information, it may not first acquire the environmental sound of the mobile platform, so as to avoid the interference of transmitting the played terminal audio information back to the terminal device for playback.

In some embodiments, the mobile platform obtains the environmental sound of the mobile platform, and filters the played terminal audio information from the environmental sound, so as to avoid transmitting the played terminal audio information back to the terminal device. Interference caused by playback.

In some embodiments, the movable platform may generate a platform sound file based on the sound of at least another movable platform and/or the sound of a user of at least another terminal device of the movable platform.

Exemplarily, as shown in FIG. 3, the mobile robot 11 is located within the sound transmission range of the mobile robot 12, and the mobile robot 11 can play to the mobile robot 12 after receiving and decoding the terminal sound file sent by the terminal device 13. The mobile robot 12 can collect and send the sound played by the mobile robot 11, and then send the collected sound to the terminal device 14 communicatively connected with the mobile robot 12, which is played by the terminal device 14 to the user of the terminal device 14.

In some embodiments, the control method further includes: if the platform control instruction sent by the terminal device is acquired, execute a preset task according to the platform control instruction; wherein, the platform control instruction is the terminal device according to the user It is sent by the button trigger operation of the platform control button of the terminal device, or sent by the terminal device according to the control voice of the user.

Exemplarily, the terminal device stores the mapping relationship data between the control voice and the platform control instruction, so that the user can send the corresponding platform control instruction to the movable platform through the voice control terminal device, so that the movable platform can perform the preset task. For example, if the terminal device detects the user's "launch" control instruction, it sends a launch instruction to the movable platform, and the movable platform can launch the projectile according to the launch instruction.

The control method provided by the above-mentioned embodiments of this specification collects audio data of its surrounding environment through a terminal device and sends the audio data to the corresponding movable platform, so that the user can pass through the movable platform far away from the movable platform. The platform emits sounds, for example, shouting to people near the movable platform or other movable platforms. In addition, collecting the audio data of the surrounding environment through the mobile platform and sending the audio data to the terminal device of the mobile platform can enable the user of the terminal device to listen to the mobile platform even if it is not in the vicinity of the mobile platform. The sound scene of the environment where the platform is located can facilitate the user to interact with the surrounding environment more conveniently and intuitively, facilitate the user's control of the movable platform, and meet the user's purpose of transmitting voice information.

Acquiring environmental sound in the above embodiments can be realized by collecting sound signals by a sound pickup device, and one of the collecting methods is as follows:

Acquire the first digital signal generated by the sound signal collected by the sound pickup device; acquire the second digital signal corresponding to the sound signal generated by the standard source; determine according to the conversion relationship between the first digital signal and the second digital signal The signal calibration parameters of the pickup device.

In the embodiment of the present invention, the sound signal collected by the sound pickup device is a sound signal generated by a standard source. In the process of generating sound signals, the standard sound source can play the sound signal generated by the standard sound source through its configured sound playback device, so as to transmit the sound signal generated by the standard sound source to the pickup device through the transmission medium in the space. Correspondingly, the sound pickup device collects the sound signal generated by the standard sound source, and converts the collected sound signal into an electrical signal. Further, the sound pickup device transmits the converted electric signal to the signal processing device of the sound pickup device. The signal processing device may include an analog-to-digital conversion device. The analog-to-digital conversion device converts the electric signal transmitted by the sound pickup device into a first digital signal. signal. In one embodiment, in order to reduce the energy loss in the sound signal transmission process, the standard sound source and the sound pickup device can be placed in the same closed cavity; the distance between the standard sound source and the sound pickup device can also be limited to a certain Within the distance. The certain distance range is, for example, [30cm, 1.5m]. In another embodiment, in order to avoid the interference of external sound signals, sound insulation materials can be used to construct the sealed cavity. The external sound signal refers to the sound signal produced by other sound sources other than the standard sound source.

In the embodiment of the present invention, the second digital signal may be pre-stored in a storage medium, and the signal processing device of the sound pickup device directly obtains the second digital signal from the storage medium. The second digital signal may also be pre-stored in other smart terminals or servers, and the signal processing device of the sound pickup device obtains the second digital signal from other smart terminals or servers after establishing a communication connection with other smart terminals or servers.

In an embodiment, the sound signal generated by the standard sound source includes one or more groups of sound signals, and each group of sound signals corresponds to one or more sound intensity values. When there are multiple sound intensity values corresponding to a group of sound signals generated by the standard sound source, the multiple sound intensity values may be linearly distributed.

In the embodiment of the present invention, the signal processing device of the sound pickup device first obtains the first sampling point set corresponding to the first digital signal, and obtains the second sampling point set corresponding to the second digital signal; and then according to the first sampling point set and the first sampling point set A set of two sampling points to determine the conversion relationship between the first digital signal and the second digital signal.

In an embodiment, the signal processing device of the sound pickup device determines the conversion relationship between the first digital signal and the second digital signal according to the first sampling point set and the second sampling point set: first according to the first set of sampling points and the second set of sampling points. The sampling points in the sampling point set determine the first fitting curve; and the second fitting curve is determined according to the sampling points in the second sampling point set. Specifically, the signal processing device of the sound pickup device uses a polynomial fitting method to fit the sampling points in the first sampling point set to obtain the first fitting function and the first fitting curve. Wherein, the first fitting function is a function expression corresponding to the first fitting curve; the first fitting curve is a curve with the best goodness of fit when fitting sampling points in the first sampling point set. In the same way, the polynomial fitting method can be used to fit the sampling points in the second sampling point set to obtain the second fitting function and the second fitting curve. Wherein, the second fitting function is a function expression corresponding to the second fitting curve; the second fitting curve is a curve with the best goodness of fit when fitting sampling points in the second sampling point set. It should be noted that the fitting method is not limited to polynomial fitting, and those skilled in the art can set the fitting method according to actual needs.

Further, the signal processing device of the sound pickup device obtains the first target conversion relationship between the first fitting curve and the second fitting curve, and determines the first target conversion relationship as the one between the first digital signal and the second digital signal The conversion relationship between. Wherein, the first target conversion relationship makes the first fitting curve approach or coincide with the second fitting curve. The first target conversion relationship may be determined according to the function conversion relationship between the first fitting function corresponding to the first fitting curve and the second fitting function corresponding to the second fitting curve. Further, the signal calibration parameters of the sound pickup device can be determined according to the function conversion relationship between the first fitting function and the second fitting function.

In the embodiment of the present invention, the signal processing device of the sound pickup device determines the signal calibration parameter of the sound pickup device according to the conversion relationship between the first digital signal and the second digital signal, and then saves the signal calibration parameter for the sound pickup device. After the device subsequently collects the sound signal, the digital signal corresponding to the sound signal collected by the sound pickup device is calibrated by using the signal calibration parameters to calibrate the sound signal collected by the sound pickup device. By adopting the above method, the calibration accuracy of the sound pickup device can be improved, so that different sound pickup devices have the same or similar output levels for the same sound signal.

The following processing may be performed after the environmental sound is acquired in the above embodiment, which may specifically include:

Step S401: Use multiple pre-processing circuits to process analog audio signals to be processed to obtain multiple digital audio signals, where each of the multiple pre-processing circuits includes an amplifier and an analog-to-digital converter, and each pre-processing circuit includes an amplifier and an analog-digital converter. The analog gains of the amplifiers of the processing circuit are different from each other;

Specifically, multiple pre-processing circuits are connected in parallel, and each pre-processing circuit includes an amplifier and an analog-to-digital converter. It should be noted that the multiple pre-processing circuits include at least two pre-processing circuits; wherein the amplifier can be used for analog The audio signal is power amplified, and the amplification factor is usually expressed by gain. In this embodiment, the analog gains of the amplifiers of the preprocessing circuits are different. Further, the dynamic range of the two preprocessing circuits with adjacent analog gains is at least partially Overlap; and analog-to-digital converters can be used to convert analog audio signals into digital audio signals to facilitate subsequent signal processing.

Assuming that the analog audio signal x to be processed can be collected by a microphone, and then input into multiple pre-processing circuits in parallel, respectively, through power amplification with different gains, and converted into digital audio signals x1,...xi...,xI, that is, each The input of a pre-processing circuit is the same analog audio signal to be processed, and the digital audio signal output by each pre-processing circuit is different, thereby obtaining multiple digital audio signals.

In this embodiment, the analog audio signal to be processed can be intercepted into different segments with a predetermined duration as a frame signal, or a predetermined number of sampled data can be used as a frame signal after analog-to-digital conversion. The subsequent audio signal processing procedures are all It can be processed in units of one frame of signal. In order to ensure signal continuity, there may be a certain overlap between adjacent frame signals, that is, the tail of the previous frame signal and the head of the next frame signal have an overlap amount, thereby establishing the correlation between adjacent frames.

Step S402: Perform frequency domain conversion on the multiple channels of digital audio signals to obtain multiple channels of frequency domain data.

In this embodiment, frequency domain conversion is performed on each channel of digital audio signal in the multiple channels of digital audio signal, so as to obtain frequency domain data corresponding to each channel of digital audio signal. In this embodiment, the multiple channels of digital audio signals are subjected to frequency domain conversion to obtain multiple channels of frequency domain data, and the fusion of multiple channels of frequency domain data in the frequency domain can be realized. The frequency domain conversion method can adopt Fourier transform (such as discrete Fourier transform), Laplace transform, Z transform, etc. The specific frequency domain conversion process will not be repeated here.

Step S403: Determine frequency domain fusion data according to one or at least two channels of target frequency domain data among the multiple channels of frequency domain data.

In this embodiment, since each pre-processing circuit amplifies the same analog audio signal to be processed with different gains, each channel of audio signal has a different maximum and minimum value, and the maximum value of the audio signal with a larger gain. And the minimum value are relatively large, and the maximum and minimum values of the audio signal with a smaller gain are relatively small, and the dynamic range is the ratio of the maximum and minimum values of the audio signal without distortion. In this embodiment Determine the frequency domain fusion data according to one or at least two channels of the target frequency domain data in the multi-channel frequency domain data, and the maximum and minimum values of the audio signal finally obtained after the fusion can be adjusted through the fusion. For example, the louder sound can be adjusted by A pre-processing circuit with a relatively large gain is provided, and a smaller sound can be provided by a pre-processing circuit with a relatively small gain, so that the dynamic system of the recording system can be improved, with higher sensitivity and lower noise at the same time. In this embodiment, the size of the sound can be measured by the energy feature information of the audio signal, such as the sound pressure level of the analog audio signal or digital audio signal, or the amplitude of the analog audio signal or digital audio signal, and so on.

Step S404: Convert the frequency domain fusion data into a time domain audio signal, and obtain an output audio signal according to the time domain audio signal.

In this embodiment, after the frequency domain fusion data is obtained, the frequency domain fusion data can be converted into a time domain audio signal. The conversion method can adopt the inverse transform of the Fourier transform (such as the discrete Fourier transform), the Lap The inverse transform of the Lass transform, the inverse transform of the Z transform, etc., will not be repeated here. After the conversion is completed, the output audio signal can be obtained according to the time domain audio signal. In the process of obtaining the output audio signal according to the time domain audio signal, operations such as compression and noise reduction can be performed on the time domain audio signal. In addition, if the audio signal processing procedure described above is performed in the unit of a frame signal in this embodiment, the process of obtaining the output audio signal according to the time domain audio signal also needs to be spliced between the frame signals to establish the correlation between adjacent frames. Specifically, the time domain audio signal of the current frame and the previous frame time domain audio signal of the previous frame can be superimposed.

The audio signal processing method of this embodiment uses multiple preprocessing circuits to process analog audio signals to be processed to obtain multiple digital audio signals, wherein each of the multiple preprocessing circuits includes an amplifier and an analog audio signal. Digital converter, and the analog gains of the amplifiers of each preprocessing circuit are different; frequency domain conversion is performed on the multiple channels of digital audio signals to obtain multiple channels of frequency domain data; according to the multiple channels of frequency domain data One or at least two channels of target frequency domain data determine frequency domain fusion data; convert the frequency domain fusion data into a time domain audio signal, and obtain an output audio signal according to the time domain audio signal. The method of this embodiment can effectively improve the dynamic range of the recording system, has high sensitivity, and can reduce the noise floor and meet the requirements of high signal-to-noise ratio.

On the basis of any of the foregoing embodiments, the audio signal processing method further includes:

Obtain the energy feature information of the analog audio signal to be processed.

In this embodiment, the energy characteristic information of the analog audio signal to be processed may be the sound pressure level of the analog audio signal or the amplitude of the analog audio signal, etc., and specifically may be the maximum, minimum, or instantaneous amplitude of the analog audio signal. The intermediate value can also be the maximum, minimum, or intermediate value of the average amplitude of the analog audio signal in a short time (preset duration).

Further, in step S403, determining frequency domain fusion data according to one or at least two channels of frequency domain data among the multiple channels of frequency domain data includes:

Step S501: Determine one or at least two channels of target frequency domain data from the multiple channels of frequency domain data according to the energy characteristic information;

Step S502: Determine frequency domain fusion data according to the one or at least two channels of target frequency domain data.

In this embodiment, during fusion, one or at least two channels of target frequency domain data to be fused can be determined from multiple channels of frequency domain data according to the energy characteristic information of the analog audio signal to be processed, for example, according to the simulation to be processed The size of the energy feature information of the audio signal determines the number of target frequency domain data. For example, the greater the energy feature information, the greater the number of target frequency domain data; in addition, the reference energy feature parameters of the processing circuit can also be preset according to the energy feature information Compare and determine one or at least two channels of target frequency domain data to be fused. Each of the multiple preset processing circuits corresponds to a different reference energy characteristic parameter. The reference energy characteristic parameter is included by the preprocessing circuit. Determined by the analog gain of the amplifier circuit, the reference energy characteristic parameter and the energy characteristic information belong to the same parameter, that is, the sound pressure level or amplitude of the digital audio signal output by the preprocessing circuit, which can be specified without distortion The maximum, minimum or intermediate value of the instantaneous amplitude of the digital audio signal, or the maximum, minimum or intermediate value of the average amplitude of the digital audio signal in a short time (preset duration), when the preprocessing circuit includes the amplifier circuit The larger the simulation gain of, the larger the corresponding reference energy characteristic parameter.

More specifically, in an optional embodiment, the determining one or at least two channels of target frequency domain data from the multiple channels of frequency domain data according to the energy characteristic information includes:

The first target frequency domain data and the second target frequency domain data are determined from the multiple channels of frequency domain data according to the energy feature information and multiple reference energy feature parameters, where the multiple reference energy feature parameters are based on the multiple frequency domain data. The analog gain of the amplifying circuit included in each preprocessing circuit is determined.

In this embodiment, at least two channels of target frequency domain data are determined from multiple channels of frequency domain data, where the first target frequency domain data may be only one channel of target frequency domain data, and of course, it may also be more than one channel of target frequency domain data; Similarly, the second target frequency domain data may also be only one channel of target frequency domain data or more than one channel of target frequency domain data. Wherein, the determining the first target frequency domain data and the second target frequency domain data from the multiple channels of frequency domain data according to the energy characteristic information may specifically include:

Step S601: Determine a first reference energy characteristic parameter and a second reference energy characteristic parameter adjacent to the energy characteristic information from a plurality of reference energy characteristic parameters;

Step S602: Determine first target frequency domain data and second target frequency domain data from the multiple channels of frequency domain data according to the first reference energy characteristic parameter and the second reference energy characteristic parameter;

Wherein, the first target frequency domain data and the second target frequency domain data are obtained by performing the frequency domain conversion on the first digital audio signal and the second digital audio signal in the multi-channel digital audio signal, respectively. A digital audio signal and a second digital audio signal are obtained from the analog audio signal to be processed by a first preprocessing circuit and a second preprocessing circuit corresponding to the first reference energy characteristic parameter and the second reference energy characteristic parameter, respectively .

Step S603: Determine frequency domain fusion data according to the first target frequency domain data and the second target frequency domain data.

In this embodiment, the energy of the analog audio signal to be processed is selected from the reference energy characteristic parameters of multiple preset processing circuits (indicated by L1, L2,..., LI, where I is the number of preset processing circuits) The feature information (represented by Lc) is adjacent to the first reference energy feature parameter ( _{represented by L i′} , where 1≤i′≤I-1) and the second reference energy feature parameter (represented by L _i′+1 ), That is, Lc is between L _i′ and L _i′+1 , where the first reference energy characteristic parameter L _i′ corresponds to the first preprocessing circuit, and the first digital audio signal output by the first preprocessing circuit undergoes frequency domain conversion The obtained frequency domain data is the first target frequency domain data, and the second reference energy characteristic parameter L _i′+1 corresponds to the second preprocessing circuit, and the second digital audio signal output by the second preprocessing circuit passes through the frequency domain. The converted frequency domain data is the second target frequency domain data, and the frequency domain fusion data can be obtained by performing superposition operation according to the first target frequency domain data and the second target frequency domain data.

Of course, in this embodiment, after determining the first reference energy characteristic parameter Li and the second reference energy characteristic parameter L _i′+1 , L _i′ and L _i′-1 (in this case, i′>1 is required) correspond to The frequency domain data obtained by the frequency domain conversion of the digital audio signal output by the preprocessing circuit is the first target frequency domain data, which can also be L _i′+1 and L _i′+2 (in this case, i′<I- 1) The frequency domain data obtained by frequency domain conversion of the digital audio signal output by the corresponding preprocessing circuit is the second target frequency domain data. Of course, the first target frequency domain data and the second target frequency domain data may also include more For multiple channels of frequency domain data, no examples are given here.

In another optional embodiment, the determining one or at least two channels of target frequency domain data from the multiple channels of frequency domain data according to the energy characteristic information includes:

Step S701: When the energy feature information is less than the smallest third reference energy feature parameter among the multiple reference energy feature parameters, determine from the multiple channels of frequency domain data according to the third reference energy feature parameter The third target frequency domain data;

Wherein, the third target frequency domain data is obtained by performing the frequency domain conversion on a third digital audio signal in the multi-channel digital audio signal, and the third digital audio signal is corresponding to a third reference energy characteristic parameter Obtained by the third preprocessing circuit on the analog audio signal to be processed;

Step S702: Acquire the frequency domain fusion data according to the third target frequency domain data.

In this embodiment, if among the reference energy characteristic parameters (L1, L2,..., LI) of the plurality of preset processing circuits, the smallest third reference energy characteristic parameter L1 is greater than the energy characteristic information of the analog audio signal to be processed Lc, that is, Lc is less than L1, where the third reference energy characteristic parameter L1 corresponds to the third digital audio signal output by the third preprocessing circuit, and the frequency domain data obtained by frequency domain conversion is the third target frequency domain data, and then Acquire the frequency domain fusion data according to the third target frequency domain data.

Of course, in this embodiment, after determining that the energy feature information Lc of the analog audio signal to be processed is smaller than the third reference energy feature parameter L1, the digital audio signal output by the preprocessing circuit corresponding to L1 and L2 can be obtained by frequency domain conversion. The frequency domain data is the third target frequency domain data. Of course, the third target frequency domain data may also include more channels of frequency domain data, and no examples are given here.

In another optional embodiment, the fusing the multiple channels of frequency domain data to obtain frequency domain fusion data further includes:

Step S801: When the energy feature information is greater than the largest fourth reference energy feature parameter among the multiple reference energy feature parameters, determine from the multiple channels of frequency domain data according to the fourth reference energy feature parameter The fourth target frequency domain data;

Wherein, the fourth target frequency domain data is obtained by performing the frequency domain conversion on the fourth digital audio signal in the multi-channel digital audio signal, and the fourth digital audio signal is corresponding to the fourth reference energy characteristic parameter Obtained by the fourth preprocessing circuit on the analog audio signal to be processed;

Step S802: Acquire the frequency domain fusion data according to the fourth target frequency domain data.

In this embodiment, if among the reference energy characteristic parameters (L1, L2,..., LI) of the plurality of preset processing circuits, the largest fourth reference energy characteristic parameter LI is less than the energy characteristic information of the analog audio signal to be processed Lc, that is, Lc is greater than LI, where the fourth reference energy characteristic parameter LI corresponds to the fourth digital audio signal output by the fourth preprocessing circuit and the frequency domain data obtained by frequency domain conversion is the fourth target frequency domain data, and then The frequency domain fusion data is acquired according to the fourth target frequency domain data.

Of course, in this embodiment, after it is determined that the energy feature information Lc of the analog audio signal to be processed is greater than the fourth reference energy feature parameter LI, the digital audio signal output by the preprocessing circuit corresponding to LI and LI-1 is converted into the frequency domain. The obtained frequency-domain data is the fourth target frequency-domain data. Of course, the fourth target frequency-domain data may also include more channels of frequency-domain data, which will not be illustrated here.

On the basis of the foregoing embodiment, when one or at least two channels of target frequency domain data are determined from the multiple channels of frequency domain data according to the energy feature information, the energy feature information Lc of the analog audio signal to be processed may be first determined. Compare with the reference energy characteristic parameters of multiple preset processing circuits, if Lc is less than (or equal to) the third reference energy characteristic parameter L1, perform steps 701-702; if Lc is greater than (or equal to) the fourth reference energy characteristic If the parameter LI is between steps 801-802; if Lc is between the adjacent third reference energy characteristic parameter L1 and the fourth reference energy characteristic parameter LI, then steps 601-603 are performed.

On the basis of any of the foregoing embodiments, the determining frequency domain fusion data according to the one or at least two channels of target frequency domain data includes:

Perform a superposition operation on the at least two channels of frequency domain data to obtain frequency domain fusion data.

In this embodiment, when at least two channels of target frequency domain data are fused, the frequency domain fusion data is obtained by performing a spectrum superposition operation on the frequency domain data.

Further, when the one or more channels of frequency domain data include the first target frequency domain data and the second target frequency domain data, the acquiring frequency domain fusion data according to the one or more channels of frequency domain data includes:

Perform a superposition operation on the first channel and the second channel of frequency domain data according to the weights corresponding to the first target frequency domain data and the second target frequency domain data to obtain frequency domain fusion data.

In this embodiment, performing the superposition operation can set weights for the first target frequency domain data and the second target frequency domain data, by superposing the first target frequency domain data and the second target frequency domain data with different weights. , You can get frequency-domain fusion data with different dynamic ranges.

Further, each of the plurality of preset processing circuits corresponds to a different reference energy characteristic parameter, and the plurality of reference energy characteristic parameters are determined according to the analog gain of the amplifying circuit included in the plurality of preprocessing circuits , Wherein the weights corresponding to the first target frequency domain data and the second target frequency domain data are based on the first preprocessing circuit and the first preprocessing circuit corresponding to the first target frequency domain data among the plurality of preset processing circuits The reference energy characteristic parameter of the second preprocessing circuit corresponding to the second target frequency domain data is determined.

In this embodiment, the weights corresponding to the first target frequency domain data and the second target frequency domain data are determined by the reference energy characteristic parameters of the corresponding pre-processing circuit. More specifically, the weights corresponding to the first target frequency domain data can be determined. The magnitude relationship between the first reference energy feature parameter Li and the second reference energy feature parameter Li+1 corresponding to the second target frequency domain data and the energy feature information Lc of the analog audio signal to be processed is determined, wherein the reference energy feature parameter is about If the energy characteristic information Lc is close to the analog audio signal, the greater the weight, for example, Lc is closer to Li, it means that the analog audio signal is closer to the digital audio signal of the preprocessing circuit corresponding to Li, and the first target frequency corresponding to Li needs to be increased. The weight of the domain data. In this embodiment, the weight a1 of the first target frequency domain data and the weight a2 of the second target frequency domain data can be determined by the following formula:

On the basis of any of the foregoing embodiments, the determining frequency domain fusion data according to one or at least two channels of target frequency domain data among the multiple channels of frequency domain data includes:

Step S901: Perform compression processing on the one or at least two channels of target frequency domain data according to the compression coefficient corresponding to each channel of the one or at least two channels of target frequency domain data;

Step S902: Acquire frequency domain fusion data according to one or more channels of frequency domain data after the compression processing.

In this embodiment, since the output of the recording system has a digital quantization range, that is, there are restrictions on the maximum amplitude and minimum amplitude of the audio signal, the maximum amplitude of the audio signal cannot be greater than the maximum threshold, and the minimum amplitude of the audio signal The value cannot be less than the minimum threshold. Therefore, when acquiring the frequency domain fusion data, the target frequency domain data needs to be compressed to avoid the fusion frequency domain fusion data from exceeding the digital quantization range. In this embodiment, the step of compressing the target frequency domain data can be performed simultaneously with the above-mentioned superposition operation, or can be completed before the superposition operation.

Further, the compression processing is linear compression processing.

On the basis of the foregoing embodiment, the compression coefficient corresponding to each channel of frequency domain data of the one or more channels of frequency domain data is determined according to the analog gain of the amplifier included in the preprocessing circuit corresponding to each channel of frequency domain data.

In this embodiment, the compression coefficient corresponding to any channel of frequency domain data can be the product of the channel equalization parameter and the scaling factor of the corresponding preprocessing circuit. A certain preprocessing circuit is used as a reference preprocessing circuit, and the The channel equalization parameter is the ratio of the analog gain of the preprocessing circuit to the analog gain of the reference preprocessing circuit, and the scaling factor is obtained according to the size of the digital audio signal output by the preprocessing circuit. In this embodiment, the compression coefficient corresponding to any channel of frequency domain data can be obtained by the following formula:

among them,

Is the channel equalization parameter, used to perform amplitude equalization between the frequency domain data of the i′th preprocessing circuit and the frequency domain data of the reference preprocessing circuit, G _i′ is the analog gain of the preprocessing circuit, and G _ref is the reference preprocessing The analog gain of the circuit, α is the scaling factor, used to scale the frequency domain data. Generally speaking, for small signals, α≥1, so that the signal is maintained or amplified; for large signals, α<1, so that the signal is reduced , So as to achieve the purpose of compressing the dynamic range.

It should be noted that steps S901-902 can be executed only when the frequency domain fusion data exceeds the digital quantization range, or it can be judged before step S901 whether the frequency domain fusion data has the possibility of exceeding the digital quantization range, if there is a possibility that the frequency domain fusion data exceeds the digital quantization range. Steps S901-902 are executed only when the range is possible.

On the basis of any of the foregoing embodiments, the time domain audio signal is the current frame time domain audio signal, wherein the step S404 in the foregoing embodiment described in step S404 obtaining the output audio signal according to the time domain audio signal includes:

Superimposing the current frame time domain audio signal with the historical frame time domain audio signal obtained before the current frame time domain audio signal to obtain the current frame time domain fusion audio signal;

The output audio signal is determined according to the time domain fusion audio signal of the current frame.

In this embodiment, the above-mentioned audio signal processing procedures are performed in units of one frame signal. In order to ensure the continuity of the signal, there may be a certain overlap between adjacent frame signals, that is, the tail of the previous frame signal and the next The header of the frame signal has an overlap amount, thereby establishing the correlation between adjacent frames. Therefore, after converting the frequency domain fusion data of the current frame into the time domain audio signal of the current frame in S404, the overlapping part of the time domain audio signal of the current frame and the time domain audio signal of the previous frame can be overlapped and superimposed. The non-overlapping portion of the time domain audio signal and the previous frame of time domain audio signal is not superimposed, so as to obtain the current frame time domain fused audio signal, and the output audio signal can be determined according to the current frame time domain fused audio signal.

Please refer to FIG. 11 in conjunction with the foregoing embodiment. FIG. 11 is a schematic block diagram of a terminal device 600 according to an embodiment of this specification. The terminal device 600 includes a processor 601 and a memory 602, and also includes an audio sensor 603 and a speaker 604.

The audio sensor 603 is used to collect environmental sounds of the terminal device 600, and the speaker 604 is used to play audio information.

Exemplarily, the processor 601 and the memory 602 are connected by a bus 605, and the bus 605 is, for example, an I2C (Inter-integrated Circuit) bus.

Specifically, the processor 601 may be a micro-controller unit (MCU), a central processing unit (Central Processing Unit, CPU), a digital signal processor (Digital Signal Processor, DSP), or the like.

Specifically, the memory 602 may be a Flash chip, a read-only memory (ROM, Read-Only Memory) disk, an optical disk, a U disk, or a mobile hard disk.

Wherein, the processor 601 is used to run a computer program stored in the memory 602, and implement the aforementioned control method for terminal equipment when the computer program is executed.

Exemplarily, the processor 601 is configured to run a computer program stored in the memory 602, and implement the following steps when the computer program is executed:

The specific principles and implementation manners of the terminal device provided in the embodiment of this specification are similar to the control method for the terminal device in the foregoing embodiment, and will not be repeated here.

The embodiments of this specification also provide a computer-readable storage medium, the computer-readable storage medium stores a computer program, the computer program includes program instructions, and the processor executes the program instructions to implement the foregoing implementation The example provides the steps of the control method for terminal equipment.

The computer-readable storage medium may be the internal storage unit of the terminal device described in any of the foregoing embodiments, such as the hard disk or memory of the terminal device. The computer-readable storage medium may also be an external storage device of the terminal device, such as a plug-in hard disk equipped on the terminal device, a smart memory card (Smart Media Card, SMC), and Secure Digital (SD). ) Card, Flash Card, etc.

Please refer to FIG. 12, which is a schematic block diagram of a movable platform 700 according to an embodiment of the present specification. The mobile platform 700 includes a processor 701 and a memory 702, and also includes an audio sensor 703 and a speaker 704.

The audio sensor 703 is used to collect environmental sounds of the movable platform 700, and the speaker 704 is used to play audio information.

Exemplarily, the processor 701 and the memory 702 are connected by a bus 705, and the bus 705 is, for example, an I2C (Inter-integrated Circuit) bus.

Specifically, the processor 701 may be a micro-controller unit (MCU), a central processing unit (Central Processing Unit, CPU), a digital signal processor (Digital Signal Processor, DSP), or the like.

Specifically, the memory 702 may be a Flash chip, a read-only memory (ROM, Read-Only Memory) disk, an optical disk, a U disk, or a mobile hard disk.

Wherein, the processor 701 is configured to run a computer program stored in the memory 702, and implement the aforementioned control method for a movable platform when the computer program is executed.

In some embodiments, as shown in FIG. 2, the movable platform may be a movable robot, and the movable robot may include:

An audio sensor and a speaker are arranged on the robot body 110, the audio sensor is used to collect environmental sounds, and the speaker is used to play audio;

Exemplarily, the processor 701 is configured to run a computer program stored in the memory 702, and implement the following steps when the computer program is executed:

The specific principles and implementation manners of the movable platform provided in the embodiment of this specification are similar to the control method for the movable platform of the foregoing embodiment, and will not be repeated here.

The embodiments of this specification also provide a computer-readable storage medium, the computer-readable storage medium stores a computer program, the computer program includes program instructions, the computer program is executed by a processor to cause the processing The device implements the steps of the control method for the movable platform provided in the above embodiment.

Wherein, the computer-readable storage medium may be the movable platform described in any of the foregoing embodiments, such as an internal storage unit of a movable robot, for example, a hard disk or memory of the movable robot. The computer-readable storage medium may also be an external storage device of the movable platform, for example, a plug-in hard disk equipped on the movable platform, a smart memory card (Smart Media Card, SMC), and Secure Digital (Secure Digital). , SD) card, flash card (Flash Card), etc.

The mobile platform, terminal device and its control method, and storage medium provided in the above-mentioned embodiments of this specification collect audio data of its surrounding environment through the terminal device and send the audio data to the corresponding mobile platform, so that users can A place far away from the movable platform makes a sound through the movable platform, for example, shouting to a person near the movable platform or other movable platforms. In addition, collecting the audio data of the surrounding environment through the mobile platform and sending the audio data to the terminal device of the mobile platform can enable the user of the terminal device to listen to the mobile platform even if it is not in the vicinity of the mobile platform. The sound scene of the environment where the platform is located can facilitate the user to interact with the surrounding environment more conveniently and intuitively, facilitate the user's control of the movable platform, and meet the user's purpose of transmitting voice information.

It should be understood that the terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the specification.

It should also be understood that the term "and/or" used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

The above are only specific implementations of this specification, but the scope of protection of this specification is not limited to this. Any person skilled in the art can easily think of various equivalents within the technical scope disclosed in this specification. Modifications or replacements, these modifications or replacements shall be covered within the protection scope of this manual. Therefore, the protection scope of this specification should be subject to the protection scope of the claims.

Claims

A control method, characterized in that it is applied to a system composed of a terminal device and a movable platform, and both the terminal device and the movable platform are provided with audio sensors and speakers;

The method includes:

The terminal device obtains a terminal sound file according to the environmental sound of the terminal device, and sends the terminal sound file to the mobile platform;

The mobile platform receives the terminal sound file sent by the terminal device, decodes the terminal sound file, and plays it;

The movable platform generates a platform sound file according to the environmental sound of the movable platform;

After receiving the platform sound file sent by the mobile platform, the terminal device decodes the platform sound file to obtain platform audio information, and plays the platform audio information.
A control method, characterized in that it is used in a terminal device, the terminal device is used to communicate with a movable platform, and both the terminal device and the movable platform are provided with an audio sensor and a speaker;

The method includes:

Acquiring a terminal sound file according to the environmental sound of the terminal device;

Sending the terminal sound file to the mobile platform, so that the mobile platform decodes the terminal sound file and plays it;

Acquiring a platform sound file sent by the mobile platform, the platform sound file being generated by the mobile platform after collecting the environmental sound of the mobile platform;

Decoding the platform sound file to obtain platform audio information, and playing the platform audio information.
The method according to claim 2, further comprising:

A platform control interface is displayed, and the platform control interface includes an intercom button.
The method according to claim 3, wherein the obtaining a terminal sound file according to the environmental sound of the terminal device comprises:

According to the user's intercom control operation on the intercom button, the environmental sound of the terminal device is acquired, and the terminal sound file is obtained by encoding.
The method according to claim 4, wherein the method further comprises:

The processing status of the environmental sound is displayed on the platform control interface, and the processing status includes at least one of silent recording, recording, transmission, and transmission completed.
The method according to claim 4 or 5, wherein the method further comprises:

When the environmental sound of the terminal device is acquired, the sound spectrogram of the environmental sound is displayed on the platform control interface.
The method according to claim 4, wherein the obtaining the environmental sound of the terminal device according to the intercom control operation of the intercom button by the user, and encoding to obtain the terminal sound file comprises:

The environmental sound of the terminal device is acquired in the time period after the intercom button is pressed to before the intercom button is released, and the terminal sound file is obtained by encoding.
8. The method according to claim 7, wherein said acquiring the environmental sound of the terminal device according to the user's intercom control operation of the intercom button, further comprising:

If the intercom button is pressed for a preset duration, stop acquiring the ambient sound of the terminal device, and encode the acquired sound to obtain the terminal sound file.
The method according to claim 4, further comprising:

According to the setting operation of the intercom button by the user, the intercom control operation of the intercom button by the user is enabled or disabled, and the display mode of the intercom button is adjusted.
The method according to claim 3, wherein the platform control interface further comprises a platform control button, and the method further comprises:

According to a button trigger operation of the platform control button by the user, a corresponding platform control instruction is generated and sent to the movable platform, so that the movable platform executes a preset task according to the platform control instruction.
The method according to any one of claims 2-5 and 7-10, wherein the method further comprises:

Display the recording record list, and obtain the terminal sound file corresponding to the recording record according to the user's playback control operation on the recording record in the recording record list.
The method according to claim 11, wherein the method further comprises:

According to the new recording operation of the user, the newly recorded sound is acquired and processed to obtain a new terminal sound file, and the corresponding recording record is updated in the recording record list.
The method according to claim 11, wherein the method further comprises:

Obtaining a terminal sound file corresponding to the recording record according to the user's loop playback operation of the recording record in the recording record list;

The sending the terminal sound file to the mobile platform so that the mobile platform decodes the terminal sound file and then plays it includes:

The terminal sound file and the loop instruction are sent to the movable platform, so that the movable platform decodes the terminal sound file and plays it in a loop.
The method according to any one of claims 2-5 and 7-10, wherein the method further comprises:

Display a list of recording records;

According to the user's playback control operation of the recording records in the recording record list, the information of the recording records is sent to the mobile platform, so that the mobile platform can play the terminal audio information corresponding to the recording records.
The method according to claim 14, wherein the method further comprises:

According to the user's new recording operation, acquiring and processing the newly recorded sound to obtain a new terminal sound file, and updating the corresponding recording record in the recording record list;

The terminal sound file is sent to the mobile platform, so that the mobile platform stores terminal audio information obtained by decoding the terminal sound file.
The method according to claim 15, wherein the sending the terminal sound file to the mobile platform comprises:

Send the terminal sound file to the mobile platform immediately after obtaining the new terminal sound file; or

Send the terminal sound file to the mobile platform according to the user's playback control operation on the recording record corresponding to the new terminal sound file.
The method according to claim 14, wherein the method further comprises:

According to the user's circular playback operation of the recording records in the recording record list, send the recording record information and cycle instructions to the mobile platform, so that the mobile platform can play the terminal audio corresponding to the recording record in a loop information.
The method according to any one of claims 11-17, wherein the method further comprises:

The playback status and/or playback progress of the audio recording are displayed in the audio recording list.
The method according to any one of claims 2-5 and 7-10, wherein the platform sound file is generated by the movable platform according to a sound from at least another movable platform, or The movable platform is generated based on the voice of a user of a terminal device originating from at least another movable platform.
The method according to any one of claims 2-5 and 7-10, wherein the method further comprises:

Acquire a user's control voice, generate and send a corresponding platform control instruction to the movable platform according to the control voice, so that the movable platform executes a preset task according to the platform control instruction.
The method according to claim 20, wherein said obtaining the control voice of the user comprises:

Acquire the environmental sound of the terminal device, and detect the control voice in the environmental sound; and/or

When the user triggers the voice control function, the control voice sent by the user is obtained.
The method according to any one of claims 2-5 and 7-10, wherein the method further comprises:

Determine the playback object of the terminal sound file according to the user's object setting operation;

The sending the terminal sound file to the mobile platform so that the mobile platform decodes the terminal sound file and then plays it includes:

The terminal sound file and the information of the playback object are sent to the movable platform, so that the movable platform plays the terminal sound file when the playback object is recognized.
The method according to claim 22, wherein the determining the playback object of the terminal sound file according to the user's object setting operation comprises:

Acquiring an image taken by the movable platform from the movable platform, and displaying the image;

According to the user's selection operation of the playback object in the image, the playback object is determined.
The method according to claim 22, wherein the determining the playback object of the terminal sound file according to the user's object setting operation comprises:

Displaying the local image selected by the user according to the user's selection operation on the local image of the terminal device;

The playback object is determined according to the user's determining operation on the playback object in the local image.
The method according to any one of claims 2-5 and 7-10, wherein the method further comprises:

Send a sound return instruction to the movable platform according to the user's operation, so that the movable platform collects the environmental sound of the movable platform according to the sound return instruction, generates the platform sound file, and sends the sound file to the mobile platform. The terminal device sends.
The method according to any one of claims 4-9, wherein the obtaining the environmental sound of the terminal device and encoding to obtain the terminal sound file comprises:

Encode the currently collected environmental sound in real time, and generate a terminal sound file in the corresponding encoding format after the recording ends.
The method according to any one of claims 2-5 and 7-10, wherein the movable platform comprises a movable robot, and the movable robot comprises:

The robot body includes a chassis main body and a pan/tilt main body provided on the chassis main body, and the pan/tilt main body is used to carry a camera device;

A power device, which is provided on the chassis body, and is used to provide moving power to the robot body;

An audio sensor and a speaker are provided on the robot body, the audio sensor is used to collect environmental sounds, and the speaker is used to play audio;

The communication device is arranged on the robot body and used to communicate with the terminal device.
A control method, characterized in that it is applied to a movable platform, the movable platform is used to communicate with a terminal device, and both the terminal device and the movable platform are provided with audio sensors and speakers;

The method includes:

Acquire the terminal sound file sent by the terminal device, decode the terminal sound file to generate terminal audio information, and play the terminal audio information; wherein the terminal sound file is collected by the terminal device from the environment of the terminal device Generated after the sound;

Generate a platform sound file according to the environmental sound of the movable platform;

Send the platform sound file to the terminal device, so that the terminal device decodes the platform sound file and plays it.
The method according to claim 28, wherein the generating a platform sound file according to the environmental sound of the movable platform comprises:

If a sound return instruction sent by the terminal device is received, the environmental sound of the movable platform is collected according to the sound return instruction, and the platform sound file is generated.
The method according to claim 28, wherein the method further comprises:

If the information of the recording record sent by the terminal device is acquired, the terminal audio information corresponding to the recording record is determined, and the terminal audio information corresponding to the recording record is played.
The method according to any one of claims 28-30, wherein the playing the terminal audio information comprises:

If the loop instruction corresponding to the terminal audio information is acquired, the terminal audio information is played in a loop.
The method according to any one of claims 28-30, wherein the method further comprises:

Send the playback status and/or playback progress of the terminal audio information to the terminal device.
The method according to any one of claims 28-30, wherein the method further comprises:

When the terminal audio information is played, the display parameters of the display device of the movable platform are adjusted according to the terminal audio information.
The method according to claim 33, wherein the adjusting the display parameters of the display device of the movable platform according to the terminal audio information comprises:

Adjusting the display brightness of the display device according to the sound intensity of the terminal audio information; and/or

Adjust the flicker frequency of the display device according to the sound frequency of the terminal audio information.
The method according to any one of claims 28-30, wherein the generating a platform sound file according to the environmental sound of the movable platform comprises:

A platform sound file is generated based on the sound of at least another movable platform and/or the sound of a user of at least another terminal device of the movable platform.
The method according to claim 35, wherein the method further comprises:

When the terminal audio information is not played, obtain the environmental sound of the movable platform; and/or

Acquire the environmental sound of the movable platform, and filter the played terminal audio information from the environmental sound.
The method according to any one of claims 28-30, wherein the method further comprises:

If the platform control instruction sent by the terminal device is acquired, execute a preset task according to the platform control instruction;

Wherein, the platform control instruction is sent by the terminal device according to a button trigger operation of the platform control button of the terminal device by the user, or sent by the terminal device according to the control voice of the user.
The method according to any one of claims 28-30, wherein the movable platform is equipped with a camera device, and the method further comprises:

If acquiring the information of the playback object sent by the terminal device, identify the playback object in the image captured by the camera device according to the information;

If the playback object is recognized, the corresponding terminal audio information is played to the playback object.
The method according to claim 38, wherein the method further comprises:

The image taken by the camera device is sent to the terminal device, so that the terminal device determines the playback object according to the user's selection operation of the playback object in the image.
A terminal device, characterized in that it includes an audio sensor, a speaker, a memory, and a processor;

The audio sensor is used to collect environmental sounds of the terminal device, and the speaker is used to play audio information;

The memory is used to store a computer program;

The processor is configured to execute the computer program and, when executing the computer program, implement the following steps:

Acquiring a terminal sound file according to the environmental sound of the terminal device;

Sending the terminal sound file to a mobile platform, so that the mobile platform decodes the terminal sound file and plays it;

Acquiring a platform sound file sent by the mobile platform, the platform sound file being generated by the mobile platform after collecting the environmental sound of the mobile platform;

Decoding the platform sound file to obtain platform audio information, and playing the platform audio information.
A movable platform, which is characterized in that it comprises an audio sensor, a speaker, a memory and a processor;

The audio sensor is used to collect environmental sounds of the movable platform, and the speaker is used to play audio information;

The memory is used to store a computer program;

The processor is configured to execute the computer program and, when executing the computer program, implement the following steps:

Obtain the terminal sound file sent by the terminal device, decode the terminal sound file to generate terminal audio information, and play the terminal audio information; wherein, the terminal sound file is collected by the terminal device after the terminal device’s environmental sound generate;

Generate a platform sound file according to the environmental sound of the movable platform;

Send the platform sound file to the terminal device, so that the terminal device decodes the platform sound file and plays it.
A computer-readable storage medium, characterized in that, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor realizes as described in any one of claims 1-39. The method described.