CN112104963A

CN112104963A - Following type robot sound amplification method and system

Info

Publication number: CN112104963A
Application number: CN202011185926.4A
Authority: CN
Inventors: 陈孝良; 常乐; 阮明江
Original assignee: Beijing SoundAI Technology Co Ltd
Current assignee: Beijing SoundAI Technology Co Ltd
Priority date: 2020-10-30
Filing date: 2020-10-30
Publication date: 2020-12-18
Anticipated expiration: 2040-10-30
Also published as: CN112104963B

Abstract

The invention is suitable for the technical field of sound amplification, and provides a following robot sound amplification method and a following robot sound amplification system, wherein the following robot sound amplification method comprises the following steps: under the condition that the first sound collection device obtains the voice of the commentator, controlling a sound amplification device of the robot to perform sound amplification playing on the voice of the commentator; determining the real-time distance between the robot and a positioning label according to the positioning label arranged at the preset position of the commentator; and adjusting the position of the robot relative to the positioning label according to the real-time distance and the preset distance. The following type movable robot is used for playing the voice of the commentator in a sound amplification way, so that the burden that the commentator needs to wear the sound amplification equipment with him is avoided; through the position of adjusting the robot for the commentator in real time at the in-process of explaining, ensure that the digital information transmission between first sound collection system and the robot is stable, pronunciation can be transmitted to the robot with the best mode and carry out the public address broadcast, have improved whole explanation effect.

Description

Following type robot sound amplification method and system

Technical Field

The invention belongs to the technical field of sound amplification, and particularly relates to a following robot sound amplification method and system.

Background

In the display process of the exhibition hall, the commentator is generally required to introduce all parts of the exhibition hall to the visitors, and when the exhibition hall is large in area or numerous in number, the commentator needs to continuously improve the explanation sound to ensure that the visitors can hear the explanation. However, prolonged loud explanation can make the narrator fatiguing.

At present, a common mode is that the commentator carries with him a public address device, such as a bee or a loudspeaker, which can play the role of amplifying sound, but on one hand, the sound amplifying effect of the bee or the loudspeaker is limited, and on the other hand, because the bee or the loudspeaker have a certain weight, the burden of the commentator is increased when the commentator is continuously worn, so that the commentator is easy to fatigue, and the overall commentary effect is affected.

Disclosure of Invention

In view of this, the embodiment of the present invention provides a following robot sound amplification method and system, so as to solve the technical problem in the prior art that the overall explanation effect is not good in the exhibition hall display process.

In a first aspect of the embodiments of the present invention, a following robot sound amplification method is provided, including:

under the condition that the first sound collection device obtains the voice of the commentator, controlling a sound amplification device of the robot to perform sound amplification playing on the voice of the commentator;

determining the real-time distance between the robot and a positioning label according to the positioning label arranged at the preset position of the commentator;

and adjusting the position of the robot relative to the positioning label according to the real-time distance and the preset distance.

In a second aspect of the embodiments of the present invention, there is provided a mobile robot-based sound reinforcement system, including: the robot is provided with a control device and a sound amplification device, and the first sound acquisition device is in communication connection with the control device;

the control device includes:

the voice amplification control module is used for controlling the voice amplification device of the robot to perform voice amplification playing on the voice of the commentator under the condition that the first voice collection device obtains the voice of the commentator;

the distance determining module is used for determining the real-time distance between the robot and a positioning label according to the positioning label arranged at the preset position of the commentator;

and the position adjusting module is used for adjusting the position of the robot relative to the positioning label according to the real-time distance and the preset distance.

A third aspect of the embodiments of the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the method when executing the computer program.

A fourth aspect of embodiments of the present invention provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements the steps of the above-described method.

Compared with the prior art, the embodiment of the invention has the following beneficial effects: in the embodiment, the voice of the commentator is acquired in real time through the first sound acquisition device, and is played after being amplified through the following type movable robot, so that the burden that the commentator needs to wear a public address device with him is avoided; meanwhile, the position of the robot relative to the commentator is adjusted in real time in the commentary process, so that the distance between the robot and the commentator is always maintained within a preset range, the stable transmission of digital information between the first sound collection device and the robot is ensured, voice can be transmitted to the robot in an optimal mode to be played in a sound amplification mode, and the overall commentary effect is improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed for the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

Fig. 1 is a schematic structural diagram i of a following robot acoustic amplification system according to an embodiment of the present invention;

fig. 2 is a schematic flow chart of a first implementation process of a following robot sound amplification method provided by an embodiment of the present invention;

fig. 3 is a schematic flow chart illustrating an implementation flow of step S21 in the following robot acoustic amplification method according to the embodiment of the present invention;

fig. 4 is a schematic flow chart illustrating the implementation of step S213 in the following robot acoustic amplification method according to the embodiment of the present invention;

fig. 5 is a schematic flow chart of an implementation process of a following robot acoustic amplification method according to an embodiment of the present invention;

fig. 6 is a schematic structural diagram ii of a following robot acoustic amplification system according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram three of a following robot acoustic amplification system provided in the embodiment of the present invention;

fig. 8 is a first schematic flow chart illustrating an implementation process of step S22 in the following robot acoustic amplification method according to the embodiment of the present invention;

fig. 9 is a schematic flow chart illustrating an implementation process of step S22 in the following robot acoustic amplification method according to the embodiment of the present invention;

fig. 10 is a first schematic diagram of a control device in a following robot acoustic amplification system according to an embodiment of the present invention;

fig. 11 is a schematic diagram of a sound amplification control module in a following robot sound amplification system provided in an embodiment of the present invention;

fig. 12 is a second schematic diagram of a control device in a following robot acoustic amplification system according to an embodiment of the present invention;

fig. 13 is a schematic diagram of a terminal device according to an embodiment of the present invention.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

In order to explain the technical means of the present invention, the following description will be given by way of specific examples.

In the display process of the exhibition hall, the commentator is generally required to introduce all parts of the exhibition hall to the visitors, and when the exhibition hall is large in area or numerous in number, the commentator needs to continuously improve the explanation sound to ensure that the visitors can hear the explanation. In order to reduce the burden of the commentator, the commentator usually wears public address equipment, such as a bee or a loudspeaker, during the explanation. However, although the portable public address equipment can play a role of amplifying sound, on one hand, because the bee or the loudspeaker has a certain weight, the burden of the commentator is increased when the public address equipment is continuously worn on the body, and the commentator is easy to fatigue; on the other hand, in order to reduce the physical burden of the commentator as much as possible, the portable sound amplifying devices are usually small, so that the sound amplifying effect is limited, and the overall commentary effect is affected.

With the development of society and scientific progress, service robots have been widely used in various industries, for example, tour guide and explanation robots are used as functional public service robots and play an increasingly important role in exhibition in museums, science museums, product exhibition halls, and the like. However, the conventional navigation and explanation robot can only perform navigation and explanation by itself according to a set program, and cannot interact with an explanation person, so that the conventional navigation and explanation robot cannot cooperate with the explanation person to perform more efficient explanation.

The embodiment fully considers the explanation burden of the commentator in the explanation process and the possible effect of the service robot in the explanation process of the exhibition hall, and is creatively applied to the explanation process of the commentator, so that the commentator can cooperate with the commentator to play the commentary sound in an amplifying way in real time in the explanation process of the commentator, and the commentary sound can move according to the position change of the commentator, so that the signal transmission between the robot and the commentator is stable, the voice of the commentator can be transmitted to the robot in an optimal way to play the amplifying way, and the overall commentary effect is improved.

Referring to fig. 1, in a first aspect of the present embodiment, a mobile robot-based public address system 10 is provided, which includes a mobile robot 11, a positioning tag 12 disposed at a predetermined position of an announcer, and a first sound collection device 13. The robot 11 is provided with a control device 110, a sound reinforcement device 111 and a moving device 112, wherein the moving device 112 and the sound reinforcement device 111 are both connected with the control device 110, the sound reinforcement device 111 can perform sound reinforcement playback under the control of the control device 110, and the moving device 112 can drive the robot to move under the control of the control device 110. The moving device 112 may be specifically configured as required, and may be a universal wheel mounted at the bottom of the robot, for example, and may move in any direction. The control device 110 may adjust the position of the robot relative to the commentator by positioning the position of the robot with reference to the position of the positioning tag 12 and adjusting the position of the robot relative to the positioning tag 12 according to the positioning result. The control device 110 processes the voice collected by the first sound collection device 13, and plays the voice of the commentator after amplifying the voice.

The first sound collection device 13 may be of different types as long as it can collect the sound of the commentator. For example, the first sound collection device 13 may be a hand-held microphone or a microphone worn on the body of the commentator to facilitate the use of the commentator by holding the commentator, and the microphone is provided with a communication module, preferably a wireless communication module, which can wirelessly communicate with the control device 110 of the robot 11 to transmit collected voice to the control device 110. Of course, the microphone may also be connected to the control device in a wired manner, which is not limited herein. For another example, the first sound collection device 13 may be a headset, which is connected to an electronic device such as a mobile phone, and sends collected voice to the control device 110 through the electronic device. The first sound collection device 13 may also be a microphone array, which can effectively improve the sound collection quality.

The positioning tag 12 can be of different types according to different positioning modes, and can be arranged at any position of the commentator. For example, the positioning tag 12 may be provided on the first sound collection device 13, and in particular, when the first sound collection device 13 is a handheld microphone, the positioning tag 12 may be provided on the handheld microphone; when the first sound collection device 13 is a headset, the positioning tag 12 may be provided on the headset or on the electronic device. As another example, the positioning tag 12 may be disposed at a predetermined position on the commentator, for example, on a work card of the commentator, which is worn on the commentator. Of course, the positioning tag 12 may be disposed at other positions on the body of the commentator, and is not limited herein.

Referring to fig. 2, a second aspect of the present embodiment provides a following robot sound amplification method, which may be implemented by the sound amplification system described above, or implemented in other manners. The following robot amplification method will be described by taking the above-described amplification system as an example. The following robot sound amplification method can comprise the following steps:

step S21: and under the condition that the first sound collection device obtains the voice of the commentator, controlling a sound amplification device of the robot to perform sound amplification playing on the voice of the commentator.

When the commentator needs to explain, the commentator can start the first sound collection device 13, at the moment, the first sound collection device 13 is connected with the control device 110 of the robot, the first sound collection device 13 sends the voice collected in real time to the control device 110, the control device 110 receives the voice and then carries out voice processing on the voice, and controls the sound amplification device 111 to work, so that the voice is amplified and played, the volume of the commentator is increased, and the commentator can clearly hear the commentary content of the commentator even if the voice is small.

Step S22: and determining the real-time distance between the robot and the positioning label according to the positioning label arranged at the preset position of the commentator.

Since the position of the commentator changes during the commentary process or may change according to the commentary content, the distance between the commentator and the robot 11 changes continuously, and the user moves correspondingly along with the movement of the commentator. At this time, if the position of the robot 11 is kept unchanged all the time, on one hand, the digital information transmission path between the first sound collection device 13 and the control device 110 becomes unstable due to the change of the position between the commentator and the robot 11, so that the data transmission quality is reduced, and the sound amplification effect is influenced; on the other hand, the sound amplification effect of the robot 11 is weakened due to the change of the relative position between the robot 11 and the commentator and the user, for example, the increase of the distance between the robot 11 and the user may reduce the volume of the sound heard by the user and even make the played content inaudible, resulting in poor sound amplification effect. In order to ensure the sound amplification effect of the robot, in the present embodiment, the positioning tag 12 is disposed at a preset position of the commentator, and the control device 110 determines the real-time distance between the robot and the positioning tag 12 according to a preset frequency during the sound amplification playing process, so that the position of the robot relative to the commentator can be determined in real time. The frequency of determining the real-time distance may be set as needed, for example, the frequency of determining the real-time distance may be higher for a scene where the commentator changes positions frequently, and the frequency of determining the real-time distance may be lower for a scene where the commentator environment is relatively fixed.

Step S23: and adjusting the position of the robot relative to the positioning label according to the real-time distance and the preset distance.

After the real-time position of the robot relative to the commentator is obtained, whether the position of the commentator relative to the robot 11 changes or not needs to be confirmed, at this time, the real-time distance needs to be compared with the preset distance, whether the difference value between the real-time distance and the preset distance is larger than a threshold value or not is judged, if the difference value between the real-time distance and the preset distance is larger than the threshold value, the fact that the position of the commentator relative to the robot is changed greatly means, at this time, the robot 11 needs to be controlled to move towards the commentator by a distance at least larger than the difference value; if the difference between the real-time distance and the preset distance is not greater than the threshold, it means that the position of the commentator relative to the robot 11 does not change greatly, and the position of the robot 11 does not need to be changed. It should be understood that the difference between the real-time distance and the preset distance may be a positive value or a negative value. When the difference between the real-time distance and the preset distance is a positive value, it means that the distance of the robot 11 relative to the commentator is increased, and at this time, the robot needs to move towards a direction close to the commentator; when the difference between the real-time distance and the preset distance is a negative value, it means that the distance of the robot 11 relative to the commentator is reduced, and the robot needs to move away from the commentator.

The following robot sound amplification method provided by the embodiment has the beneficial effects that: in the embodiment, the voice of the commentator is collected in real time through the first sound collection device 13, and is played after being amplified through the movable robot 11, so that the burden that the commentator needs to wear a public address device with him is avoided; meanwhile, the position of the robot 11 relative to the commentator is adjusted in real time in the commentary process, so that the distance between the robot 11 and the commentator is always maintained within a preset range, stable transmission of digital information between the first sound collection device 13 and the robot 11 is ensured, voice can be transmitted to the robot 11 in an optimal mode for sound amplification playing, and the overall commentary effect is improved.

Further, the scheme shown in fig. 2 is that when receiving the voice of the commentator, the voice is played in a sound amplification manner directly, but some robots have a voice interaction function at present, and the robots can be controlled to perform some action or operation through the voice, so when the commentator speaks a certain command word for controlling the robots, if the robots play the command word of the commentator in a sound amplification manner, the interaction experience and the commentary effect are affected. In order to solve the problem, the present application improves step S21 in the solution shown in fig. 2, and as shown in fig. 3, in step S21, the step of controlling the sound reinforcement device of the robot to play the sound reinforcement of the voice of the commentator when the sound collection device acquires the voice of the commentator includes:

step S211: and performing voice recognition on the voice of the commentator, and determining whether the voice contains command words.

The first sound collection device 13 converts the analog signal into a digital signal after collecting the voice of the commentator, and sends the digital signal corresponding to the voice to the control device 110 of the robot 11, and the control device 110 performs voice processing after acquiring the digital signal to obtain corresponding text information. Optionally, the voice processing includes voice recognition, and in a specific implementation process, if it is considered that the local computing power of the robot 11 is insufficient or the situation that the robot 11 does not need to perform recognition work is considered, the control device 110 may send the digital signal to the cloud end/server end, and the cloud end/server end completes the voice recognition work, so as to obtain corresponding text information.

In order to make the explanation process more vivid, the commentator can also control the robot 11 to correspondingly perform some actions/operations through command words during the explanation process. The command words may include action command words, such as may cause the robot to "move forward", "move backward", "move left", "move right", "turn around", and the like. The command words may also include audio adjustment command words, such as volume adjustments, including "volume up", "volume down", "stereo", or timbre switching, including "standard timbre", "female timbre", "male timbre", timbre of a public character, i.e. "XXX" and the like.

Since the commentator may speak the command word at any time during the commentary process, in order to avoid the command word spoken by the commentator being played in a loud-speaker manner and to enable the robot to accurately execute the actions/operations, voice recognition needs to be performed on the voice of the commentator. Therefore, after the text information is obtained, the text information needs to be matched with a preset instruction text to determine whether the text information contains the instruction text.

If the text information does not contain the instruction text, the voice does not contain the command word, the explicator does not need the robot to execute the action/operation, only the robot needs to perform sound amplification playing, and the following steps are executed at this moment:

step S212: and controlling a sound amplification device of the robot to perform sound amplification playing on the voice.

If the text information contains an instruction text, it means that the voice contains a command word, and the commentator needs the robot to execute corresponding actions/operations, and then the following steps are executed:

step S213: and controlling the robot to execute the operation corresponding to the command word according to the command word.

Since the instruction text is associated with the corresponding command word, when the instruction text in the text information is determined, the robot may be controlled to perform corresponding actions/operations according to the instruction text, for example, the moving device 112 is controlled to drive the robot to move "forward", "backward", "left", "right", "turn", and the like.

In the process of playing the voice of the commentator in the way of amplifying the voice, the command words in the voice can be acquired by performing voice recognition on the voice, so that the robot can perform corresponding actions/operations in cooperation with the commentator in the process of amplifying the voice, and the commentary process is more vivid.

Further, in step S211, since the commentator may randomly or happens to speak some command words during the commentary process, the robot 11 may erroneously perform the action/operation corresponding to the random command words, resulting in some misoperation. In order to solve this problem, step S21 is further improved, that is, when it is determined that the voice of the commentator includes the command word (step S211), it is also necessary to perform text analysis on the voice to determine the intention of the commentator, that is, step S213 shown in fig. 4, and this step includes:

step S2131: and acquiring voices before and after the command word.

Here, the speech before and after the command word may be a sentence in which the command word is located, or may be a sentence in which the command word is located and a preset number of sentences before and after the sentence.

Step S2132: and performing semantic analysis on the command words and the voices before and after the command words to determine the intention of the commentator.

The command words in the voice are determined whether the command words are control instructions which are intentionally sent by the explainer to the robot or instructions which are randomly given or wrongly given according to semantic analysis, so that the intention of the explainer is clear. The express commentator's intent includes two possibilities, namely determining that the commentator's intent is to control the robot to execute the command word, and determining that the commentator's intent is not to control the robot to execute the command word.

Step S2133: controlling the robot to perform an operation corresponding to the intention based on the intention of the commentator.

And if the intention of the explicator is to control the robot to execute the command words, controlling the robot to execute the operation corresponding to the command words according to the command words.

If the intention of the commentator is not to control the robot to execute the command word, the step S212 is executed to control the voice amplification device of the robot to perform voice amplification playing.

According to the embodiment, semantic analysis is further performed after the command words are recognized through voice, so that the intention of the commentator can be determined, the robot is controlled to execute the action/operation corresponding to the intention based on the intention, and misoperation is avoided.

Further, in the process of controlling the robot to execute the operation corresponding to the command word according to the command word, if the command word includes an action command word, controlling the robot to execute a preset action corresponding to the action command word according to the action command word, for example, enabling the robot to "move forward", "move backward", "move left", "move right", "turn round", and the like; and if the command words comprise audio adjusting command words, controlling the sound reinforcement device to execute audio adjusting operation corresponding to the audio adjusting command words according to the audio adjusting command words, such as volume adjustment or tone switching.

Referring to fig. 5, further, before playing the voice of the commentator in the public address, the identity of the commentator needs to be verified to determine whether the commentator has the right to control the robot to play the public address. In addition, because a large amount of noise usually exists in a scene where the commentator is located, especially in the communication voice of the visitor, in order to avoid the sound amplification device of the robot to amplify other irrelevant voices, the present application improves the scheme shown in fig. 1, as shown in fig. 5, before the step of controlling the sound amplification device of the robot to amplify and play the voice of the commentator, a step of performing voiceprint recognition on the voice of the commentator is further included, which specifically includes:

step S20: and carrying out voiceprint recognition on the voice, and confirming whether the user sending the voice is the identity of the commentator.

According to the characteristics of the specificity and the stability of the voiceprint information, the voiceprint information of the voice is collected, the voiceprint information is matched with the preset voiceprint information to confirm the identity of the user providing the voice, the user is shown as the identity of the commentator only if the voiceprint information with the matching degree meeting the preset requirement, and therefore the corresponding authority is achieved, and the voice playing device of the robot is controlled to play the voice of the commentator in a voice playing mode. If the matching degree of the voice voiceprint information and the preset voiceprint information does not meet the preset requirement, the user is not the explicator, so that the user does not have corresponding authority, and the sound amplification device of the robot cannot be controlled to perform sound amplification playing.

It can be understood that in the process of playing the public address, the voiceprint recognition can be performed after each voice is received to confirm the identity, so as to ensure that the public address device only performs public address on the voice of the commentator. In addition, if voiceprint recognition is performed each time a voice is received, which may cause a problem of voice amplification delay due to too long voice processing time, in the process of one explanation, the voiceprint recognition may be performed only when the voice amplification playing is started, that is, the step S20 may be performed only once before the step of controlling the voice amplification device of the robot to perform the voice amplification playing on the voice of the commentator, and the voiceprint recognition is verified by the voiceprint recognition without repeating the voiceprint recognition in the voice amplification playing process, so that the problem of voice amplification delay is avoided.

Further, during the explanation process of the commentator, the visitors can also randomly ask questions, and the questions of the visitors can also be amplified through the follow-up robot sound amplification system, so that other visitors or the commentators can hear the questions. Therefore, the following robot sound amplification method provided by the embodiment further includes:

step S24: and acquiring the user voice through a second voice acquisition device, carrying out voice processing on the user voice, and determining whether the user voice is a question of the user.

The second sound collection device 113 may be disposed on the robot 11 (see fig. 6), or may be a mobile microphone, which is held by the visitor (see fig. 7). After the user voice is acquired through the second voice acquisition device, voice processing modes such as voice recognition and semantic analysis need to be performed on the user to determine whether the user voice is a question of the user. When confirming that the user voice is the user question, the step of confirming whether the user is the identity of the commentator is not needed to be executed at this time, and the user voice is directly played in a sound amplification way, namely:

step S25: and controlling a sound amplification device of the robot to perform sound amplification playing on the user voice.

When the user voice is not the user question, the user voice does not need to be played in a sound amplifying mode.

Further, in step S22, when acquiring the real-time distance between the robot and the positioning tag, different manners of acquiring the real-time distance may be provided according to different specific structures of the robot.

Referring to fig. 8, in an embodiment, the robot 11 is provided with at least three UWB (Ultra Wide Band) base stations with triangular positions, and the positioning tag is a UWB positioning tag, and the real-time distance between the robot and the positioning tag can be obtained through a UWB positioning technology. At this time, step S22 may include:

step S221: controlling a UWB base station of the robot to send a UWB signal to the positioning tag;

step S222: and determining the position information of the robot relative to the positioning tag according to the time of the positioning tag for receiving each UWB signal and the position relation between the UWB base stations, wherein the position information at least comprises the real-time distance of the robot relative to the positioning tag.

In this embodiment, three UWB base stations are arranged in an equilateral triangle, the UWB positioning tag is time-synchronized with the UWB base stations, the distance between the UWB positioning tag and each UWB base station can be obtained by TOA (time of arrival) algorithm, and then the coordinates of the UWB positioning tag can be obtained by geometric algorithm, so that the distance of the robot with respect to the UWB positioning tag can be determined.

Referring to fig. 9, in an embodiment, the robot 11 is provided with an image capturing device, in this case, step S22 may include:

step S223: controlling an image acquisition device of the robot to acquire an image of the positioning label;

step S224: and determining the real-time distance between the positioning label and the robot according to the acquired image of the positioning label.

According to different image acquisition devices, the above-mentioned manner of acquiring the real-time distance between the positioning tag and the robot is also different.

In one embodiment, the image acquisition device includes binocular camera, and the robot is equipped with left camera and right camera promptly, utilizes triangle range finding principle, can acquire the real-time distance of robot and location label, specifically can include:

controlling a binocular camera of the robot to acquire images of the positioning labels according to a preset frequency;

and matching the images of the positioning tags acquired by the binocular camera, and acquiring the real-time distance between the positioning tags and the robot according to the matching result.

Specifically, in ranging by the binocular cameras, camera calibration is first performed, internal parameters of each of the binocular cameras are acquired, and the relative position between the two cameras (i.e., the rotation matrix R and translation vector t of the right camera with respect to the left camera) is measured. And then binocular correction is carried out, namely distortion elimination and line alignment are respectively carried out on the left view and the right view according to monocular internal reference data (focal length, imaging origin, distortion coefficient) and binocular relative position relation (rotation matrix and translation vector) obtained after camera calibration, so that the imaging origin coordinates of the left view and the right view are consistent, the optical axes of the two cameras are parallel, the left imaging plane and the right imaging plane are coplanar, and the epipolar line is aligned, so that any point on one image obtained by one camera and the corresponding point on the other image obtained by the other camera have the same line number, and the corresponding point can be matched only by one-dimensional search of the line. And then, images of the positioning labels are obtained by controlling a binocular camera of the robot according to a preset frequency, wherein the left camera and the right camera simultaneously obtain the images of the positioning labels, binocular matching is carried out, and parallax data of the left camera and the right camera are obtained, so that depth information, namely the distance between the binocular camera and the positioning labels, can be calculated according to the parallax data, and further the real-time distance between the robot and the positioning labels can be determined. The location tag here may be a marker of any shape or structure that is easily captured and recognized by the camera.

In one embodiment, the image capturing device includes a structured light module and a camera, and projects a structured light pattern with encoded information to a space by using the structured light module, and the structured light pattern is captured by the camera and then analyzed, so as to obtain a real-time distance between the robot and the positioning tag, which specifically includes:

controlling a camera of the robot to acquire an image of the positioning label according to a preset frequency;

performing image processing on the image of the positioning label to acquire angle information of the camera relative to the positioning label;

according to the angle information, controlling a structured light module of the robot to project a structured light pattern to the positioning label;

and determining the real-time distance between the positioning label and the robot according to the collected structured light pattern reflected by the positioning label.

In the process of sound reinforcement, the camera of the robot does not face the position of the positioning tag at any time, so that before the structured light pattern is projected to the positioning tag, the angle information of the structured light module and the camera relative to the positioning tag needs to be acquired, so that the structured light module is controlled to project the structured light pattern to the positioning tag according to the angle information. The image of the positioning label with the structured light pattern is collected through the camera, the structured light pattern is obtained, the structured light pattern is analyzed, the distance between the structured light module and the positioning label can be obtained, and the real-time distance between the robot and the positioning label can be further determined. The location tag here may be a marker of any shape or structure that is easily captured and recognized by the camera. The structured light pattern may be a point structured light, a line structured light, or a simple area structured light, or an encoded optical pattern, without limitation.

In one embodiment, the image acquisition device includes an infrared ranging sensor and a camera, and by using the principle of triangulation, the real-time distance between the robot and the positioning tag can be acquired, which specifically includes:

controlling an infrared distance measuring sensor of the robot to project infrared rays to the positioning tag according to the angle information;

and determining the real-time distance between the positioning label and the robot according to the received infrared rays reflected by the positioning label.

In the process of sound reinforcement, the camera of the robot does not face the position of the positioning tag at any time, so that before the structured light pattern is projected to the positioning tag, angle information of the infrared ranging sensor and the camera relative to the positioning tag needs to be acquired, so that the infrared ranging sensor is controlled to emit infrared rays to the positioning tag according to the angle information. After the camera receives the reflected infrared rays, the distance between the infrared distance measuring sensor and the positioning label can be calculated, and further the real-time distance between the robot and the positioning label can be determined. The location tag here may be a marker of any shape or structure that is easily captured and recognized by the camera.

In one embodiment, the image acquisition device includes a laser speckle light source and a camera, and the real-time distance between the robot and the positioning tag can be acquired by acquiring a speckle pattern of the laser speckle light source, and specifically includes:

controlling a laser speckle light source of the robot to project a speckle pattern to the positioning label;

controlling a camera of the robot to acquire an image of the positioning tag and acquire a speckle pattern in the image;

and determining the real-time distance between the positioning label and the robot according to the acquired speckle pattern on the surface of the positioning label.

Laser speckle is a diffraction spot that is randomly formed after laser light is irradiated to a rough object or penetrates ground glass. These speckles are highly random and vary in pattern with distance, i.e., the speckle pattern is different at any two locations in space. As long as the laser speckle is printed in the space, the speckle pattern of the whole space can be obtained through light source calibration, and therefore, the position of the positioning label relative to the laser speckle light source can be known only by analyzing the speckle pattern on the surface of the positioning label. In the embodiment, the laser speckle light source projects speckle patterns to the positioning label, and the real-time distance between the positioning label and the robot can be determined by analyzing the speckle patterns in the positioning label images acquired by the camera.

Of course, in other embodiments, the real-time distance between the robot and the positioning tag may be obtained by other means, and is not limited to the above situation, and is not limited herein.

It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

Referring to fig. 10, based on the same inventive concept, in the mobile robot-based sound reinforcement system 10 according to the embodiment of the present invention, the control device 110 includes a sound reinforcement control module 1101, a distance determination module 1102, and a position adjustment module 1103. The sound reinforcement control module 1101 is configured to control the sound reinforcement device of the robot to perform sound reinforcement playing on the voice of the commentator when the first sound collection device acquires the voice of the commentator. The distance determining module 1102 is configured to determine a real-time distance between the robot and the positioning tag according to the positioning tag arranged at the preset position of the commentator. The position adjusting module 1103 is configured to adjust a position of the robot relative to the positioning tag according to the real-time distance and the preset distance.

Referring to fig. 11, further, the public address control module 1101 includes a command word determination unit 301, a public address playing unit 302, and an operation execution control unit 303. The command word determining unit 301 is configured to perform voice recognition on the voice of the commentator, and determine whether the voice contains a command word. The sound amplifying playing unit 302 is configured to control a sound amplifying device of the robot to perform sound amplifying playing on the voice. The operation execution control unit 303 is configured to control the robot to execute the operation corresponding to the command word according to the command word

Further, when the semantic analysis of the voice is required, the operation execution control unit 303 is specifically configured to acquire the voice before and after the command word; performing semantic analysis on the command words and the voices before and after the command words to determine the intention of the commentator; controlling the robot to perform an operation corresponding to the intention based on the intention of the commentator.

Referring to fig. 12, further, before playing the voice of the commentator in a public address, the identity of the commentator needs to be verified, the control device 110 further includes a voiceprint recognition module 1100, and the voiceprint recognition module 1100 is configured to perform voiceprint recognition on the voice to determine whether the user who sends the voice is the identity of the commentator.

Further, the distance determination module 1102 may have a different structure according to the specific structure of the robot.

In one embodiment, the robot is provided with at least three UWB base stations which are positioned in a triangle, the positioning tag is a UWB positioning tag, and the real-time distance between the robot and the positioning tag is obtained through a UWB positioning technology. At this time, the distance determining module 1102 may include a signal transmitting unit and a distance acquiring unit, wherein the signal transmitting unit is configured to control the UWB base station of the robot to transmit a UWB signal to the positioning tag; the distance obtaining unit is used for determining the position information of the robot relative to the positioning tag according to the time of the positioning tag for receiving each UWB signal and the position relation between the UWB base stations, and the position information at least comprises the real-time distance of the robot relative to the positioning tag.

In one embodiment, the robot 11 is provided with an image acquisition device and the distance determination module 1102 comprises an image acquisition unit and a distance acquisition unit. The image acquisition unit is used for controlling an image acquisition device of the robot to acquire the image of the positioning label. The distance acquisition unit is used for determining the real-time distance between the positioning label and the robot according to the acquired image of the positioning label.

Further, the robot 11 is further provided with a second sound collecting device 113 for collecting the voice of the visitor. Alternatively, the second sound collection device 113 is disposed in the hand of the visitor, so as to collect the voice of the visitor.

Of course, in other embodiments, each module of the control device 110 may further include one or more units for implementing corresponding functions, which are not described herein.

Fig. 13 is a schematic diagram of a terminal device according to an embodiment of the present invention. As shown in fig. 13, the terminal device 4 of this embodiment includes: a processor 40, a memory 41 and a computer program 42, such as a mobile robot based sound amplification program, stored in said memory 41 and executable on said processor 40. The processor 40, when executing the computer program 42, implements the steps in the following robot sound amplification method embodiments, such as the steps S21 to S23 shown in fig. 2. Alternatively, the processor 40, when executing the computer program 42, implements the functions of each module/unit in each device embodiment described above, for example, the functions of the modules 1101 to 1103 shown in fig. 10.

Illustratively, the computer program 42 may be partitioned into one or more modules/units that are stored in the memory 41 and executed by the processor 40 to implement the present invention. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program 42 in the terminal device 4.

The terminal device 4 may be a desktop computer, a notebook, a palm computer, a cloud server, or other computing devices. The terminal device may include, but is not limited to, a processor 40, a memory 41. Those skilled in the art will appreciate that fig. 13 is merely an example of a terminal device 4 and does not constitute a limitation of terminal device 4 and may include more or fewer components than shown, or some components may be combined, or different components, e.g., the terminal device may also include input-output devices, network access devices, buses, etc.

The Processor 40 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 41 may be an internal storage unit of the terminal device 4, such as a hard disk or a memory of the terminal device 4. The memory 41 may also be an external storage device of the terminal device 4, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the terminal device 4. Further, the memory 41 may also include both an internal storage unit and an external storage device of the terminal device 4. The memory 41 is used for storing the computer program and other programs and data required by the terminal device. The memory 41 may also be used to temporarily store data that has been output or is to be output.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

In the embodiments provided in the present invention, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.

The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.

Claims

1. A following robot sound amplification method is characterized by comprising the following steps:

2. The follow-up robot sound reinforcement method according to claim 1, wherein in a case where the first sound collection device obtains the voice of the commentator, controlling a sound reinforcement device of the robot to play the sound reinforcement of the voice of the commentator includes:

carrying out voice recognition on the voice of the explicator, and determining whether the voice contains command words;

if the voice does not contain command words, controlling a voice amplification device of the robot to perform voice amplification playing on the voice;

and if the voice contains a command word, controlling the robot to execute the operation corresponding to the command word according to the command word.

3. The follow-up robot vocal fold method according to claim 2, wherein the step of controlling the robot to execute the operation corresponding to the command word according to the command word comprises:

acquiring voices before and after the command word;

performing semantic analysis on the command words and the voices before and after the command words to determine the intention of the commentator;

controlling the robot to perform an operation corresponding to the intention based on the intention of the commentator.

4. The follow-on robot megaphone method according to claim 3, wherein the step of controlling the robot to perform an operation corresponding to the intention based on the intention of the commentator, includes:

if the intention of the explicator is to control the robot to execute the command word, controlling the robot to execute the operation corresponding to the command word according to the command word;

and if the intention of the explicator is not to control the robot to execute the command words, controlling a sound reinforcement device of the robot to perform sound reinforcement playing on the voice.

5. The follow-up robot vocal fold method according to claim 4, wherein the step of controlling the robot to execute the operation corresponding to the command word according to the command word comprises:

if the command words comprise action command words, controlling the robot to execute preset actions corresponding to the action command words according to the action command words;

and/or the presence of a gas in the gas,

and if the command words comprise audio adjusting command words, controlling the sound reinforcement device to execute audio adjusting operation corresponding to the audio adjusting command words according to the audio adjusting command words.

6. The follow-on robot public address method according to claim 1, wherein before the step of playing the public address of the commentator by the public address device of the control robot, the method further comprises:

carrying out voiceprint recognition on the voice, and confirming whether the user sending the voice is the identity of the commentator; and under the condition that the user sending the voice is confirmed to be the identity of the commentator, controlling a sound amplifying device of the robot to perform sound amplifying playing on the voice of the commentator.

7. The follow-up robot public address method according to claim 1, wherein the determining the real-time distance between the robot and the positioning tag according to the positioning tag provided at the preset position of the commentator comprises:

controlling a UWB base station of the robot to send a UWB signal to the positioning tag;

and determining the position information of the robot relative to the positioning tag according to the time of the positioning tag for receiving the UWB signals and the position relation among the UWB base stations, wherein the position information at least comprises the real-time distance of the robot relative to the positioning tag.

8. The follow-up robot public address method according to claim 1, wherein the determining the real-time distance between the robot and the positioning tag according to the positioning tag provided at the preset position of the commentator comprises:

controlling an image acquisition device of the robot to acquire an image of the positioning label;

and determining the real-time distance between the positioning label and the robot according to the acquired image of the positioning label.

9. The follow-up robot sound reinforcement method according to any one of claims 1 to 8, wherein the positioning tag is provided to a first sound collection device carried by the commentator;

or the positioning label is arranged on the work card worn by the commentator;

or, the positioning tag is arranged on the electronic device carried by the commentator.

10. The follow-on robot public address method of any one of claims 1 to 8, characterized in that the method further comprises:

acquiring user voice through a second voice acquisition device, performing voice processing on the user voice, and determining whether the user voice is a question of the user;

and if the user voice is the user question, controlling a sound amplifying device of the robot to perform sound amplifying playing on the user voice.

11. A follow-on robotic sound amplification system, comprising: the robot is provided with a control device and a sound amplification device, and the first sound acquisition device is in communication connection with the control device;

the control device includes:

12. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any of claims 1 to 10 when executing the computer program.

13. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 10.