TWI622474B - Robot system and control method thereof - Google Patents

Abstract

The invention provides a robot system and a control method thereof. The robot system comprises: a microphone array, receiving a voice signal sent by a user; a binocular ranging lens, synchronously capturing two channels of image signals; and processing a module, The voice signal and the image signal are processed to obtain a voice recognition signal and a rotation angle signal; the motor rotation module receives the rotation angle signal and performs smooth rotation according to the rotation angle signal. The robot system, through the human voice, the most natural human-machine dialogue mode, liberates people's hands in the home environment, and makes the robot system a high-quality companion.

Description

Robot system and control method thereof

The present invention relates to the field of intelligent robot technology, and in particular, to a robot system and a control method thereof.

With the development of artificial intelligence and computer software and hardware technology, the development of robot technology has experienced a development process from low to high. The first generation of robots is equipped with a memory memory, which is used by a person to demonstrate various requirements of the job to the robot, so that it remembers the program and the method of operation, and when it receives the command of reproduction, it autonomously imitates the demonstration action. The second generation of robots is a discretely programmed industrial robot equipped with small computers and sensors that senses external signals and "thinks". It is more flexible and adaptable to the needs of the environment than the first generation of robots. The third generation robot is an intelligent robot. It not only has the sensory function and simple adaptive ability of the second generation robot, but also can fully recognize the working object and working environment, and can automatically determine according to the instructions given by the person and its own judgment results. The work that is compatible with it is a high-level product of the development of artificial intelligence, and it is also a hot spot in the development of robots today.

The existing robot system is more used in industrial scenes. The control of industrial robots has problems such as stylization, high programming difficulty, and unfriendly operation interface. In the industrial scene, the shape of the robot is also limited to the mechanical equipment such as the robot arm. Application, it is difficult to exchange with people better. At the same time, existing robot systems and control technologies are difficult to popularize and have high development costs. In addition, the existing robot system and control technology are difficult to apply in the home scene, and cannot be friendly exchanged with family members in the home scene, and it is more difficult to provide high-quality and convenient services for family members.

The present application provides a robot system, including: a microphone array that receives a voice signal from a user; a binocular ranging lens that simultaneously captures two channels of image signals; and a processing module that respectively The microphone array is connected to the binocular distance measuring lens to receive and process the voice signal and the image signal to generate a voice recognition signal and a rotation angle signal; the motor rotation module is connected to the processing module to receive And smoothly rotating according to the rotation angle signal.

In a preferred embodiment, the processing module includes: a DSP acceleration auxiliary processor, which is respectively connected to the microphone array and the binocular ranging lens for receiving the voice signal and the image. a signal, performing voice enhancement processing on the voice signal, processing the image signal to obtain a ranging signal; and the robot main controller is connected to the DSP acceleration auxiliary processor through a USB interface, and receiving the voice enhanced processing Decoding the voice signal and the ranging signal, and performing automatic gain processing on the voice signal according to the ranging signal to obtain the voice recognition signal; and processing the multi-core ARM according to the robot main controller Device.

In a preferred embodiment, the voice signal is positioned to obtain the rotation angle signal.

In a preferred embodiment, the robot system further includes: a cloud speech recognition engine, which receives the speech recognition signal and exchanges with the robot main controller, and returns a result of automatic speech recognition and natural semantic processing to the user.

In a preferred embodiment, the robot system further includes: a DLP internal projection display module located inside the head of the robot system, and the exchange user interface of the robot is projected on the face of the robot.

In a preferred embodiment, the robot main controller is connected to the DLP internal display mode through an I2C serial control protocol and a HDIM interface.

In a preferred embodiment, the microphone array is a multi-channel microphone array.

The present invention also provides a robot control method suitable for the robot system, comprising the steps of: S1: receiving the voice signal and the image signal, and transmitting to the DSP acceleration auxiliary processor; S2: The DSP acceleration auxiliary processor processes the voice signal and the image signal and transmits the voice signal to the robot main controller to obtain a voice recognition signal and a rotation angle signal; S3: the cloud voice recognition engine receives the voice recognition The signal is sent back to the household for the result of automatic speech recognition and natural semantic processing; S4: the rotation control module receives the rotation angle signal and performs smooth rotation according to the rotation angle signal.

In a preferred embodiment, the step S1 includes the following steps: S11: the multi-channel microphone array synchronously captures the voice signals in different directions and transmits the voice signals to the DSP acceleration auxiliary processor; S12: the eyes The ranging photographic lens synchronously captures the image signals of the two channels and transmits them to the DSP acceleration auxiliary processor.

In a preferred embodiment, the step S2 includes the following steps: S21: the DSP acceleration auxiliary processor performs beamforming processing on the voice signal, and performs voice enhancement on the mixed noise and echo signals in the voice signal. Processing; S22: the DSP acceleration auxiliary processor performs hardware parallel acceleration on the binocular ranging visual algorithm, and processes the image signal to obtain the ranging signal; S23: the robot main controller is configured according to The ranging signal performs automatic gain processing on the voice signal; S24: the robot main controller receives the voice signal to acquire the voice recognition signal, and locates the voice signal to obtain the rotation angle signal.

In summary, due to the adoption of the above technical solution, the present patent application describes a robot system and a control method thereof, which have the beneficial effects of: the robot system, through the human voice, the most natural human-machine dialogue mode, at home In the environment, the hands of the people are liberated, and the robot system becomes a high-quality companion; the binocular distance measuring lens automatically recognizes and measures the home user, and realizes the automatic gain of the voice signal according to the ranging signal. The intelligentization of the control makes it easy for the user to obtain a high-accuracy speech recognition effect by using the natural speech volume within different distances in the home scene; and positioning the speech signal and implementing the motor rotation module Smooth rotation, the multi-channel microphone array and the binocular ranging lens are automatically aligned with the target, achieving mask noise and improving the voice pickup signal-to-noise ratio; and the DLP internal projection display module solves the exchange use The problem of unfriendly interface is to provide users with a more convenient and future-proof display experience.

The specific embodiments of the present invention are further described below in conjunction with the accompanying drawings.

Embodiment 1

As shown in FIG. 1 , the present invention provides a robot system, comprising: a microphone array, synchronously capturing multi-channel microphone voice signals, and transmitting the captured voice signals through the I2S audio interface in real time. To the DSP acceleration auxiliary processor 3; the binocular ranging lens 2, synchronously captures two channels of image signals, and transmits the captured image signals to the DSP acceleration auxiliary processor 3 through the USB interface in real time; DSP Accelerating the auxiliary processor 3, receiving the voice signal and the image signal, accelerating the speech enhancement algorithm and the binocular ranging visual algorithm, respectively performing voice enhancement processing on the voice signal and the image signal, and The number visual processing acquires the ranging signal, and transmits the processing result to the robot main controller 4; the robot main controller 4 is connected to the DSP acceleration auxiliary processor 3 through the USB interface, and receives the processed voice signal and the Deriving a distance measurement signal, performing automatic gain control on the voice signal according to the ranging signal, and acquiring a rotation angle according to positioning the voice signal No., realizes scheduling and controlling the robot system; the cloud speech recognition engine 5 exchanges with the robot main controller 4, and feeds back to the user the result of accurate and personalized automatic speech recognition and natural semantic processing to realize far field Automatic speech recognition for the Hands-free product experience.

The motor rotation module 6 is seamlessly connected with the rotating structure of the robot system, and performs smooth rotation according to the rotation angle signal sent by the robot main controller 4; the DLP internal projection display module 7 is located at the robot head. Internally, the high-definition user interface of the robot is projected on the face of the robot, which brings the user a sense of technology in the future conversation mode.

The microphone array is a multi-channel microphone array 1 capable of synchronously capturing voice signals in different directions. The DSP acceleration auxiliary processor 3 adopts a T1 Texas Instruments high performance floating point arithmetic digital signal processor platform, and the robot main controller 4 adopts a multi-core ARM processor. The DSP acceleration auxiliary processor 3 receives the image signal, and identifies a target user in the image signal, and acquires a distance that the target uses for the robot, that is, the ranging signal.

The robot main controller 4 receives the voice signal of the high signal-to-noise ratio enhanced by the DSP acceleration auxiliary processor 3, implements voice initiation, voice homing, local automatic voice recognition, and acquires a voice recognition signal, and The cloud speech recognition engine 5 exchanges and feeds the user with accurate and personalized automatic speech recognition and natural semantic processing results. Thus, the Hands-free product experience of far-field automatic voice recognition is realized. In addition, the robot main controller 4 receives and updates the ranging signal extracted by the DSP acceleration auxiliary processor 3 after being processed by the binocular visual depth algorithm, and automatically performs the voice signal according to the ranging signal. Gain control, so that the user can quickly start the robot with natural speaking volume at different distances under the remote conditions of the home scene, and perform automatic speech recognition exchange.

The robot main controller 4 is connected to the DLP internal projection display module 7 through an I2C serial control protocol and an HDMI interface to implement a robot expression, and the said image is extracted from the DSP acceleration auxiliary processor 3 After the ranging signal processing, the rotation angle signal is obtained, and the motor rotation module 6 rotates according to the rotation angle signal, and the multi-channel microphone array 1 is aligned with the speaking user, and the far field environment is suppressed by beam forming. Noise and echo in the voice, improve the signal-to-noise ratio of the voice signal, thereby improving the accuracy of voice recognition. At the same time, the voice homing rotation can maximize the user interface of the robot interface, thereby further increasing the interest of the robot system to exchange with the user.

The robot system, through the human voice, the most natural human-machine dialogue mode, liberates people's hands in the home environment, and makes the robot system a high-quality companion. The binocular distance measuring lens 2 automatically recognizes and measures the home user, and realizes the intelligentization of the voice signal automatic gain control according to the ranging signal, so that the user can adopt the natural in different distances in the home scene. The speaking volume enables the robot system to acquire a high-accuracy speech recognition effect. And positioning the voice signal and the motor rotation module 6 to achieve smooth rotation, so that the multi-channel microphone array 1 and the binocular distance measuring lens 2 are automatically aligned with the target, thereby achieving mask noise and improving voice pickup. Signal to noise ratio. Moreover, the DLP internal display module 7 solves the problem that the user interface is unfriendly, and provides the user with a more convenient and more future display experience.

Embodiment 2

According to a robot system according to the first embodiment, the present embodiment provides a robot control method based on the system, as shown in FIG. 2, which specifically includes the steps of: S1: taking a voice signal and an image signal, and transmitting the two signals. Accelerating the auxiliary processor 3 to the DSP; S2: the DSP acceleration auxiliary processor 3 processes the voice signal and the image signal and transmits the signal to the robot main controller 4 to acquire a voice recognition signal and rotate An angle signal; S3: the cloud speech recognition engine 5 receives the speech recognition signal, and returns a result of automatic speech recognition and natural semantic processing to the user; S4: the motor rotation module 6 receives the rotation angle signal, and Smooth rotation is performed according to the rotation angle signal.

The step S1 shown in FIG. 3 includes: S11: the multi-channel microphone array 1 synchronously captures voice signals in different directions and transmits the voice signals to the DSP acceleration auxiliary processor 3; S12: the binocular distance measuring lens 2 synchronously capturing the image signals of the two channels and transmitting them to the DSP acceleration auxiliary processor 3.

As shown in FIG. 4, the step S2 includes: S21: the DSP acceleration auxiliary processor 3 performs beamforming processing on the voice signal, and performs voice enhancement processing on the mixed noise and echo signals in the voice signal; S22: The DSP acceleration auxiliary processor 3 performs hardware parallel acceleration on the binocular ranging visual algorithm, processes the image signal to obtain a ranging signal, and S23: the robot main controller 4 according to the ranging The signal is subjected to automatic gain processing on the voice signal; S24: the robot main controller 4 receives the voice signal to acquire the voice recognition signal, and locates the voice signal to obtain the rotation angle signal.

The robot control method acquires the voice signal and the image signal through the multi-channel microphone array 1 and the binocular ranging lens 2, and processes the image signal to obtain identification and ranging. a signal, wherein the voice signal performs corresponding automatic gain processing according to the identification and ranging signals, so that the user can obtain the high-accuracy speech recognition by using the natural speaking volume within different distances in the home scene. effect. Positioning the voice signal, the motor rotation module 6 performs smooth rotation according to the positioning result, so that the multi-channel microphone array 1 and the binocular distance measuring lens 2 are aligned with the target, thereby achieving mask noise, Improve the effect of voice pickup signal to noise ratio.

The above is only a preferred embodiment of the present invention, and is not intended to limit the scope of the embodiments and the scope of the present invention. Those skilled in the art should be able to Combinations of equivalent substitutions and obvious variations are intended to be included within the scope of the invention.

1‧‧‧Multichannel Microphone Array
2‧‧‧Binocular distance measuring lens
3‧‧‧DSP Acceleration Auxiliary Processor
4‧‧‧Robot master controller
5‧‧‧Cloud speech recognition engine
6‧‧‧Motor rotation module
7‧‧‧DLP internal projection display module
S1-S‧‧ steps

1 is a schematic structural view of a robot system according to the present invention; FIG. 2 is a flow chart 1 of a robot control method according to the present invention; FIG. 3 is a flow chart 2 of a robot control method according to the present invention; Flow chart three.

Claims (8)

A robot system comprising: a microphone array for receiving a voice signal sent by a user; a binocular ranging lens for simultaneously capturing two channels of image signals; and a processing module respectively for the microphone array and the binocular ranging The photographic lens is connected to receive and process the voice signal and the image signal to generate a voice recognition signal and a rotation angle signal; the processing module includes: a DSP acceleration auxiliary processor, and the microphone array and the pair respectively The eye-ranging photographic lens is connected to receive the voice signal and the image signal, perform voice enhancement processing on the voice signal, identify a target user in the image signal, and identify the image The signal is processed to obtain the distance between the target user and the robot system as a ranging signal, and the robot main controller is connected to the DSP acceleration auxiliary processor via a USB interface to receive the voice signal processed by the voice enhancement and the a ranging signal, and performing automatic gain processing on the voice signal according to the ranging signal to obtain the voice recognition signal And positioning the voice signal to obtain the rotation angle signal; and a motor rotation module connected to the processing module, receiving and smoothly rotating according to the rotation angle signal, and the microphone array and The binocular distance measuring lens is aimed at the speaking target. The robot system of claim 1, wherein the robot main controller is a multi-core ARM processor. The robot system of claim 1, wherein the robot system further comprises: The cloud speech recognition engine is connected to the robot main controller, receives the speech recognition signal and exchanges with the robot main controller, and returns the result of automatic speech recognition and natural semantic processing to the user. The robot system of claim 2, wherein the robot system further comprises: a DLP internal projection display module, connected to the robot main controller, located inside the robot system head, and the robot The exchange user interface is projected on the face of the robot. The robot system of claim 4, wherein the robot main controller is connected to the DLP internal projection display module through an I2C serial control protocol and an HDMI interface. The robot system of claim 1, wherein the microphone array is a multi-channel microphone array. A robot control method for the robot system according to any one of claims 1 to 6, comprising the steps of: S1: receiving a voice signal and an image signal, and transmitting the signal to a DSP acceleration auxiliary processor; S2: DSP acceleration assistance The processor processes the voice signal and the image signal and transmits the signal to the main controller of the robot to obtain a voice recognition signal and a rotation angle signal. The step S2 includes the following steps: S21: the DSP accelerates the auxiliary processor Performing beamforming processing on the voice signal, and performing voice enhancement processing on the mixed noise and echo signals in the voice signal; S22: the DSP acceleration auxiliary processor performs hardware parallel acceleration on the binocular ranging visual algorithm, The image signal is processed to obtain a ranging signal; S23: the robot main controller performs automatic gain processing on the voice signal according to the ranging signal; S24: the robot main controller receives the voice signal to obtain the voice recognition signal, and locates the voice signal to obtain the rotation angle signal; S3: the cloud voice recognition engine receives the voice recognition signal, and sends the voice recognition signal to the household The result of the automatic speech recognition and the natural semantic processing is fed back; S4: the rotation control module receives the rotation angle signal, and performs smooth rotation according to the rotation angle signal. The robot control method according to claim 7, wherein the step S1 includes the following steps: S11: the multi-channel microphone array synchronously captures the voice signals in different directions and transmits the voice signals to the DSP acceleration auxiliary processor; S12 The binocular ranging photographic lens synchronously captures the image signals of the two channels and transmits the image signals to the DSP acceleration auxiliary processor.