KR100901483B1 - Method and system for network-based speech processing in a robot - Google Patents

Abstract

The present invention relates to a voice processing technology for a network-based robot, and receives sound through a microphone and a sound board to a robot provided with a plurality of different microphones and sound boards, and frames sound data input for each microphone and sound board. It stores in the buffer and assigns ID to each frame, and temporarily stores sound data with the same contents for each ID on the stack of the same location and synchronizes them, and drives voice related applications of the robot with the synchronized sound data. It is done. According to the present invention, it is possible to remotely control a number of robots with a central computer, and these network-based robots can be widely used because they can lower the price of the robots themselves, and integrate the voice-related applications of the robots through synchronization. Since it can be performed by, it can increase the voice recognition performance of the robot.

Robot, Speech Processing, Synchronization

Description

Network-based voice processing method and system for robots {METHOD AND SYSTEM FOR NETWORK-BASED SPEECH PROCESSING IN A ROBOT}

The present invention relates to a voice processing technology for a robot, and in particular, a network-based control of a plurality of robots (clients) by one central computer (server), and voice-related application programs through synchronization of data input to the robot. The present invention relates to a network-based robot voice processing method and system suitable for combining or executing voice and video programs.

The present invention is derived from research conducted as part of the development and standardization of embedded component technology for the URC of the Ministry of Information and Communication [Task Management Number: 2005-S-033-03, Task name: Development of embedded component technology for URC and standardization].

Recently, the commercialization of intelligent service robots used for general households is increasing. Intelligent service robots for home use are equipped with various external sensors such as cameras and multi-array microphones, and act intelligently by recognizing the situation and interacting with humans according to the output of these external sensors. .

1 is a block diagram showing the structure of a general robot.

Referring to FIG. 1, the robot includes a peripheral device 100, a control unit 120, a driver module 140, and a data storage unit 150. The peripheral device 100 is a device for implementing various functions of the robot. The peripheral device 100 is installed on the body of the robot to photograph the surroundings, and outputs images and image data. A multi-array microphone 104 for recognizing a human voice, a speaker 106 for outputting a sound for a response, processing, or various events according to a control command, and a touch sensor 108 for receiving and transmitting a sensed touch signal. And a light emitting diode (hereinafter referred to as LED) 110 for displaying various processing conditions and response signals in the robot.

The robot controller 120 controls the peripheral devices 100 connected through the driver 140 and transmits and stores signals and data input from the peripheral device 100 to the data storage unit 150, and stores all processes in the robot. Process. Since the data storage unit 150 has an area in which the robot driving program is stored and an area in which various data to be downloaded are stored, the data storage unit 150 stores signals and data transmitted from the robot control unit 120 in a preset area.

The driver 140 is an interface for connecting the peripheral devices 100 to the robot controller 120. The driver 140 connects the speaker 106 and the multi-array microphone 104 to perform encoding and connects the camera to the camera. After the encoding is performed, the camera driver 144 for transmitting the encoded image and image data to the robot controller 120 and the motor for moving, jointing, and moving the robot are controlled to control the overall driving. Motor drivers and the like.

A number of patents have already been applied for such an intelligent service robot. In particular, Korean Patent Application No. 10-2004-0051548 discloses a conversation control means, an image recognition means, and a tracking control means for tracking the presence of a conversation partner with respect to a voice conversation apparatus having a function for conversation. 0301070 shows a control method for recognizing a voice signal of a user in a robot toy. US Patent No. 7065490 also describes a speech processing method for controlling emotion.

Patents regarding various voice processing methods and robot control methods have been filed as described above. However, research and patents on synchronization, which are processing methods for various types of microphones mounted on various sound cards (or sound boards) mounted on a robot, have been applied. The situation is insufficient.

In the conventional robot and the voice processing system of the robot operating as described above, it is necessary to synchronize the overlapping signals among the signals received from the multiple array microphones, but there is no alternative. Since the state is stored in the robot itself, there is a problem that the user must control the robot using an input unit such as a touch sensor or a command in a preset manner in order to control the robot.

Accordingly, the present invention provides a network-based robot voice processing method that can control a plurality of robots (clients) as a central computer (server) in a wired and wireless network, and can be performed by integrating robot voice related applications. Provide a system.

The present invention also provides a network-based robot voice processing method and system for controlling a plurality of robots through a central computer in a wired / wireless network environment and synchronizing voice related applications of robots with various types of microphones and soundboards. to provide.

According to one embodiment of the present invention, a method of processing a voice in a robot equipped with a plurality of microphones and a sound board includes: receiving a sound through the plurality of microphones and the sound board, and inputting the input sound data into a frame buffer. Storing and allocating IDs for each frame, synchronizing temporarily storing sound data having the same contents for each ID in a stack at the same location, and synchronizing the voice-related applications of the robot with the synchronized sound data. Driving process.

In one embodiment of the present invention, in a voice processing system for a robot, a plurality of peripheral devices, a robot control unit connected to the plurality of peripheral devices to control the robot, and a program for controlling the robot connected to the robot control unit A voice related application, a storage unit for storing data transmitted from the plurality of peripheral devices, and the robot are connected to the wired / wireless network so that the robot controller can receive and process data that cannot be processed by itself or that is heavily loaded. It includes a server.

According to another embodiment of the present invention, in a voice processing system for a robot, a plurality of peripheral devices, a robot control unit connected to the plurality of peripheral devices to control the robot, and a program for controlling the robot connected to the robot control unit And a storage unit for storing data transmitted from the voice-related application program, the plurality of peripheral devices, and the robot and the wired / wireless network so that the robot controller receives and processes data that cannot be processed by itself or takes a lot of load. A central computer server, and a virtual robot control unit for converting the control command input from the central computer server to the control command method of the robot control unit, and then transfers the converted control command to the robot control unit.

In the present invention, the effects obtained by the representative ones of the disclosed inventions will be briefly described as follows.

The present invention, by implementing a network-based voice processing system for the robot, it is possible to remotely control a number of robots with a central computer, the network-based robot can be widely used because it can lower the price of the robot itself There is this.

In addition, since the voice-related application programs of the robot can be integrated and executed through synchronization in a wireless network environment, there is an effect of improving the voice recognition performance of the robot.

Hereinafter, the operating principle of the present invention will be described in detail with reference to the accompanying drawings. In the following description of the present invention, when it is determined that a detailed description of a known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. Terms to be described later are terms defined in consideration of functions in the present invention, and may be changed according to intentions or customs of users or operators. Therefore, the definition should be made based on the contents throughout the specification.

The present invention enables to control a number of robots (clients) as one central computer (server) on a wired or wireless network, and to perform the integration of voice-related application programs for the robot.

In the network-based service robot, there is no need to passively deal with the input voice as in the conventional method. That is, the robot moves in the direction of the input voice through the sound source tracking method to amplify the voice and receives the input, or performs sound source tracking only for a specific sound source, for example, a robot name such as "webber", or the owner of a specific speaker robot. An active speech processing method may be used, such as extracting only the speech of speech and performing speech recognition.

This operation is performed several voice processing for the same voice, each voice processing can be performed in a parallel or sequential manner, the synchronization of the voice signal may be the biggest issue, through the synchronization of the voice signal The processing performance for speech recognition can be improved.

In addition, audio and video may need to be solved for the same purpose. For example, if the sound source is tracked using voice but there are various people in the location of the sound source, in order to approach the talker, other functions besides the sound source, such as speaker recognition of the talker's voice, are used to find out the talker. Face recognition can help you reach a more accurate speaker.

As described above, the robot of the present invention moves away from the passive voice / video related components and moves more actively, and can interact with a person even when the robot controller is provided with a simple processing processor through data processing through a server. It is possible to implement basic system and to improve the processing performance of control commands through synchronization of application programs.

2 is a block diagram illustrating a robot structure of a network-based server-client type according to a preferred embodiment of the present invention.

2, the robot includes a peripheral device 200, a robot controller 230, a driver 240, and a data storage 250. The server 220, which is a central computer, transmits and receives command signals and data with a robot based on a wired or wireless network, and controls the robot.

The peripheral device 200 of the robot is a device for imparting various functions to the robot and performing operations according to the robot, and includes a camera 202, a multiple array microphone 204, a speaker 206, a touch sensor 208, and an LED 210. ), And transmits the signals and sensor data received from each device to the robot controller 230. The driver 240 is an interface for connecting the peripheral devices 200 to the robot controller 230, and includes a sound driver 242, a camera driver 244, a motor driver 246, and the like.

The robot controller 230 controls the peripheral devices 200 connected through the driver 240 and transmits and stores signals and data input from the peripheral device 200 to the data storage unit 250, and stores all processes in the robot. Process. Since the data storage unit 250 has an area in which the robot driving program is stored and an area in which various data to be downloaded are stored, the data storage unit 250 stores signals and data transmitted from the robot control unit 230 in a preset area. In addition, the application programs for the signals and data input from the peripherals 200 are executed, and in particular, the voice-related applications are driven by synchronizing the voice data when processing the input sound and voice data.

The server 220 receives the application program data previously processed by the robot controller 230 and processes the data on the server 200, thereby reducing the computing power of the robot itself, which is an advantage of the network-based robot, and using a low-end processor as a robot. Enables driving of

The robot controller 230 transmits the information data received from each sensor of the peripheral device 200 to the server 220, and the server 220 drives and processes an application program on the received information data, and processes the result of the information processing. By transmitting back to the robot controller 230, the robot is controlled. However, depending on the application, there may be data that must be processed by the robot itself without being sent to the server 220 which is a central computer. For example, voice recognition, which requires a lot of computing power, is handled by the server 220, and by using a multichannel sound source, sound source tracking, an application that requires a lot of network load and must always be executed, is processed by the robot itself. It may be desirable to do so.

On the other hand, as mentioned above, the application programs for voice recognition and sound source tracking can be synergistic to be implemented with data synchronized with each other, and the role of the robot controller 230 is important for such data synchronization. The robot controller 230 should have a structure for synchronizing data processed by the server 220 and data processed by the robot itself, and the structure for synchronization will be described in detail with reference to FIG. 4.

As described above, the conventional robot shown in FIG. 1 processes all processes in the robot itself, whereas the robot shown in FIG. 2 has a central computer as the server 220 so that the server 220 directly controls the robot. It is implemented by performing information processing.

However, the structure of the robot as shown in FIG. 2 requires the server 220 to know the control system of the robot controller 230. That is, since the robot controller 230 of each robot can be designed differently in a programmed language format, available control commands and control procedures, the server 220 knows the characteristics of each robot in order to control specific robots. Disadvantage

3 is a block diagram illustrating a robot structure of a network-based integrated service type according to a preferred embodiment of the present invention.

First, FIG. 3 has a structure similar to that of FIG. 2. That is, the robots of FIG. 3 are devices for providing various functions to the robots and performing operations according to the robot, and include a camera 302, a nondirectional omnidirectional multiple array microphone 304, a speaker 308, a touch sensor ( 310, a peripheral device 300 including an LED 312, and a sound driver 362, a camera driver 364, and a motor driver 366 as an interface for connecting the peripheral devices 300 to the robot controller 350. Control the driver 360, and the peripheral device 300 connected through the driver 360, and transmits and stores the signals and data input from the peripheral device 300 to the data storage unit 370, the robot And a data controller 370 having a robot controller 350 for processing all processes therein and an area in which a robot driving program is stored, and an area in which various data to be downloaded are stored.

In addition to the peripheral 300, in addition to the omnidirectional multiple array microphone 304, a directional microphone 306 having a directivity in a specific direction, so that the voice using the omnidirectional multiple array microphone 304 and the directional microphone 306 To improve the performance of the recognition and sound source tracking, speaker recognition using the image application and the voice recognition program, the server 330 using the home computer as well as the server 320 may be implemented. In this case, the home computer server 330 is for controlling robots located in a limited space in connection with the server 320 which is a central computer, and data that cannot be processed or loads heavily in the robot itself from the virtual robot controller 340. Receives the data and performs the data processing for this, and performs the control signal and data transmission and reception between each server (320, 330).

That is, the central computer server 320 controls robots located in a plurality of spaces, and for this purpose, the home computer server 330 can be used as an access point, and the home computer server 330 is generally a home computer. A computer that can be used in (limited space and area) is a central computer server that performs information processing and robot control by connecting robots used in the home by wire or wireless, and transmits and receives data to and from each other in connection with the home computer server 330 ( 320 may control each robot through each home computer server 330 or directly connect with the virtual robot controller 340 of each robot to perform information processing and robot control.

In addition, the robot of FIG. 3 controls a robot controller 350 having different characteristics, and a virtual robot controller 340 for transmitting and receiving data is added. The virtual robot controller 340 is a standard control scheme of the robot, and each robot includes a control scheme that is designed differently because the programming language format of the robot controller, available control commands, and control procedures may be designed differently. The virtual robot controller linked with the robot controller converts a command input in a control method suitable for the robot controller and transmits the command to the robot controller.

For example, there are A, B, and C robots in the home, and the control commands of the virtual robot controllers embedded in the A, B, and C robots are all performed in a method, and the robot controllers embedded in the A, B, and C robots are Since it is designed with the control method of a-1, b-1, and c-1, respectively, if only the control commands of a-1, b-1, and c-1 are received, The home computer server simultaneously transmits control methods of a method to the A, B, and C robots.

Accordingly, the virtual robot controller embedded in the robot A converts the control method of the input method a to the control command of the method a-1, which is the control method of the robot controller, and transmits the control command to the robot controller. The robot controller of the robot A receives the received a- The robot is driven by one type of control command. The same applies to the B and C robots, and the virtual robot controller built in the B and C robots converts the control method of the a method into the control commands of the b-1 and c-1 methods, which are the control methods of the respective robot controllers. By transferring to the robot control unit, the robot control unit of each robot drives the robot with the received b-1 and c-1 control commands.

As described above, the robot structure of FIG. 3 is a common control method regardless of the feature, form, and structure of the robot through the common virtual robot control unit 340 provided with the central computer server 320 or the home computer server 330 for each robot. By exchanging information with the virtual robot controller 340 with a control command according to the above, application programs in the central computer server 320 can control any robot using the robot control method defined in the virtual robot controller 340. Will be.

To this end, the virtual robot controller 340 uses an application programming interface (API) for communicating with the central computer server 320, an API for communicating with individual robots, and these two APIs. By connecting, synchronization is performed.

4 is a diagram illustrating an API form between a robot and a server according to a preferred embodiment of the present invention.

The API structure 400 of FIG. 4 is included in the robot controller 230 of FIG. 2 and the robot controller 350 of FIG. 3, and the robot (API) 405 and the robot that exchange information with a central computer. An API Sync Layer (hereinafter referred to as an ASL) 410 is provided between an API (Robot API Presentation Layer) 425 that exchanges information with peripheral devices and drivers including various sensors of the PC. Implement to do

Here, the ASL 410 includes a data decoder 415 and a data encoder 420, and the data synchronized through the data decoder 415 and the data encoder 420 may be transferred to the Robot API Layer 405 or the Robot API Presentation Layer ( 425).

The method of performing synchronization is described. Data coming from different microphones and different soundboards is stored in a common class type. This type is called a WaveFrameBuffer and the Wave Frame Buffer is given an ID for each frame, so that data from different microphones and different soundboards are commonly stored in the Wave Frame Buffer and stored temporarily on the same stack for each ID. Be sure to That is, the data stored in the 10th stack of ID 1 and the data of the 10th stack of ID 3 differ only in the microphone or sound board and contain the same sound data or utterance. At this time, the sound frames are allocated along with the count ID per frame to ensure continuity of the front and rear frames.

On the other hand, in network-based robots, voice-related applications can use each other's programs to produce better results. For example, speech recognition or speaker recognition may be more effective using a directional microphone, but if the talker is speaking without the directional microphone pointing, the performance from the directional microphone may be worse.

At this time, through the result of sound source tracking using omnidirectional multiple array microphones, the direction of the talker is matched with the directional microphone or the voice data obtained from the talker's direction is transmitted when the application program for speech recognition or speaker recognition is driven. You can get better results. Accordingly, the data spoken through the ASL 410 as described above may be synchronized with voice-related application programs.

5 is a flowchart illustrating a synchronization procedure of voice data according to a preferred embodiment of the present invention.

Referring to FIG. 5, one robot may be provided with different microphones and sound boards, and a plurality of microphones may be installed in each direction of the robot. In step 500, the sound is input through different microphones, and in step 502, the robot controller in the robot stores the input sound data in a wave frame buffer of a common class type, and inputs the sound data input for each microphone or sound board. Assign an ID.

Here, ID is a preset ID for each microphone or sound board, and sound data input to each microphone or sound board is assigned an ID of the input microphone or sound board.

Thereafter, in step 504, the same sound data is synchronized to stack on the same position for each ID. When driving a sound source tracking application performed through the control unit of the robot in step 506, the directional omnidirectional multi-array microphone is used. The received and synchronized sound data tracks the direction of the sound source. In addition, in the case of the sound data processing method using the server in step 506, the synchronized data is transmitted to the server through the virtual robot controller, and the voice recognition application is driven through the sound data synchronized by the server to perform voice recognition. Done.

Through this process, multiple voice processing is performed for the same voice, and each microphone and sound board or direction in which the microphones are installed at the time of voice recognition and sound source tracking through parallel synchronization for the same voice. The performance of voice recognition and sound source tracking can be improved through the sample data, and it is possible to distinguish additional voices in addition to the same voice and process them in a sequential manner.

As described above, the present invention moves more actively in a manner in which the robot passively processes audio / video related components and interacts with a person even when only a simple processing processor is provided in the robot controller through data processing through a server. A basic system that can operate can be implemented, and synchronization of application programs can improve the processing performance of control commands.

Meanwhile, in the detailed description of the present invention, specific embodiments have been described, but various modifications are possible without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined not only by the scope of the following claims, but also by those equivalent to the scope of the claims.

1 is a block diagram showing the structure of a general robot;

2 is a block diagram showing a network structure of a network-based server-client type according to a preferred embodiment of the present invention;

3 is a block diagram showing a robot structure of a network-based integrated service type according to an embodiment of the present invention;

4 is a diagram illustrating an API form between a robot and a server according to a preferred embodiment of the present invention;

5 is a flowchart illustrating a synchronization procedure of voice data according to a preferred embodiment of the present invention.

Claims (19)

In the voice processing method for a robot provided with a plurality of microphones and a sound board, Receiving sound through the plurality of microphones and soundboards; Storing the input sound data in a frame buffer and assigning an ID to each frame; A synchronization process of temporarily storing sound data having the same contents for each of the IDs on a stack of the same location; Driving voice-related applications of the robot using the synchronized sound data Voice processing method for a robot comprising a. The method of claim 1, The microphone, A voice processing method for a robot, characterized in that it is a directional microphone or an omnidirectional multiple array microphone. The method of claim 1, The ID is, The sound processing method for the robot, characterized in that the sound data is assigned to each input microphone and soundboard. The method of claim 1, The voice related application, A voice processing method for a robot comprising a voice recognition, sound source tracking and speaker recognition program. In the voice processing system for a robot, Many peripherals, It is connected to the plurality of peripheral devices and drivers to control the robot, allocates IDs per frame to sound data input from a microphone and a soundboard, and stores them in a frame buffer, and stores sound data having the same content for each ID. A robot controller which performs synchronization for temporarily storing the stack of positions and drives a robot related application program with the synchronized sound data; A storage unit connected to the robot control unit to store a program for controlling the robot, the voice related application program, and data transmitted from the plurality of peripheral devices; The server is connected to the robot and the wired and wireless network to receive and process data that cannot be processed by the robot controller itself or takes a lot of load. Speech processing system for a network-based robot comprising a. The method of claim 5, The server, When the voice data for voice recognition is received from the robot controller, Speech processing system for a network-based robot, characterized in that for performing voice recognition by driving the speech-related application program. The method of claim 5, The robot control unit, When receiving voice data for sound source tracking from the peripheral device, The voice processing system for a network-based robot, characterized in that for tracking the direction of the sound source by driving the voice-related application program. delete The method according to any one of claims 6 to 7, The voice related application, Voice processing system for a network-based robot, characterized in that it comprises a voice recognition, sound source tracking and speaker recognition program. The method of claim 5, The peripheral device, Voice processing system for a network-based robot, comprising at least one of a camera means, a microphone, a speaker, a touch sensor, a display. The method of claim 5, The driver, And providing an interface for driving the peripheral device with the robot controller, and receiving signals and sensor data from the peripheral device and transmitting the signal to the robot controller. In the voice processing system for a robot, Many peripherals, A robot controller connected to the plurality of peripheral devices and a driver to control the robot; A storage unit connected to the robot control unit to store a program for controlling the robot, an application program related to voice, and data transmitted from the plurality of peripheral devices; A central computer server connected to the robot via a wired / wireless network to receive and process data that cannot be processed by the robot controller itself or takes a lot of load; The virtual robot controller converts the control command input from the central computer server into the control command method of the robot controller and then transfers the converted control command to the robot controller. Speech processing system for a network-based robot comprising a. The method of claim 12, The system, And a home computer server connected to the central computer server and the virtual robot controller through a wired / wireless network to transmit and receive data and control signals with the central computer server and to process data transmitted from the virtual robot controller. Based voice processing system for robots. The method of claim 12, The server, When the voice data for voice recognition is received from the robot controller, Speech processing system for a network-based robot, characterized in that for performing voice recognition by driving the speech-related application program. The method of claim 12, The robot control unit, When receiving voice data for sound source tracking from the peripheral device, The voice processing system for a network-based robot, characterized in that for tracking the direction of the sound source by driving the voice-related application program. The method of claim 12, The robot control unit, Receives sound data from a plurality of microphones and soundboards, stores sound data input for each microphone and soundboard in a frame buffer, assigns IDs for each frame, and assigns sound data having the same contents for each ID to the same location. Performing a synchronization to temporarily store in a stack of, and driving the voice-related applications of the robot with the synchronized sound data. The method according to any one of claims 14 to 16, The voice related application, Speech processing system for a network-based robot, characterized in that it includes speech recognition, sound source tracking and speaker recognition. The method of claim 12, The peripheral device, Voice processing system for a network-based robot, comprising at least one of a camera means, a microphone, a speaker, a touch sensor, a display. The method of claim 12, The driver, And providing an interface for driving the peripheral device with the robot controller, and receiving signals and sensor data from the peripheral device and transmitting the signal to the robot controller.

Images