KR102643172B1 - Display apparatus, voice acquiring apparatus and voice recognition method thereof - Google Patents

Abstract

The present invention relates to a display device, a voice acquisition device, and a voice recognition method thereof, wherein the display device includes a display unit that displays an image; a communication unit that communicates with a plurality of external devices; It includes a voice recognition engine that recognizes a voice uttered by a user, and the communication unit receives a voice signal from a voice acquisition unit and receives a candidate command for recognizing the received voice signal from at least one of the plurality of external devices. It includes a control unit that controls. As a result, it is possible to increase the utilization of various voice acquisition equipment and provide linked services using them, and by allowing candidate commands for voice recognition to be dynamically registered/deleted, flexibly coping with problems such as processing delay or low recognition rate The efficiency of the voice recognition system can be improved.

Description

Display device, voice acquisition device, and voice recognition method {DISPLAY APPARATUS, VOICE ACQUIRING APPARATUS AND VOICE RECOGNITION METHOD THEREOF}

The present invention relates to a display device, a voice acquisition device, and a voice recognition method thereof, and more specifically, to a display device that recognizes a voice uttered by a user, a voice acquisition device, and a voice recognition method thereof.

The use of electronic devices with a voice recognition function that recognizes the voice uttered by the user is gradually increasing. Voice recognition is being actively used not only in PCs and mobile communication devices, but also in various electronic devices such as digital TVs, air conditioners, and home theaters.

In order to perform this voice recognition function, a microphone that receives the voice spoken by the user is installed in the main device, such as a TV, and a voice recognition engine that recognizes the input voice. The voice recognition engine converts the voice input into the microphone into pre-stored candidates. By comparing it with the command, the voice uttered can be recognized according to the result.

However, in the case of electronic devices with a conventional voice recognition function, the means for receiving voice input from the user is fixed, so it is difficult to utilize various input means capable of voice input, such as a mobile phone. In addition, the more diverse the candidate commands, the higher the recognition rate, but as the number of candidate commands increases, the number of comparison targets increases, slowing down the voice recognition processing speed, and the storage capacity of the main device is inevitably limited, so candidate commands are continuously added. It has the disadvantage of being difficult.

A display device according to an embodiment of the present invention includes a display unit that displays an image; a communication unit that communicates with a plurality of external devices; It includes a voice recognition engine that recognizes a voice uttered by a user, and the communication unit receives a voice signal from a voice acquisition unit and receives a candidate command for recognizing the received voice signal from at least one of the plurality of external devices. It includes a control unit that controls.

A plurality of voice acquisition units are provided, and when a voice input is detected from at least one of the plurality of voice acquisition units, the control unit can receive a voice signal from the voice acquisition unit that senses the voice input.

The voice acquisition unit includes at least one of a built-in microphone provided in the display device, a first external microphone provided in at least one of the plurality of external devices, and a second external microphone separated from the display device and the plurality of external devices. can do.

The external device includes at least one application, and the candidate command may be managed by the application.

It manages the candidate commands and may further include a built-in application provided in the display device.

It further includes a storage unit for storing the received candidate commands, and the speech recognition engine can recognize speech using the received candidate commands.

When the control unit detects that a wake-up keyword for one of the plurality of voice acquisition units is uttered, it activates a voice acquisition unit corresponding to the detected wake-up keyword and receives a voice signal from the activated voice acquisition unit. You can receive it.

The control unit, when a trigger signal according to the operation of a predetermined button provided on any one of the plurality of voice acquisition units is input, activates a voice acquisition unit corresponding to the input trigger signal, and outputs a voice signal from the activated voice acquisition unit. can receive.

The control unit may control the display unit to display a voice recognition result for the voice signal and a candidate command according to the voice recognition result.

The display unit may further display application information that manages the candidate command.

The voice recognition engine may recognize the voice by determining a command that matches or is similar to the received voice signal among the received candidate commands.

Meanwhile, a voice acquisition device according to an embodiment of the present invention includes a communication unit that communicates with a display device having a voice recognition function; a voice acquisition unit that receives the voice uttered by the user; a voice conversion unit that converts the input voice into an electrical voice signal; and a control unit that controls the communication unit to transmit the converted voice signal and a candidate command for recognizing the voice signal to the display device.

It further includes at least one application, and the candidate command can be managed by the application.

Meanwhile, a voice recognition method of a display device according to an embodiment of the present invention includes receiving a voice signal from a voice acquisition unit; receiving a candidate command for recognizing the received voice signal from at least one of a plurality of external devices; and recognizing a voice uttered by the user according to the received voice signal and candidate command.

It may further include detecting a voice input from at least one of the plurality of voice acquisition units, and the step of receiving the voice signal may include receiving a voice signal from the voice acquisition unit that detects the voice input.

The voice acquisition unit includes at least one of a built-in microphone provided in the display device, a first external microphone provided in at least one of the plurality of external devices, and a second external microphone separated from the display device and the plurality of external devices. can do.

The external device includes at least one application, and the candidate command may be managed by the application.

It manages the candidate commands and may further include a built-in application provided in the display device.

It may further include the step of storing the received candidate command, and the step of recognizing the voice may recognize the voice using the received candidate command.

The step of detecting the voice input includes detecting that a wake-up keyword for one of the plurality of voice acquisition units is uttered; It may include activating a voice acquisition unit corresponding to the detected wake-up keyword.

The step of detecting the voice input includes detecting that a trigger signal is input according to the operation of a predetermined button provided on one of the plurality of voice acquisition units; It may include activating a voice acquisition unit corresponding to the input trigger signal.

The method may further include displaying a voice recognition result for the voice signal and a candidate command according to the voice recognition result.

The displaying step may further display application information that manages the candidate command.

In the step of recognizing the voice, the voice may be recognized by determining a command that matches or is similar to the received voice signal among the received candidate commands.

1 is an exemplary diagram of a voice recognition system according to an embodiment of the present invention;
Figure 2 is a block diagram showing the configuration of a voice recognition system according to an embodiment of the present invention;
Figure 3 is a diagram illustrating an example of performing voice recognition according to an embodiment of the present invention;
Figure 4 is a diagram showing an example of a screen displayed as a result of voice recognition in Figure 3;
Figure 5 is a diagram illustrating an example of performing voice recognition according to another embodiment of the present invention;
Figure 6 is a flowchart showing the voice recognition method of the voice recognition system according to an embodiment of the present invention;
Figure 7 is a flowchart showing in more detail the process of detecting the voice input of Figure 6;
FIG. 8 is a flowchart showing the process of performing voice recognition in FIG. 6 in more detail.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

1 is an exemplary diagram of a voice recognition system according to an embodiment of the present invention.

As shown in Figure 1, the voice recognition system according to an embodiment of the present invention includes a main device 100, a plurality of voice acquisition devices 201, ... , 202, and a plurality of external devices 301, 302, .. , 303). The main device 100, a plurality of voice acquisition devices 201, 202, and a plurality of external devices 301, 302, 303 are connected to enable communication with each other.

The main device 100 is provided with a voice acquisition unit 140 such as a microphone into which the user's voice is input, and a voice recognition engine 181 that recognizes the input voice, and a voice acquisition device ( 201, 202) and external devices (301, 302, 303). In addition, the main device 100 is provided with one or more built-in applications 171 and 172 (Native Applications), which are operated so that the main device 100 can perform various functions (services). In the built-in applications 171 and 172 of this embodiment, candidate commands corresponding to the function to be performed, that is, included in a service scenario that can be performed, are stored in advance, and the candidate commands stored in the built-in applications 171 and 172 are used by the voice recognition engine during voice recognition. It is transmitted to (181), allowing the voice recognition engine (181) to perform voice recognition.

The voice acquisition devices 201 and 202 include a voice acquisition unit such as a microphone that receives the voice uttered by the user, and a voice signal corresponding to the input voice is transmitted to the main device 100 for voice recognition.

The voice acquisition devices 201 and 202 receive the voice uttered by the user, convert it into an electrical voice signal, and transmit it to the main device 100. The voice acquisition devices 201 and 202 are external devices capable of wireless communication with the main device 100, and wireless communication includes wireless LAN, RF (radio frequency) communication, Bluetooth, Zigbee, Includes infrared (IR) communication, etc.

One or more external applications (Dev. Application) are provided on the external devices 301, 302, and 303, and can perform functions (services) required by each external device 301, 302, and 303. In the external application of this embodiment, candidate commands corresponding to the functions performed by the corresponding external devices 301, 302, and 303 are stored in advance, and the candidate commands stored in the external application are transmitted to the voice recognition engine 181 during voice recognition. , Enables the voice recognition engine 181 to perform voice recognition.

Here, candidate commands pre-registered in the built-in applications 171 and 172 and external applications may be commands related to the function/operation of the corresponding application. For example, if the main device 100 is a TV, candidate commands related to channel movement, volume control, etc. of the TV may be stored in one of the built-in applications 171 and 172, and the external device 302 is an air conditioner. In this case, candidate commands related to temperature control (up/down), wind strength control (strong wind/weak wind/light wind), etc. may be stored in an application included in the external device 302.

The present invention can be implemented so that an external device or voice acquisition device includes both a voice acquisition unit and an external application, such as external device 1 (301) in FIG. 1. In this case, when a voice is input from the voice acquisition unit provided in external device 1 (301), the candidate command pre-stored in the external application of external device 1 (301) is transmitted to the voice recognition engine 181 of the main device 100. , voice recognition can be performed.

The voice recognition system of this embodiment includes at least one voice acquisition unit, and when a voice input is detected from the voice acquisition unit, the voice acquisition unit that senses the voice input is activated to receive a voice stream. When a plurality of voice acquisition units are provided, a voice stream can be received by activating a voice acquisition unit that detects a voice input among the plurality of voice acquisition units. Here, the plurality of voice acquisition units includes a built-in microphone provided in the main device 100, a first external microphone provided in at least one of the plurality of external devices 301, 302, and 303, the main device 100, and a plurality of external devices. It may include a second external microphone provided in a separate voice acquisition device (201, 202) (301, 302, 303).

When the main device 100 detects that a wakeup keyword for one of the plurality of voice acquisition units is uttered, it activates the voice acquisition unit corresponding to the detected wakeup keyword, and selects the voice acquisition unit from the activated voice acquisition unit. Voice signals can be received. In addition, when a trigger signal according to the operation (event generation) of a predetermined button provided on one of the plurality of voice acquisition units is input, the main device 100 activates the voice acquisition unit corresponding to the input trigger signal. , Voice signals can be received from the activated voice acquisition unit.

The main device 100 of this embodiment operates in voice recognition mode, and when one voice acquisition unit is activated by a wake-up keyword or trigger signal, the other voice acquisition unit is deactivated to prevent errors in voice recognition. there is. The main device 100 can operate in a long-distance or short-distance voice recognition mode. In addition, a UI indicating a voice acquisition unit connected to the display unit 130, which will be described later, is displayed to assist the user's convenience.

Additionally, the main device 100 may receive a candidate command for recognizing the received voice signal from at least one of the plurality of external devices 301, 302, and 303. The received candidate command can be transmitted to the voice recognition engine 181 for voice recognition.

The plurality of external devices 301, 302, and 303 include at least one application, and candidate commands may be managed by each corresponding application. In addition, the main device 100 further includes built-in applications 171 and 172, and the built-in applications 171 and 172 manage corresponding candidate commands. Candidate commands managed by the built-in applications 171 and 172 can be transmitted to the voice recognition engine 181 for voice recognition.

The main device 100 of the present invention can be implemented as a display device such as a TV, as shown in FIG. 2. FIG. 2 is a block diagram showing the configuration of a voice recognition system according to an embodiment of the present invention.

The display device 100 processes an image signal provided from an external image source (not shown) according to a preset image processing process so that it can be displayed as an image.

In the system of this embodiment, the case where the display device 100 is implemented as a TV or set top box that processes broadcast images based on broadcast signals/broadcast information/broadcast data received from the broadcasting station's transmission equipment will be described. . However, the spirit of the present invention is not limited to the implementation example of the display device 100, and the display device 100 may be applied to various types of implementation examples capable of processing images other than TV and set-top boxes.

In addition, the type of video that the display device 100 can display is not limited to broadcast video. For example, the display device 100 includes video based on signals/data received from various types of video sources (not shown), It can be processed to display images such as still images, applications, OSD (on-screen display), and GUI (graphic user interface) for controlling various operations.

According to an embodiment of the present invention, the display device 100 may be implemented as a smart TV. Smart TV is a TV that can receive and display broadcasting signals in real time and has a web browser function so that it can display real-time broadcasting signals and simultaneously search for and consume various contents through the Internet and provide a convenient user environment for this purpose. am. Additionally, smart TVs include an open software platform and can provide interactive services to users. Accordingly, smart TVs can provide users with various contents, for example, applications that provide certain services, through an open software platform. These applications are applications that can provide various types of services, and include applications that provide services such as SNS, finance, news, weather, maps, music, movies, games, and e-books.

The display device 100 of this embodiment is provided with a voice recognition engine 181 that recognizes the user's voice. The command corresponding to the recognized voice, that is, the control command, is transmitted to the corresponding application to perform an operation according to the corresponding function. Here, when the application corresponding to the control command is one of the built-in applications 171 and 172, the display device 100 performs an operation according to the control command by the corresponding application. In addition, if the application corresponding to the control command is an external application, the control command is transmitted to the external device (301, 302, 303) including the external application, and the external device (301, 302, 303) is controlled by the application. Operations can be performed according to control commands.

Meanwhile, Figure 2 shows a case where the main device is implemented as a display device such as a TV, but the present invention is not limited to this and the main device may be implemented as various electronic devices, such as a PC.

In the embodiment shown in FIG. 2, a plurality of voice acquisition devices are provided, for example, implemented with a mobile phone 200 and a remote controller 300. Here, the remote control 300 functions as both a voice acquisition device and an external device, and the mobile phone 200 may be a smartphone with a voice acquisition function.

The remote control 300 transmits a command (control command) preset by the user's operation to the corresponding device. The remote control of this embodiment may be preset to transmit commands to the display device 100 or an external device, and in some cases, may be implemented as an integrated remote control that transmits commands to a plurality of devices. Additionally, the remote control 300 may include a TV remote control and/or an air conditioner remote control.

Meanwhile, the voice acquisition device of the present invention can be implemented with various devices that can receive voice uttered by the user, such as a mobile phone 200 and a remote control 300, as well as a portable terminal device and a microphone transmitter.

In the embodiment shown in FIG. 2, a plurality of external devices are provided, for example, implemented as a remote control 300 and an air conditioner 400. As described above, the remote control 300 functions as both a voice acquisition device and an external device.

In the embodiment of Figure 2, the external device is implemented as an example of a remote control 300 and an air conditioner 400, but the external device of the present invention is not limited to this, and various electronic devices capable of wireless communication, such as a home It can be implemented as a theater, radio, VCR, DVD, washing machine, refrigerator, robot vacuum cleaner, etc. Additionally, if a voice acquisition unit such as a microphone is provided in an external device, it can also serve as a voice acquisition device.

The external device of this embodiment is equipped with applications 372 and 472 to perform functions. Candidate commands are stored and managed in advance in each application 372 and 472, and the candidate commands are sent to the display device 100 for voice recognition. It can be sent to .

External devices, that is, the remote control 300 and the air conditioner 400, perform operations corresponding to the control command received from the display device 100 according to the voice recognition result.

Hereinafter, with reference to FIG. 2, each configuration of the voice recognition system will be described in more detail.

As shown in FIG. 2, the display device 100 includes an image receiver 110 that receives an image signal, an image processor 120 that processes the image signal received by the image receiver 110, and an image processor 120. A display unit 130 that displays the image signal processed by the image as an image, a first voice acquisition unit 140 that receives the voice uttered by the user, and a first voice conversion unit that converts the input voice into an electrical voice signal ( 150), a first communication unit 160 that communicates with an external device, a first storage unit 170 that stores various data, and a first control unit 180 that controls the display device 100.

The image receiving unit 110 receives an image signal and transmits it to the image processing unit 120, and may be implemented in various ways depending on the standard of the received image signal and the implementation type of the display device 100. For example, the video receiver 110 wirelessly receives a radio frequency (RF) signal transmitted from a broadcasting station (not shown), or transmits composite video, component video, super video, and SCART. , video signals according to the HDMI (high definition multimedia interface) standard can be received by wire. When the video signal is a broadcast signal, the video receiver 110 includes a tuner that tunes the broadcast signal for each channel.

Additionally, video signals can be input from external devices, for example, video signals can be input from external devices such as PCs, AV devices, smartphones, smart pads, etc. Additionally, the video signal may originate from data received through a network such as the Internet. In this case, the display device 100 may perform network communication through the first communication unit 160 or may further include a separate network communication unit. Additionally, the image signal may originate from data stored in the non-volatile first storage unit 170, such as a flash memory or hard disk. The first storage unit 170 may be provided inside or outside the display device 100, and when provided outside, it may further include a connection part (not shown) to which the first storage unit 170 is connected.

The image processing unit 120 performs various preset image processing processes on video signals. The image processing unit 120 outputs an image signal that has undergone this process to the display unit 130, thereby displaying an image on the display unit 130.

The type of image processing process performed by the image processing unit 120 is not limited, for example, decoding, de-interlacing, frame refresh rate conversion, and scaling corresponding to various image formats. It may include scaling, noise reduction, detail enhancement, line scanning, etc. to improve image quality. The image processing unit 120 may be implemented as a group of individual components that can independently perform each of these processes, or may be implemented as a system-on-chip (SoC) that integrates several functions.

The display unit 130 displays an image based on the image signal processed by the image processing unit 120. The implementation method of the display unit 130 is not limited, and includes, for example, liquid crystal, plasma, light-emitting diode, organic light-emitting diode, surface conduction electron gun, etc. It can be implemented in various display methods such as -conduction electron-emitter, carbon nano-tube, and nano-crystal.

The display unit 130 may additionally include additional components depending on its implementation method. For example, when the display unit 130 is a liquid crystal display panel, the display unit 130 drives a liquid crystal display panel (not shown), a backlight unit (not shown) that supplies light to the panel, and a panel (not shown). It includes a panel drive board (not shown).

The display unit 130 of the present invention can display the voice recognition result as information about the recognized voice. Here, voice recognition results can be displayed in various forms such as text, graphics, and icons, and text includes letters and numbers. The display unit 130 of this embodiment can further display candidate commands and application information according to the voice recognition results. This will be explained in more detail in FIG. 4 described later.

The user can check whether the voice has been recognized correctly by the voice recognition result displayed on the display unit 130 and select a command corresponding to the voice uttered by the user from among the displayed candidate commands by manipulating the user input unit 390 provided on the remote control. Alternatively, you can select and check information related to the voice recognition result.

The first voice acquisition unit 140 receives the voice uttered by the user and may be implemented as a microphone.

The first voice conversion unit 150 converts the voice input from the first voice acquisition unit 140 into an electrical voice signal. The converted voice signal has a PCM (pulse code modulation) state or a compressed audio waveform. Here, the first voice conversion unit 150 may be implemented as an A/D conversion unit that converts the user's input voice into digital.

Meanwhile, when the first voice acquisition unit 140 is a digital microphone, separate A/D conversion is not required, so the first voice acquisition unit 140 may include the first voice conversion unit 150. .

The first communication unit 160 communicates with the voice acquisition device and external devices, that is, the mobile phone 200, remote control 300, and air conditioner 400 of FIG. 2. The first communication unit 160 of this embodiment can perform wireless communication, and the wireless communication includes at least one of infrared communication, RF, ZigBee, and Bluetooth.

The first storage unit 170 stores unlimited data under the control of the first control unit 180. The first storage unit 170 is implemented as a non-volatile storage medium such as flash memory or hard disk drive. The first storage unit 170 is accessed by the first control unit 180, and data reading/recording/editing/deleting/updating, etc. is performed by the first control unit 180.

Data stored in the first storage unit 170 includes, for example, an operating system for driving the display device 100, various applications executable on this operating system, image data, additional data, etc.

The first storage unit 170 of this embodiment may store various data for recognizing a voice uttered by a user. For example, the first storage unit 170 may store a command table 171 containing candidate commands (hereinafter also referred to as a command candidate group) as speech recognition target information corresponding to a received speech signal. In the command table 171, candidate commands can be managed for each application.

The first storage unit 170 of this embodiment further stores one or more applications, for example, applications 1 and 2 (172, 173), for performing the functions of the display device 100. Applications 1 and 2 (172, 173) are driven under the control of the first control unit 180, which will be described later, and perform various functions of the display device 100. 2 illustrates an example in which two applications 172 and 173 are built into the display device 100. However, since the functions performed by the display device 100 are diverse, the display device 100 of the present invention includes three applications. More than one application may be built-in.

Applications 1 and 2 (172, 173) each manage candidate instructions corresponding to the functions they perform. Candidate instructions managed in applications 1 and 2 (172, 173) may be registered/deleted in the instruction table 171 as needed.

When the candidate commands of the first and/or second applications 172 and 173 are registered in the command table 171, the voice recognition engine 181 performs voice recognition using the candidate commands registered in the command table 171. I do it.

Candidate commands that can be registered/deleted in the command table 171 of this embodiment include candidate commands managed by application 3 (372) of the remote control 300, which will be described later, and applications managed by application 4 (472) of the air conditioner 400. It can be included.

The first control unit 180 performs control operations for various components of the display device 100. For example, the first control unit 180 controls the overall operation of the display device 100 by progressing the image processing process processed by the image processing unit 120 and performing control operations corresponding to commands from the remote control 300. Control.

The first control unit 180 may be implemented, for example, in a form in which software is combined with a CPU.

The first control unit 180 includes a voice recognition engine 181 that recognizes the voice uttered by the user. The voice recognition function of the voice recognition engine 181 can be performed using a known voice recognition algorithm. For example, the voice recognition engine 181 extracts a voice feature vector from a voice signal, compares the extracted voice feature vector with candidate commands stored in the command table 171 of the first storage unit 170, and recognizes the voice. It can be recognized. In addition, if there is no command among the candidate commands stored in the command table 171 that matches the extracted speech feature vector, the speech can be recognized by correcting the speech recognition result with a command with high similarity. Here, when there are multiple candidate commands with high similarity, the first control unit 180 displays a plurality of candidate commands on the display unit 130, and the user can select one.

The voice recognition engine 181 of this embodiment is explained by taking as an example that it is implemented as an embedded voice recognition engine 181 residing in the CPU, but the present invention is not limited to this. For example, the voice recognition engine 181 may be implemented as a device built into the display device 100 that is separate from the CPU, that is, a separate chip such as a microcomputer.

Meanwhile, the present invention includes a case where the voice recognition engine 181 is provided on a server (hereinafter also referred to as a cloud server (not shown)) that is physically separate from the display device 100. The cloud server communicates with the display device 100 through a network such as the Internet. Here, the network may be a wired or wireless network. In this case, the voice recognition engine 181 may be implemented as an embedded voice recognition engine residing in the CPU of the cloud server, or as a separate chip such as a device embedded in the cloud server separately from the CPU, that is, a microcomputer. .

The first control unit 180 may perform an operation corresponding to the recognition result of the voice recognition engine 181. For example, when the display device 100 is a TV and the user is watching a movie or news, when the voice recognition engine 181 recognizes volume up, volume down, volume up, volume down, etc., the first control unit (180) can adjust the sound level (volume) of movies or news to correspond to this.

As a result of the recognition of the voice recognition engine 181, if the recognized voice is a voice that controls an external device such as the remote control 300 or the air conditioner 400, the first control unit 180 sends a control command corresponding to the recognized voice. The first communication unit 160 can be controlled to transmit to the corresponding external device. For example, when the voice recognition engine 181 recognizes raising the temperature, the first control unit 180 recognizes this as a voice for the air conditioner and transmits a command to raise the temperature of the air conditioner to the air conditioner 400. The first communication unit 160 can be controlled.

Hereinafter, the specific configuration of the mobile phone 200 will be described.

As shown in FIG. 2, the mobile phone 200 is a voice acquisition device that includes a second voice acquisition unit 240 that receives the voice uttered by the user, and a second voice conversion unit that converts the input voice into an electrical voice signal. (250), a second communication unit 260 that communicates with the outside, a second storage unit 270 that stores unlimited data, and a second control unit 280 that controls the mobile phone 200.

The second voice acquisition unit 240 receives the voice uttered by the user and can be implemented as a microphone. The second voice conversion unit 250 converts the voice input from the second voice acquisition unit 240 into an electrical voice signal. The converted voice signal has a PCM (pulse code modulation) state or a compressed audio waveform. Here, the first voice conversion unit 250 may be implemented as an A/D conversion unit that converts the user's input voice into digital.

Meanwhile, when the second voice acquisition unit 240 is a digital microphone, separate A/D conversion is not required, so the second voice acquisition unit 240 may include a second voice conversion unit 240. .

The second communication unit 260 communicates with the display device 100. The second communication unit 260 of this embodiment performs wired or wireless communication, and the wireless communication includes at least one of RF, ZigBee, and Bluetooth.

The second communication unit 260 of this embodiment transmits the voice signal transmitted from the second voice conversion unit 250 to the display device 100.

The second storage unit 270 stores unlimited data under the control of the second control unit 280. The second storage unit 270 is implemented as a non-volatile storage medium such as flash memory. The second storage unit 270 is accessed by the second control unit 280, and reading/recording/editing/deleting/updating of data is performed by the second control unit 280.

Data stored in the second storage unit 270 includes, for example, an operating system for driving the mobile phone 200, various applications executable on this operating system, video data, additional data, etc.

The second control unit 280 performs control operations for various components of the mobile phone 200. For example, the second control unit 280 may generate a command corresponding to the user's operation, perform an operation corresponding to the generated command, and display the result on a display unit (not shown).

For example, the second control unit 280 may be implemented in a form that combines software with an MCU (Micro Controller Unit).

The second control unit 280 of this embodiment controls the second voice conversion unit 250 to convert the voice uttered by the user into an electrical voice signal when the voice uttered by the user is input through the second voice acquisition unit 240. The second communication unit 260 is controlled to transmit the voice signal to the display device 100.

Hereinafter, the specific configuration of the remote control 300 will be described.

As shown in FIG. 2, the remote control 300 is a voice acquisition device and an external device, and includes a user input unit 330 that receives the user's operation, and a third voice acquisition unit 340 that receives the voice uttered by the user. , a third voice conversion unit 350 that converts the input voice into an electrical voice signal, a third communication unit 360 that communicates with the outside, a third storage unit 370 that stores unlimited data, and a remote control. It includes a third control unit 380 that controls 300.

The user input unit 330 transmits various preset control commands or unlimited information to the third control unit 380 through the user's manipulation and input. The user input unit 330 may be implemented as a button installed on the outside of the remote control 300, including a menu key and number keys. When the remote control 300 is a TV remote control, the user input unit 330 may further include a touch detection unit that receives the user's touch input and/or a motion detection unit that detects motion of the remote control 300.

The third voice acquisition unit 340 receives the voice uttered by the user and can be implemented as a microphone.

The third voice conversion unit 350 converts the voice input from the third voice acquisition unit 340 into an electrical voice signal. The converted voice signal has a PCM (pulse code modulation) state or a compressed audio waveform. Here, the third voice conversion unit 350 may be implemented as an A/D conversion unit that converts the user's input voice into digital.

Meanwhile, when the third voice acquisition unit 340 is a digital microphone, separate A/D conversion is not required, so the third voice acquisition unit 340 may include a third voice conversion unit 350. .

The third communication unit 360 communicates with the display device 100. The third communication unit 360 of this embodiment performs wireless communication, and the wireless communication includes at least one of infrared communication, RF, ZigBee, and Bluetooth.

The third communication unit 360 of this embodiment transmits the voice signal transmitted from the third voice conversion unit 350 to the display device 100, and is transmitted by application 3 (372) in the third storage unit 370, which will be described later. The managed candidate command is transmitted to the display device 100.

The third storage unit 370 stores unlimited data under the control of the third control unit 380. The third storage unit 370 is implemented as a non-volatile storage medium such as flash memory. The third storage unit 370 is accessed by the third control unit 380, and data reading/recording/editing/deleting/updating, etc. is performed by the third control unit 380.

Data stored in the third storage unit 370 includes, for example, an operating system for driving the remote control 300, various applications executable on this operating system, video data, additional data, etc.

In the third storage unit 370 of this embodiment, one or more applications, for example, application 3 372, for performing the functions of the remote control 300 are further stored. Application 3 372 is driven under the control of the third control unit 380, which will be described later, and performs various functions of the remote control 300. In the present invention, application 3 (372) and application 4 (472), which will be described later, are referred to as external applications to distinguish them from the applications (172 and 173) built into the display device (100).

Although FIG. 2 shows an example in which one application 372 is provided in the remote control 300, the present invention also includes a case in which two or more applications are provided in the remote control 300.

Application 3 372 manages candidate instructions corresponding to the function being performed. Candidate commands managed in application 3 372 may be registered/deleted in the command table 171 of the display device 100 as needed.

The third control unit 380 performs control operations for various components of the remote control 300. For example, the third control unit 260 generates a command corresponding to the user's operation on the user input unit 330 and controls the third communication unit 360 to transmit the generated command to the display device 100. You can.

For example, the third control unit 380 may be implemented in the form of software combined with an MCU (Micro Controller Unit).

The third control unit 380 of this embodiment controls the third voice conversion unit 350 to convert the voice uttered by the user into an electrical voice signal when the voice uttered by the user is input through the third voice acquisition unit 340. The third communication unit 360 is controlled to transmit the audio signal to the display device 100.

The third control unit 380 may transmit a candidate command managed in application 3 372 of the third storage unit 370 to the display apparatus 100 while communicating with the display apparatus 100 . The transmitted candidate command is registered in the command table 171 of the display device 100 and used for the voice recognition operation of the voice recognition engine 181.

In addition, when a control command based on voice recognition is received from the display device 100, the third control unit 380 can receive it through the third communication unit 360 and perform an operation corresponding to the received control command. there is.

Hereinafter, the specific configuration of the air conditioner 400 will be described.

As shown in FIG. 2, the air conditioner 400 is an external device and includes a fourth communication unit 340 that communicates with the outside, a fourth storage unit 470 that stores unlimited data, and the air conditioner 400. It includes a fourth control unit 480 that controls.

The fourth communication unit 460 performs communication with the display device 100. The fourth communication unit 460 of this embodiment performs wireless communication, and the wireless communication includes at least one of RF, ZigBee, and Bluetooth.

The fourth communication unit 460 of this embodiment transmits a candidate command managed by application 4 472 in the fourth storage unit 370, which will be described later, to the display device 100.

The fourth storage unit 470 stores unlimited data under the control of the fourth control unit 480. The fourth storage unit 470 is implemented as a non-volatile storage medium such as flash memory. The fourth storage unit 470 is accessed by the fourth control unit 480, and data reading/writing/editing/deleting/updating, etc. is performed by the fourth control unit 480.

Data stored in the fourth storage unit 470 includes, for example, an operating system for driving the mobile phone 200, various applications executable on this operating system, and additional data.

In the fourth storage unit 470 of this embodiment, one or more applications (external applications) for performing the functions of the air conditioner 300, for example, application 4 472, are further stored. Application 4 472 is driven under the control of the fourth control unit 480, which will be described later, and performs various functions of the air conditioner 400.

Although FIG. 2 shows an example in which one application 472 is provided in the air conditioner 400, the present invention also includes a case in which two or more applications are provided in the air conditioner 400.

Application 4 472 manages candidate instructions corresponding to the function being performed. Candidate commands managed in application 4 472 may be registered/deleted in the command table 171 of the display device 100 as needed.

The fourth control unit 460 performs control operations for various components of the air conditioner 400. For example, the fourth control unit 260 may receive a control command corresponding to the user's operation of the air conditioner remote control and perform control operations according to the received control command, such as temperature control.

For example, the fourth control unit 460 may be implemented in a form in which software is combined with an MCU (Micro Controller Unit).

The fourth control unit 460 of this embodiment may transmit a candidate command managed in application 4 472 of the fourth storage unit 470 to the display device 100 while communicating with the display device 100. . The transmitted candidate command is registered in the command table 171 of the display device 100 and used for the voice recognition operation of the voice recognition engine 181.

In addition, when a control command according to voice recognition is received from the display device 100, the fourth control unit 480 can receive it through the fourth communication unit 460 and perform an operation corresponding to the received control command. there is.

According to the embodiment of FIG. 2, the first control unit 180 of the display device 100 as the main device of the voice recognition system of the present invention receives input voice from at least one of the plurality of voice acquisition units 140, 240, and 340. When detected, the first communication unit 140 is controlled to receive a voice signal from the voice acquisition unit that detects the voice input. In addition, the first control unit 180 sends a candidate command for recognizing the received voice signal to at least one of the external applications 372 and 472 of the plurality of external devices 300 and 400 or the built-in command through the first communication unit 140. Candidate commands are received from the applications 172 and 173, and are registered in the command table 171 of the first storage unit 170. The voice recognition engine 181 compares the voice signal with the candidate command registered in the command table 171 and performs voice recognition.

In this way, the display device 100, as the main device according to this embodiment, detects voice input from various devices that can input the user's speech, thereby increasing the utilization of various voice acquisition devices. In addition, since the candidate commands for voice recognition are received and used from the running application, candidate commands for voice recognition can be dynamically registered/deleted, preventing the candidate commands provided by the main device from increasing unnecessarily infinitely. You can.

Meanwhile, the display device 100 according to an embodiment of the present invention receives voice recognition input from the voice acquisition unit 140 provided in the display device 100, and internal applications 171, 172, 173 and external applications 372, 472. It can be implemented to perform voice recognition in the voice recognition engine 181 by receiving a candidate command from at least one of the commands.

Hereinafter, with reference to FIGS. 3 and 4, an embodiment of performing voice recognition in the voice recognition system of the present invention will be described in more detail.

Figure 3 is a diagram illustrating an example of performing voice recognition according to an embodiment of the present invention, and Figure 4 is a diagram illustrating an example of a screen displayed as a result of voice recognition in Figure 3.

As shown in FIG. 3, the display device 100 can register candidate commands stored in the command table 171 from one or more applications (including built-in applications and external applications).

Specifically, referring to FIG. 3, commands A and B are transferred from application 1 (172) (built-in application) to the command table 171 (501), and the delivered commands A and B are stored in the command table 171. becomes (502). The voice recognition engine 181 registers commands A and B stored in the command table 171 as candidate commands (504).

Additionally, commands C and D are transferred from application 3 372 (external application) to the command table 171 (505), and the delivered commands C and D are registered in the command table 171 (507). The voice recognition engine 181 registers commands C and D stored in the command table 171 as candidate commands (508).

Accordingly, A, B, C, and D delivered from Application 1 (172) and Application 3 (372) are registered as candidate commands in the voice recognition engine 181.

With A, B, C, and D registered as candidate commands, the second voice acquisition unit 240, which is separate from the display device 100, can detect A as a voice input. The detected voice A is converted into a voice signal in the second voice conversion unit 250 and transmitted to the voice recognition engine 181 through the second and first communication units 260 and 160 (509).

The voice recognition engine 181 compares the voice signal for voice A with the registered candidate commands A, B, C, and D and determines a matching or similar command to recognize voice A (510).

The first control unit 180 transmits the recognition result to the display unit 130 (511), and the display unit 130 may display a voice recognition result screen as shown in FIG. 4.

As shown in FIG. 4, the display unit 130 has a UI showing voice recognition result A (60) and candidate commands A, B, C, and D (61, 62, 63, 64) according to the voice recognition result. can be displayed. In addition, the display unit 130 may further display a UI indicating application information (application 1) 65 that manages command A according to the voice recognition result.

The user can check not only the voice recognition results but also the candidate commands through the UI displayed on the screen as shown in Figure 4, and if the voice recognition results do not match the utterance intention, the user can select one of the candidate commands. In addition, it is possible to obtain device information related to voice recognition results through application information.

The first control unit 180 transmits a control command according to the voice recognition result as shown in FIG. 4 to application 1 (172) (512). Application 1 (172) performs a control operation according to the recognized voice A under the control of the first control unit (180). For example, when voice A says volume down, the volume of the display device 100 is adjusted to be low.

As described in FIGS. 3 and 4, the display device 100 of the present invention pre-registers commands (e.g., A, B, C, D) of some applications as candidate words according to preset conditions, and registers the commands of some applications as candidate words according to preset conditions. When a voice utterance is detected, the voice can be recognized using a pre-registered candidate command, the result can be displayed, and the corresponding control action can be executed.

3 and 4, the commands of application 1 (172) and application 3 (372) are registered as candidate commands and the user's voice is input through the second voice acquisition unit 240. , the present invention can dynamically register/delete candidate commands by receiving commands from various built-in and external applications, and of course includes all cases of receiving voice input from various voice acquisition units.

Hereinafter, with reference to FIG. 5, another embodiment of performing voice recognition in the voice recognition system of the present invention will be described in more detail.

Figure 5 is a diagram illustrating an example of performing voice recognition according to another embodiment of the present invention.

As shown in FIG. 5, the third voice acquisition unit 340, which is separate from the display device 100, can detect that E is being voice input. The detected voice E is converted into a voice signal in the third voice conversion unit 350 and transmitted to the voice recognition engine 181 through the third and first communication units 360 and 160 (701).

The display device 100 may register a candidate command stored in the command table 171 from application 3 372 corresponding to the device for which voice input was detected. Specifically, referring to FIG. 5, commands E and F are transferred from application 3 372 to the command table 171 (702), and the delivered commands E and F are stored in the command table 171 (703). . The voice recognition engine 181 registers the commands E and F stored in the command table 171 as candidate commands (705).

Accordingly, E and F delivered from application 3 372 are registered as candidate commands in the voice recognition engine 181.

With E and F registered as candidate commands, the voice recognition engine 181 compares the voice signal for voice E with the registered candidate commands E and F, determines a matching or similar command, and recognizes voice E. (706).

The first control unit 180 transmits the recognition result to the display unit 130 (707), and a voice recognition result screen can be displayed on the display unit 130.

The first control unit 180 transmits a control command according to the voice recognition result to application 3 (372) (708). Application 3 (372) performs a control operation according to the recognized voice E under the control of the third control unit (380). Here, when the control command according to the voice recognition result is a command to control the display device 100, the control command may be transmitted to Application 1 (172) or Application 2 (173).

As described in FIG. 5, when the user's voice utterance is detected, the display device 100 of the present invention registers the command (for example, E, F) of the application corresponding to the device through which the voice was input as a candidate command, and , the voice can be recognized using the registered candidate command, the result can be displayed, and the corresponding control operation can be executed. Candidate instructions can be dynamically registered/deleted.

In the embodiment of FIG. 5, the case where the user's voice is input through the third voice acquisition unit 340 and the command of application 3 (372) is registered as a candidate command has been described as an example, but the present invention is provided by various voice acquisition units. Voice input can be received, and of course, all cases of receiving commands from various built-in and external applications and dynamically registering/deleting candidate commands are included.

Hereinafter, the voice recognition method of the voice recognition system according to this embodiment will be described with reference to the drawings.

Figure 6 is a flowchart showing a voice recognition method of a voice recognition system according to an embodiment of the present invention.

As shown in FIG. 6, the voice recognition system of the present invention can detect voice input from at least one of the plurality of voice acquisition units 140, 240, and 340 (S810). Here, the detected user voice is converted into an electrical voice signal in the corresponding voice conversion unit (150, 250, 350).

The first control unit 180 receives the voice signal detected and converted in step S810 (S820). Here, when a voice input is detected by the external voice acquisition units 240 and 340, a voice signal can be received through the first communication unit 160.

A candidate command for voice recognition of the voice signal received in step S820 is registered in the voice recognition engine 181 (S830). Here, the registered candidate command may be pre-stored in the command table 171, or may be received from a built-in or external application (172, 173, 372, 472) and dynamically stored in the command table 171.

The voice recognition engine 181 recognizes the voice uttered by the user in step S810 using the candidate command registered in step S830 (S840).

The first control unit 180 displays the voice recognition result of step S840 on the display unit 130. Here, the display unit 130 may display voice recognition results for voice signals, candidate commands according to the voice recognition results, application information, etc.

The first control unit 180 generates a control command according to the voice recognition result of step S840 and transmits it to an application that can execute it (S860). Accordingly, an operation according to the generated control command can be performed.

Figure 7 is a flowchart showing the process of detecting the voice input of Figure 6 in more detail.

As shown in FIG. 7, the display device 100, which is the main device, can operate in a voice input mode that allows user input by voice (S811). The voice input mode in the present invention operates to enable voice input from various voice acquisition units (140, 240, 340).

As one of the methods for detecting voice input, the first control unit 180 can detect the utterance of a wake-up keyword in any one of the plurality of voice acquisition units 140, 240, and 340 (S812). Here, the wake-up keyword activates voice input through a specific voice acquisition unit and can be set in advance. For example, the first voice acquisition unit 140 of the display device 100 may set a voice related to TV control such as channel and volume as a wake-up keyword, and the second voice acquisition unit 240 of the mobile phone 200 may set the voice related to TV control, such as channel and volume, as a wake-up keyword. ) can set voices related to calls, contacts, etc. as wake-up keywords.

Meanwhile, as another method of detecting voice input, the first control unit 180 receives a trigger signal according to the operation of a predetermined button (voice input button) from any one of the plurality of voice acquisition units 140, 240, and 340. , Voice input can be detected from the corresponding voice acquisition unit (S813). For example, when a user operates a voice input button provided on a specific voice acquisition device, voice input is detected from a voice input unit provided on the device.

The first control unit 180 activates one of the plurality of voice acquisition units 140, 240, and 340 corresponding to voice input according to the detection results of steps S812 and S813 (S812). By activating any one voice acquisition unit in this way, it is possible to prevent unnecessary voices from being detected and causing confusion.

Then, a voice signal is received from the voice acquisition unit activated in step S820 to the voice recognition engine 181, allowing voice recognition to be performed.

FIG. 8 is a flowchart showing the process of performing voice recognition in FIG. 6 in more detail.

As shown in FIG. 8, in step S830 of FIG. 6, the voice recognition engine 181 may receive and register a candidate command from one or more applications among the plurality of applications 172, 372, and 472.

The voice recognition engine 181 may determine whether the candidate command registered in step S830 matches/is similar to the voice signal received in step S820 of FIG. 6 (S841).

As a result of the determination in step S840, if there is a matching/similar candidate command, the voice recognition engine 181 determines the matching/similar command and performs voice recognition, and the first control unit 180 displays the result to the display unit 130. It will be displayed in .

As a result of the determination in step S840, if there is no matching/similar candidate command, the voice recognition engine 181 may determine whether to receive and register a candidate command of an application different from the application in step S830 (S842). Here, the first control unit 180 can receive and register candidate commands of other applications according to the user's selection or pre-entered settings, and can sequentially receive and register candidate commands of a plurality of applications according to a preset order. do. Additionally, considering the capacity of the first storage unit 170 of the display device 100, previously registered candidate instructions may be selectively deleted.

If there is no matching/similar candidate command among the candidate commands registered in step S842, steps S842 and 841 are sequentially repeated to perform voice recognition.

Meanwhile, if it is determined in step S842 not to receive and register the candidate command of another application, the voice recognition engine 181 stops voice recognition, and the first control unit 180 causes voice recognition failure as a result of the display unit 130 ) can be displayed.

In this way, according to the embodiment of the present invention, the main device detects voice input from various devices that can input the user's speech, thereby increasing the utilization of various voice acquisition devices and enabling linked services using them.

In addition, since candidate commands for voice recognition are received and used from multiple applications, candidate commands for voice recognition can be dynamically registered/deleted, thereby preventing the candidate commands provided in the main device from increasing unnecessarily. It can flexibly deal with problems such as processing speed delays or low recognition rates, and can increase the efficiency of the entire voice recognition system.

In addition, users can easily recognize not only voice recognition results, candidate commands, and application information, but also information on various voice acquisition equipment and candidate command provision equipment used for voice recognition, thereby improving user convenience.

Above, the present invention has been described in detail through preferred embodiments, but the present invention is not limited thereto and may be implemented in various ways within the scope of the patent claims.

100: main device, display device 110: video receiver
120: image processing unit 130: display unit
140: first voice acquisition unit 150: first voice conversion unit
160: first communication unit 170: first storage unit
171: Command table 172, 173: Application
180: first control unit 161: voice recognition engine
201, 202: Voice acquisition device 200: Cell phone
240: Second voice acquisition unit 250: Second voice conversion unit
260: second communication unit 270: second storage unit
280: second control unit 301, 302, 303: external device
300: remote control 330: user input unit
340: Third voice acquisition unit 350: Second voice conversion unit
360: Third communication unit 370: Third storage unit
371: Application 380: Third control unit
400: Air conditioner 460: 4th Communications Department
470: Fourth storage unit 472: Application
180: 4th control unit

Claims (20)

In the display device,
display unit;
Voice receiver;
a first communication unit that communicates with a first external device including a first external voice receiver; and
In the voice input detection state of the plurality of voice receivers including the voice receiver and the first external voice receiver, when a trigger signal is received from the first external device through the first external voice receiver, the first external voice receiver Activate the first voice recognition function using
In a voice input detection state of the plurality of voice receivers, when a wake-up keyword is received through the voice receiver, activating an internal voice recognition function using the voice receiver,
A display device comprising a processor that deactivates a voice recognition function different from the activated voice recognition function. According to paragraph 1,
When the first voice recognition function using the first external voice receiver is activated, the internal voice recognition function using the voice receiver is deactivated. According to paragraph 1,
When the internal voice recognition function using the voice receiver is activated, the first voice recognition function using the first external voice receiver is deactivated. According to paragraph 1,
It further includes a second communication unit that communicates with a second external device including a second external voice receiver,
The processor,
A display device that deactivates a second voice recognition function using the second external voice receiver corresponding to the second external device when a trigger signal is received from the first external device through the first external voice receiver. According to paragraph 4,
A display device wherein the first communication unit includes a Bluetooth module, and the second communication unit includes a wireless LAN module. According to paragraph 1,
The voice input detection state of the plurality of voice receivers is:
A display device in a state of waiting to receive both the trigger signal and the wake-up keyword. According to clause 6,
While waiting to receive both the trigger signal and the wake-up keyword,
Upon receiving at least one of the trigger signal and the wakeup keyword,
A display device that disables a voice recognition function using a voice receiver corresponding to at least one of an unreceived trigger signal and a wake-up keyword. According to paragraph 1,
It further includes a second communication unit that communicates with a second external device,
The processor,
Obtaining a result corresponding to an operation function of the second external device from at least one of the first voice recognition function and the internal voice recognition function,
A display device that transmits a control signal corresponding to an operation function of the second external device to the second external device through the second communication unit, so that the second external device performs the operation function. According to paragraph 1,
The processor,
A display device that controls the display unit to display a UI corresponding to the activated first voice recognition function. According to paragraph 1,
It further includes a signal receiver,
The processor,
Receiving a video signal through the signal receiver,
Processing the received video signal,
A display device that controls the display unit to display an image corresponding to the processed image signal. In a method of controlling a display device including a first communication unit that communicates with a first external device including a voice receiver and a first external voice receiver,
In the voice input detection state of the plurality of voice receivers including the voice receiver and the first external voice receiver, when a trigger signal is received from the first external device through the first external voice receiver, the first external voice receiver An operation of activating a first voice recognition function using;
In a voice input detection state of the plurality of voice receivers, when a wake-up keyword is received through the voice receiver, activating an internal voice recognition function using the voice receiver; and
A method comprising deactivating a voice recognition function different from the activated voice recognition function. According to clause 11,
When the first voice recognition function using the first external voice receiver is activated, the method further includes deactivating the internal voice recognition function using the voice receiver. According to clause 11,
The method further includes deactivating the first voice recognition function using the first external voice receiver when the internal voice recognition function using the voice receiver is activated. According to clause 11,
When a trigger signal is received from the first external device through the first external voice receiver, the second voice recognition function using the second external voice receiver corresponding to the second external device including the second external voice receiver is deactivated. A method that further includes an operation. According to clause 14,
The display device further includes a second communication unit that communicates with the second external device,
The method wherein the first communication unit includes a Bluetooth module, and the second communication unit includes a wireless LAN module. According to clause 11,
The voice input detection state of the plurality of voice receivers is:
The method is in a state of waiting to receive both the trigger signal and the wakeup keyword. According to clause 16,
While waiting to receive both the trigger signal and the wake-up keyword,
When receiving at least one of the trigger signal and the wake-up keyword, the method further includes deactivating a voice recognition function using a voice receiver corresponding to at least one of the trigger signal and the wake-up keyword that has not been received. According to clause 11,
Obtaining a result corresponding to an operation function of a second external device from at least one of the first voice recognition function and the internal voice recognition function; and
The method further includes transmitting a control signal corresponding to an operation function of the second external device to the second external device through a second communication unit, so that the second external device performs the operation function. According to clause 11,
The method further includes controlling the display unit to display a UI corresponding to the activated first voice recognition function. According to clause 11,
An operation of receiving a video signal through a signal receiver;
Processing the received video signal; and
The method further includes controlling the display unit to display an image corresponding to the processed image signal.

Images