KR20190026518A - Method for operating voice recognition apparatus - Google Patents

Abstract

An objective of the present invention is to provide a voice recognition apparatus capable of providing user personalized voice guidance and an operation method thereof. According to one aspect of the present invention to achieve the above-described objective or the other objective, the operation method comprises the steps of: transmitting a voice input signal of a user through a microphone; transmitting voice data corresponding to the voice input signal to a voice recognition server system; identifying the user based on a frequency and intensity of the voice input signal; receiving a response signal based on the voice input signal from the voice recognition server system; and outputting a voice guidance message corresponding to the received response signal. The step of outputting voice guidance message may provide user personalized voice guidance by outputting the voice guidance message with voice based on tone data stored in a database corresponding to an identified user.

Description

[0001] The present invention relates to a speech recognition apparatus,

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a speech recognition apparatus and an operation method thereof, and more particularly, to a speech recognition apparatus and a method of operation thereof capable of providing a user-customized speech guidance.

The speech recognition apparatus is a device for performing a speech recognition function.

On the other hand, home appliances such as an air conditioner, a washing machine, a vacuum cleaner, and the like used in a predetermined space such as a home or an office perform unique functions and operations according to user's operations.

On the other hand, for the operation of the home appliance, the user can use a remote control device such as a remote controller to avoid the inconvenience of directly operating a button or the like provided in the home appliance body or inputting it to the main body every time.

However, even when a remote controller is used, the user has to select and input an operation key for each function, which is inconvenient to use, and when the room is dark, separate illumination for identifying the remote controller and the operation key is required .

Therefore, there is an increasing research on a method of controlling a home appliance using speech recognition technology.

Prior Art 1 (Laid-Open Patent Publication No. 10-1999-00069703) discloses a remote controller for an air conditioner having an audio input unit and a signal processing unit to generate and transmit an operation signal according to voice recognition.

In the conventional art 2 (Japanese Unexamined Patent Publication (Kokai) No. 10-2006-0015092), an input voice signal is converted into a digital signal and text, and it is checked whether there is a control command in the database. And if there is no matching control command, extracts the keyword and controls each device in the air conditioner according to the associated control command.

However, there is a limit to the system resources that individual devices such as remote controllers and air conditioners can have. Especially, it is difficult to realize embedded modules that are installed in individual devices because it requires a high computation amount to recognize natural words rather than only a few words.

Therefore, the speech recognition technology of the related art 1 and the conventional art 2 has a limitation in recognizing and processing various natural language voice commands of users all over the world.

Therefore, there is a need for a method that can recognize and process natural language without restriction of the system resources of the individual devices, and can conveniently control the home appliance.

In addition, it is required to provide a guidance message by voice in correspondence with a voice input of a user or a specific event, thereby helping a poor user to conveniently use various functions and services.

It is an object of the present invention to provide a speech recognition apparatus and a method of operating the same that can provide a user-customized voice guidance.

It is an object of the present invention to provide a speech recognition apparatus and a method of operating the same that can identify a user and provide voice guidance similar to the tone of the identified user.

An object of the present invention is to provide a speech recognition apparatus capable of performing speech recognition in natural language by communication with a server system and an operation method thereof.

It is an object of the present invention to provide a speech recognition apparatus and a method of operating the same that can enhance convenience for users because non-skilled users can conveniently use various functions and services.

According to another aspect of the present invention, there is provided a method of operating a speech recognition apparatus, comprising: receiving a user's speech input signal through a microphone; transmitting speech data corresponding to the speech input signal to a speech recognition server system; Identifying a user based on the frequency and intensity of the speech input signal; receiving a response signal based on the speech input signal from the speech recognition server system; and outputting a speech announcement message corresponding to the received response signal And the voice guidance message output step may output a voice guidance message with a voice based on the voice data stored in the database in correspondence with the identified user, thereby providing a user-customized voice guidance.

According to another aspect of the present invention, there is provided a method of operating a speech recognition device, comprising: receiving a user's speech input signal through a microphone; identifying a user based on the frequency and intensity of the speech input signal; And outputting a voice guidance message based on the voice data stored in the database in correspondence with the identified user, thereby providing voice guidance similar to the voice of the identified user.

According to at least one embodiment of the present invention, it is possible to provide a user-customized voice guidance, thereby enhancing the user's convenience and emotional quality.

Further, according to at least one of the embodiments of the present invention, it is possible to identify the user and provide voice guidance similar to the tone of the identified user.

Further, according to at least one of the embodiments of the present invention, natural language speech recognition can be efficiently performed by communication with the server system.

Also, according to at least one of the embodiments of the present invention, non-skilled users can conveniently utilize various functions and services, thereby increasing convenience for the user.

In addition, according to at least one of the embodiments of the present invention, speech recognition performance can be improved by using data collected by product and individual.

Meanwhile, various other effects will be directly or implicitly disclosed in the detailed description according to the embodiment of the present invention to be described later.

1 illustrates a home network system according to an embodiment of the present invention.
2 illustrates a home network system according to another embodiment of the present invention.
3 is a perspective view illustrating a speech recognition apparatus according to an embodiment of the present invention.
4 is a front view (a) of the speech recognition apparatus and a sectional view (b) taken along line A1-A1 shown in (a).
5 is an enlarged view of a portion of FIG.
6A is a right side view of the voice recognition device.
Fig. 6B is a cross-sectional view of the grill taken at each part shown in Fig. 6A.
7 is a block diagram showing the control relationship between the main parts constituting the speech recognition apparatus.
8 is an exploded perspective view of the cover.
Fig. 9 shows a cover in which a window is removed.
10 is an exploded view of the state before the voice input PCB is coupled to the window support.
11 is a sectional view of the state in which the voice input PCB is coupled to the window support.
12 is a view schematically illustrating a smart home system including a voice recognition server system and a voice recognition apparatus according to an embodiment of the present invention.
13A is an example of a speech recognition server system according to an embodiment of the present invention.
13B is an example of a speech recognition server system according to an embodiment of the present invention.
FIG. 14 is a block diagram of an internal block diagram of a speech recognition apparatus according to an embodiment of the present invention. Referring to FIG.
15 is a flowchart illustrating an operation method of a speech recognition apparatus according to an embodiment of the present invention.
16 is a flowchart illustrating an operation method of a speech recognition apparatus according to an embodiment of the present invention.
17 is a flowchart illustrating an operation method of a speech recognition apparatus according to an embodiment of the present invention.
18 is a flowchart illustrating an operation method of a speech recognition apparatus according to an embodiment of the present invention.
FIG. 19 is a diagram for explaining an operation method of a speech recognition apparatus according to an embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, it is needless to say that the present invention is not limited to these embodiments and can be modified into various forms.

In the drawings, the same reference numerals are used for the same or similar parts throughout the specification.

The suffix " module " and " part " for components used in the following description are given merely for convenience of description and do not give special significance or role in themselves. Thus, " module " and " part " may be used interchangeably.

1 illustrates a network system according to an embodiment of the present invention.

A network system is a collection of devices that establish a network by mutual communication within a certain space such as a home or an office. As an example of such a network system, FIG. 1 shows a home network system built in a home.

Hereinafter, the device 1 exemplifies a voice recognition device (Hub) 1 for a communication network having an acoustic output function, but is not necessarily limited thereto. Depending on the viewpoint, the apparatus 1 may be referred to as an audio output device.

1, a network system according to an exemplary embodiment of the present invention includes an accessory 2, 3a, 3b, a gateway 4, an access point 7, Or a sound output device.

The accessory 2, 3a, 3b, the gateway 4, the access point 7 and / or the speech recognition device 1 can communicate with each other according to a predetermined communication protocol, Fi, Ethernet, Zigbee, Z-wave, Bluetooth, and the like.

Wi-fi is a trademark of the Wi-Fi Alliance. It is a term commonly used as a wireless communication technology. It is a term used for wireless communication between devices in accordance with the general rules defined by the WLAN standard (IEEE 802.11) Refers to a set of technologies that support LAN connectivity, device-to-device connectivity (Wi-Fi P2P), wireless LAN connectivity, and PAN / LAN / WAN configurations. Hereinafter, " Wi-Fi module " is defined as a wireless communication device based on Wi-Fi technology.

Ethernet is a networking technology according to the IEEE 802.3 standard, and is the most representative standard for local area network (LAN) hardware, protocols, and cables. Ethernet uses mainly CSMA / CD (Carrier Sense Multiple Access with Collision Detection) for data transmission. Hereinafter, " Ethernet module " is defined as a device that performs communication based on Ethernet technology.

Zigbee is a wireless network technology for configuring and communicating a private communication network using a small, low-power digital radio. It is a communication method according to the regulations defined in IEEE 802.15. It is small size, low power consumption, low cost, and it is widely used as a ubiquitous building solution such as home network. It is used in the near field communication market of intelligent home network, building, industrial automation, logistics, human interface, telematics, environmental monitoring and military.

The mobility protocol consists of a physical layer, a media access control (MAC) layer, a network layer, and an application layer. The physical layer and the MAC layer are defined in the IEEE 802.15.4 standard.

ZigBee Home Network Public Profile and ZigBee Smart Energy Profile are used as the application profiles. The RF4CE, a new off-the-shelf specification, defines a simple network stack for star topology and a solution for remote control of appliances. RF4CE uses 2.4GHz frequency band and provides security using AES-128.

Low power transmission rate is sufficient for long battery life and security, and it is suitable for periodic or intermittent data transmission or data transmission for simple signal transmission such as sensors and input devices. Applications include wireless lighting switches, household watt-hour meters, traffic management systems, and other personal and industrial devices that require near-field low-speed communication. The advantage is that it is relatively simple and inexpensive compared to other WPAN technologies such as Bluetooth and Wi-Fi. Hereinafter, the " occupancy module " is defined as a device that performs wireless communication based on the occupancy technology.

Z-wave is a wireless transmission scheme designed for devices requiring low power and low bandwidth, such as home automation and sensor networks, to provide reliable communication between one or more nodes and control units in a wireless network. Main purpose. JiWave is composed of physical layer, media access control (MAC) layer, transport layer, routing layer and application layer. It uses 9.6kbps and 40kbps (900MHz band, Europe: 869MHz, USA: 908MHz) , And a speed of 200kbps. Hereinafter, " geo-wave module " is defined as a device that performs wireless communication based on geo-wave technology.

The accessory 2 may be installed at any position desired by the user and may include various sensors such as a temperature sensor, a humidity sensor, a vibration sensor, a proximity sensor, and an infrared (IR) sensor. The information obtained by these sensors can be transmitted to the voice recognition device 1 via the network and vice versa that signals for the control of the sensors from the voice recognition device 1 are transmitted to the accessory 2 It is possible.

Also, the accessory 2 can be configured to be able to remotely control peripheral equipment located nearby. For example, the accessory 2 may include an originating device that emits an infrared signal in accordance with a control signal transmitted over the network.

On the other hand, the infrared sensor may include a transmitter for irradiating infrared rays and a receiver for receiving infrared rays irradiated from the transmitter when the infrared rays hit the object and are reflected.

The access point 7 is a device that relays the wireless device to a network so as to connect the home network with the Internet. The home appliance 5, the voice recognition device 1, the accessory 3b and the like may be connected to the access point 7 by wire (e.g., Ethernet) or wireless (e.g., Wi-Fi).

The gateway 4 is a device for connecting these networks to each other so that other networks can exchange information with each other. For example, it is possible to relay the accessories 2 and 3b and the access point 7 by converting a signal of a lengthwise (or gyro-wave) type received from the accessory 2 or 3b into a signal of a wifi type.

On the other hand, the home network system can be connected to the Internet through the access point 7, and can be connected to the server 8 providing the service through the Internet. A server (or a cloud) 8 is managed by a manufacturer, a seller or a service provider contracted with the manufacturer (2), (3a, 3b) and / . The server 8 stores software and data, which may be received from a home network. When there is a request from the speech recognition apparatus 1, the server 8 can transmit the stored software or data to the home network via the Internet.

The server 8 can exchange information with a mobile terminal such as a PC (personal computer) or a smart phone connected to the Internet. Information transmitted from the speech recognition device 1 or the accessories 2, 3a and 3b can be stored in the server 8 and these information can be transmitted to the mobile terminal 6 connected to the server 8 . The information transmitted from the mobile terminal 6 can also be transmitted to the voice recognition device 1 or the accessory 2, 3a, 3b via the server 8, It is also possible to control the recognition device 1 or accessories 2, 3a, 3b.

As a kind of mobile terminal 6, since a smart phone widely used recently provides a convenient UI based on a graphic, it is possible to control accessories 2, 3a and 3b through the UI, 2, 3a, 3b) can be processed and displayed. In addition, by updating an application installed in the smartphone, functions that can be implemented through the accessory 2, 3a, 3b can be expanded or changed. However, it is also possible to control the accessories 2, 3a, and 3b using the voice recognition device 1 alone or to process and display information received from the accessories 2, 3a, and 3b without using the mobile terminal 6 Do.

The communication between the voice recognition apparatus 1 and the accessory 2, 3a, 3b can be performed via the gateway 4 and the access point 7. Specifically, the signal output from the accessory 2, (Or information) is transmitted to the voice recognition device 1 via the gateway 4 and the access point 7 in turn. Conversely, the information output from the voice recognition device 1 is transmitted to the access point 7, (4) to the accessory (2, 3b) in turn. According to the embodiment, the communication between the accessory 2, 3a, 3b and the speech recognition device 1 is possible even when the network is disconnected from the Internet.

In addition to the accessories 2, 3a, 3b described above, various kinds of accessories may be provided according to the embodiment. For example, the accessory can be configured to include an air quality sensor that senses the quality of the air, a smart plug, a CT sensor, a nest temperature controller, a sleep sensor, and the like.

The accessory may be attached to the home appliance (5). For example, an accessory having a vibration sensor may be attached to a washing machine to detect vibrations generated during operation of the washing machine, and a signal output from the vibration sensor may be transmitted to the network according to the sensed vibration.

However, the accessory may be attached to a place other than the home appliance 5. For example, when an attempt is made to detect the opening and closing of a door in a residence, an accessory having a motion detection sensor (for example, an infrared ray sensor) may be attached to the wall to detect opening and closing operations of the door. If the opening and closing operation of the door in the dwelling is not detected for a long period of time, there is a possibility that the dwelling person has been changed, and such a situation may be notified to the predetermined mobile terminal 6.

Further, when the opening / closing operation of the refrigerator door is detected by using the accessory having the motion detection sensor and the opening / closing operation of the refrigerator door is not detected for a long time, there is a possibility that the inhabitant is changed. This situation can be notified to the predetermined mobile terminal 6.

In such embodiments, a signal transmitted over the network from the accessory may be received by the mobile terminal 6, and the application loaded on the mobile terminal 6 may analyze the received signal and transmit the signal to the home appliance 5 (For example, an unbalance of the washing machine) and door opening / closing information of the washing machine (for example, a warning indicating the abnormal operation of the washing machine, A notification that the door is not opened or closed, a request for confirming the occupant's personal identity, etc.) may be displayed through the display of the mobile terminal 6 or the speaker.

On the other hand, the speech recognition apparatus 1 may include a microphone (not shown), and may extract commands from the speech inputted through the microphone in accordance with the installed speech recognition program and perform control accordingly.

2 illustrates a home network system according to another embodiment of the present invention.

The home network system according to another embodiment of the present invention is different from the above embodiment in that the gateway 4 is not provided and the voice recognition device 1 also functions as the gateway 4 And the other features are substantially the same as in the case of the above-described embodiment.

The accessories 2 and 3b can communicate directly with the speech recognition device 1 without sharing the gateway 4 (see Fig. 1). Preferably, the accessory 2, 3b and the voice recognition device 1 communicate via the directivity method, in which case the accessory 2, 3b and the speech recognition device 1 may each be equipped with a directivity module .

3 is a perspective view illustrating a speech recognition apparatus according to an embodiment of the present invention. 4 is a front view (a) of the speech recognition apparatus and a sectional view (b) taken along line A1-A1 shown in (a). 5 is an enlarged view of a portion of FIG. 6A is a right side view of the voice recognition device. Fig. 6B is a cross-sectional view of the grill taken at each part shown in Fig. 6A. 7 is a block diagram showing the control relationship between the main parts constituting the speech recognition apparatus. 8 is an exploded perspective view of the cover. Fig. 9 shows a cover in which a window is removed. 10 is an exploded view of the state before the voice input PCB is coupled to the window support. 11 is a sectional view of the state in which the voice input PCB is coupled to the window support.

3 to 11, a speech recognition apparatus 1 according to an embodiment of the present invention may include a cover 10, a main body 40, a grill 20, and a base 30. The main body 40 is supported by the base 30 positioned at the lower side and the cover 10 can be coupled to the upper side of the main body 40.

The body (40) is disposed in the grill (20). Not all of the body 40 has to be placed completely within the grill 20, and the body 40 can be partially projected through the top of the grill 20, as in the embodiment. The grill 20 is formed with a plurality of through holes 20h and is formed in a vertically long cylindrical shape and surrounds the main body 40.

A porous filter (not shown) may be attached to the inner surface of the grill 20 so that dust does not enter the grill 20 through the through hole 20h. The filter may be made of a material having fine pores such as a mesh or a nonwoven fabric. The filter may be attached to the inner surface of the grill 20 by an adhesive member such as double-sided tape. The filter also serves to conceal the structure of the speaker (43, 44), the body case, and the like disposed in the grill (20) so as not to be seen from the outside through the through hole (20h).

3, the through hole 20h is formed in a part of the grill 20 and is omitted in the other parts. However, the through hole 20h is omitted in order to avoid complication of the drawing. And the sound outputted from the speakers 43 and 44 to be described later can be spread evenly through the through holes 20h in front, back, left, right, and all directions.

The cover 10 may include a window 11, a window support 12, a display 13, a printed circuit board 14 and a cover housing 15. The window 11, the window support 12, the display 13 and the display PCB 14 may be disposed within the cover housing 15.

4 to 5, the cover housing 15 is made of synthetic resin, is coupled to the upper portion of the main body 40, and has an opening 15h formed on the upper surface thereof. The cover housing 15 has a cylindrical shape and includes a side wall 151 defining an opening 15h at an upper end thereof and a partition plate (not shown) extending upward from the inside of the side wall 151, 152). The display PCB 14, the display 13, the window support 12, and the window 11 are disposed on the upper side of the partition plate 152.

The lower end 151a of the side wall 151 preferably contacts the upper end of the grill 20, but a fine clearance due to the clearance may exist between them. The lower end 151a of the side wall 151 overlaps at least a part with the upper end of the grill 20. [ There is a gap between the outer surface of the side wall 151 and the outer surface of the grill 20 but the outer surface of the side wall 151 and the outer surface of the grill 20 except for the above- It forms a continuous contour.

The upper end holding portion 153 extends downward from the lower end 151a of the side wall 151 and is engaged with the grill 20. The upper end holding portion 153 and the grill 20 are not engaged with each other by using a separate fastening member such as a bolt but the upper end holding portion 153 is inserted into the upper end opening of the grill 20 And it is preferably a force fitting type using the elasticity / restoring force of the grill 20 or the upper holding portion 153 itself.

The upper end holding portion 153 is positioned further inside than the lower end portion of the side wall 151 (i.e., the outer side surface of the cover housing 15 is recessed at the lower end 151a of the side wall 151, A surface facing the upper end of the grill 20 is formed at the lower end of the side wall 151 so as to extend from the outer surface of the side wall 151 to the upper end holding portion 153.

The cover housing 15 may have protrusions 154 protruding from the inner side surface of the side wall 151 and protrusion insertion grooves 418 may be formed in the front surface of the main body 40 to engage with the protrusions 154 . The protrusion 154 moves along the outer surface of the main body 40 while the cover housing 15 and the main body 40 are assembled with each other. When the protrusion 154 reaches the protrusion insertion groove 418, the cover housing 15 itself made of synthetic resin Is inserted into the projection insertion groove 418 by the elasticity of the projection.

The top surface of the upper end holding portion 153 abuts the inner surface of the grill 20, so that the shape of the upper end of the grill 20 is maintained. Particularly, when the grill 20 is made of a metal material, the grill 20 is deformed corresponding to the shape of the upper end holding portion 153, so that the upper end of the grill 20 corresponds to the upper end holding portion 153 . ≪ / RTI >

On the other hand, when the upper end holding portion 153 extends in an elliptical shape along the lower end 151a of the side wall 151, the metal plate is rolled to form a cylindrical grill 20 having a domed shape in cross section, The shape of the grill 20 is deformed into an elliptical shape corresponding to the shape of the upper end holding portion 153 and can be maintained in the deformed state.

As in the embodiment, a garden-like window 11 having a radius r is formed at a predetermined angle (indicated by? 1 in FIG. 6A, an acute angle, hereinafter also referred to as a "first angle") with respect to a predetermined horizontal plane. When the vector Vh obtained by orthogonalizing the normal vector Vs with respect to the upper surface of the window 11 to the horizontal plane is directed toward the static room, the form in which the window 11 is orthogonally projected in the horizontal plane, And has an ellipse having a long radius of r in the left-right direction. Therefore, in order to provide a sense of unity of the outline of the voice recognition apparatus 1, the cross section of the grill 20 is also preferably formed in a shape corresponding to the ellipse (that is, a shape in which the ratio of the minor axis to the major axis is cos? The shape of the cross section of the grill 20 can be maintained in a shape corresponding to the ellipse by forming the concave portion 153 in a shape corresponding to the ellipse. Here, the " corresponding form " includes not only the case where the shapes of the both shapes are completely matched with each other, but also the case where both shapes resemble each other (for example, In the following, the same meaning is defined.

Here, the angle? 1 at which the window 11 is tilted with respect to the horizontal plane is determined in consideration of the user's gaze in a general use environment, so that the voice recognition device 1 is placed on a cradle The angle of view of a general adult positioned in front of the voice recognition device 1 is set to be close to 90 degrees with the upper surface of the window 11, preferably about 20 degrees, It should not be.

On the other hand, the display panel 131 may be arranged to be inclined at a predetermined angle with respect to the horizontal so that the displayed screen faces the upper side of the front side, and is preferably inclined at the same angle? 1 as the window 11. The window support plate 121 to be described later is also inclined at the same angle as the display panel 131 (or the window 11).

6A and 6B, the upper end of the side wall 151 of the cover housing 15 is of a garden type having an outer diameter L1, and the outer diameter of the lower end 151f of the side wall 151 is a Is inclined at an angle? 2 (? 2 <? 1 or less,? 2 is also referred to as a "second angle") with a diameter of La in the lateral direction and a diameter of Lb in the forward and backward directions. Here, since the outer surface of the side wall 151 is inclined at a predetermined angle? 3 with respect to the vertical line, the end face S1 of FIG. 6B is orthogonal to the horizontal plane, and the end face S2 does not exactly coincide with the horizontal face , and if the value of? 3 is sufficiently small (preferably, 5 degrees or less), La has a value close to L1, and hence La = L1. Further, if the difference between? 1 and? 2 is sufficiently small (preferably, 5 degrees or less), Lb also has a value close to L1, and Lb = L1 is assumed below.

The angle θ3 is an angle formed by the outer surface of the side wall 151 and the vertical line. The angle θ3 may have a constant value over the entire outer surface of the side wall 151. However, It is also possible to vary.

6B, the grill 20 has an outer diameter of elliptical shape (L1> L2), a long diameter L1 in the left-right direction and a short diameter L2 in the front-rear direction. Here, as assumed above, when La = L1 and Lb = L1, L2 becomes L1 cos? 1. That is, the outer shape of the grill 20 having a regular view on the horizontal plane is an ellipse having a shape in which the diameter L2 in the front-rear direction is shorter than the straightness L1 in the lateral direction. When the window 11 is tilted and disposed, when the speech recognition device 1 is viewed from above, its external shape becomes elliptical in its entirety to provide a sense of unity.

The upper end holding portion 153 is completely inserted into the inside of the grill 20 and the upper end holding portion 153 is inserted into the upper portion of the grill 20, The voice recognition device 1 is not visible.

A positioning protrusion (not shown) may protrude from a lower end of the side wall 151 and a positioning groove may be formed at an upper end of the grill 20 to insert the positioning protrusion in a state where the grill 20 is properly positioned. have.

The window 11 may be disposed in the opening 15h of the cover housing 15. [ The window 11 is formed by processing a transparent plate having a constant thickness, and the side surface (or the outer peripheral surface) is orthogonal to the upper surface and the lower surface.

The inner surface of the cover housing 15 is formed so that a certain portion 151b extending downward from the upper end of the cover housing 15 is positioned in a direction in which the upper surface of the window 11 faces ). An upper end inner side surface 151a of the side wall 151 is a bar that defines the opening 15h. Hereinafter, the upper end inner side surface 151b of the side wall 151 is referred to as an opening limiting surface. The opening-defining surface 151b is formed in a cylindrical shape extending from the periphery of the opening 15h, and the window 11 is disposed on the inside surrounded by the opening-defining surface 151b. Preferably, the upper surface of the window 11 belongs to the same plane as the upper end of the cover housing 15 (or the plane to which the opening 15h belongs), so that the upper surface of the voice recognition device 1 is made up of one plane A sense of unity is created.

The opening-defining surface 151b is formed in a plane parallel to the vector Vs at any position. That is, even if the cover housing 15 is cut by an arbitrary plane parallel to the vector Vs, the opening defining surface 151b on the section is parallel to the vector Vs.

When the center of the window 11 and the center of the opening defining surface 151b are aligned along the vector Vs because the opening defining surface 151b is parallel to the side surface of the window 11, It is possible to maintain a constant gap g with the opening-defining surface 151b. This is because the gap g is maintained between the window 11 and the upper end of the cover housing 15 when the speech recognition device 1 is viewed from the top, The gap g is set to a minimum under the condition that the side surface of the window 11 does not interfere with the opening defining surface 151b when the window 11 is pressed to operate the contact switch desirable. The contact switch may be electrically connected to a circuit formed on the substrate to receive a user's button operation.

When the cover housing 15 is cut into an arbitrary vertical plane, the outer side surface of the side wall 151 is parallel to the normal vector Vs or gradually away from the normal vector Vs toward the lower side Can be achieved. When the cover housing 15 is injection-molded, the cover housing 15 is taken out from the first mold forming the side wall 151 vertically downward. Accordingly, in order for the cover housing 15 to be detached from the first metal mold, the outer surface of the side wall 151 should have the above-described shape.

On the other hand, in order to form the opening 15h on the upper surface of the cover housing 15, a separate second mold having a core inserted into the opening 15h is required. The cover housing 15 can be separated from the second mold by moving the second mold while the first mold is removed. At this time, the movement of the second mold is equal to the normal vector Vs Direction.

5 and 8, the display PCB 14 is disposed on the upper surface of the partition plate 152 to support the display 13 from the lower side. The display PCB 14 comprises a circuit which is electrically connected to the display 13 and the display 13 is connected to the circuit via a connector 132. On the top surface of the display PCB 14, four contact switches may be arranged in front, back, left, and right with respect to the display 13. [

The display PCB 14 may be provided with an NFC module 50d (see FIG. 7). The NFC module 50d may be disposed in the NFC mounting portion 146a formed on the second substrate arm 146 by enabling NFC communication. Near Field Communication (NFC) is a non-contact communication technology that uses the frequency band of 13.56 MHz as one of radio frequency identification (RFID) technologies. It is a next-generation LAN technology that is attracting attention due to its relatively low security and relatively low price because of its short communication distance. Since both data reading and writing functions are available, there is no need for a dongle (reader) that was previously required for RFID use. It is similar to conventional short range communication technologies such as Bluetooth, but it does not need to set up between devices like Bluetooth.

The display 13 is an apparatus for receiving an electric signal and displaying an image. The display 13 is connected to a circuit of the display PCB 14, and displays an image according to a control signal input through the circuit. The display 13 may include a display panel 131 and a connector 132 for connecting the display panel 131 to the circuitry of the display PCB 14 (see FIG. 8). The display panel 131 may be attached to the upper surface of the display PCB 14 by an adhesive member.

The display PCB 14 is connected to a main PCB (not shown) via a predetermined cable so that a control unit for controlling the display 13 is mounted on the display PCB 14 or the main PCB It is acceptable.

Various information can be displayed on the screen of the display panel 131. The controller 240 can control not only the operation of the display panel 131 according to the program stored in the memory 250 but also the overall operation of the electrical components constituting the voice recognition device 1. [ A user interface (UI) can be displayed through the display panel 131, and the interface is implemented by executing the program.

The interface may display playback information of the speakers 43 and 44. For example, various information such as a music reproduction / stop / selection menu, a reproduction status, a song name, an artist / record information, a lyrics, and a volume can be displayed.

When the voice recognition device 1 is equipped with the communication module 50, the interface can display information exchanged through the communication module 50. For example, the interface may display a menu for controlling the accessory 2, 3a, 3b in communication with the communication module 50, or may display a menu for controlling the accessories 2, 3a, 3b based on the information transmitted from the accessories 2, Information can be displayed. Specifically, information such as the network connection status of the communication module 50, temperature, humidity, brightness, and the like sensed by the sensor provided in the accessory 2 can be displayed through the interface. In addition, a menu for controlling the output of the speakers 43 and 44 may be displayed through the interface, for example, a menu for selecting a song or an album to be outputted through the speakers 43 and 44, (E.g., song title, album name, artist), the size of the volume being output, and the like.

The communication module 50 may include various communication modules. For example, the communication module 50 may include a Wi-Fi module 50a, a Bluetooth module 50b, a position module 50c, and an NFC module 50d.

On the other hand, operations on the menus displayed on the interface are possible through the operation unit 181. [

On the other hand, the operation unit 181 may include a contact switch. How the output signals of the respective contact switches are processed is determined by the program stored in the memory 250. For example, menus displayed left and right on the interface may be selected according to the operation signals of the first and second contact switches 181a and 181b, and the operation signals of the third and fourth contact switches 181c and 181d The menus displayed up and down on the interface may be selected.

The user can communicate with the Bluetooth module 50b using an external device such as a smart phone or a notebook computer, and various data such as music and images can be stored in the memory 250. [ In particular, the controller 240 can control the speakers 43 and 44 to output the music stored in the memory 250, and various functions such as music selection, playback, and stop can be implemented through the contact switch .

Referring to FIGS. 8 and 9, a substantially circular window support 12 may be disposed above the display 13. The window support 12 is an injection molding of a synthetic resin material, and is preferably formed of one component. An opening is formed in the window support 12, and a screen of the display 13 is exposed through the opening.

The window support 12 includes a window support plate 121 having an opening 12h formed at its center and having a window 11 on its upper surface, operation protrusions projecting downward from the window support plate 121, 122b, 122c, 122d projecting downwardly from the upper and lower support pins 121, 121, respectively.

The support bosses 122a, 122b, 122c, 122d may extend vertically downward. In this case, the support bosses 122a, 122b, 122c, and 122d may be disposed at the first angle? 1 relative to the horizontal plane, as in the case of the window 11, (90 -? 1) of? 1 with the window support plate 121, as shown in FIG.

9, each portion of the window support plate 121 is divided into a first region SE1 located at the rear side with respect to the opening 12h, a second region SE2 located at the front side, And a fourth region SE4 located on the right side.

At least one of the support bosses 122a, 122b, 122c, and 122d may be formed in each of the first region SE1 and the second region SE2. The four support bosses 122a, 122b, 122c, and 122d can be formed so that the window support plate 121 can be stably supported without being shaken. Of these, the first support boss 122a and the second support boss 122b, The third support boss 122c and the fourth support boss 122d may be formed in the second region SE2.

On the other hand, the window 11 is a circular transparent plate, which transmits the screen of the display 13, and the material is preferably acrylic. The user can see the screen displayed on the display 13 through the window 11. [ The entire area of the window 11 does not need to be transparent. Only the screen of the display panel 131 exposed through the opening 12h of the window support 12 can be seen without the window support 12 on the outer side of the voice recognition device 1 being visible. Only a predetermined region 11b at a position substantially corresponding to the opening 12h is transparent and the other region 11a may be colored opaque or translucent so as to be concealed or a film or the like may be adhered thereto.

The window 11 can be adhered to the upper surface of the window support plate 121 of the window support 12 using a double-sided tape or the like. The window 11 can be flexibly bent when pressure is applied within a certain limit due to the characteristics of the synthetic resin material. This deflection makes the operation of the contact switch more smooth. However, since the warp of the window 11 is elastic, it goes without saying that when the pressing pressure is removed, the window 11 is restored to its original shape.

3 to 5, the main body 40 is supported by the base 30 disposed at the lower side, and the upper end can be engaged with the cover housing 15. The body 40 may include a speaker case defining a cavity 49 inwardly and at least one speaker 43, 44 disposed within the cavity 49. [ In the embodiment, two speakers 43 and 44 are arranged in the upper and lower parts in the speaker case, a speaker 43 disposed on the upper side is a tweeter for outputting a high frequency band, It is a woofer that outputs the band.

  Referring to FIGS. 3 to 11, voice input PCBs 17 and 18 are installed on the cover 10. The voice of the user is input to the voice input PCBs 17 and 18. The voice input PCBs 17 and 18 are connected to the voice recognition PCB 40a disposed in the main body 40 in a circuit manner. The voice input PCBs 17 and 18 may be connected to the voice recognition PCB 40a through harness cables 17b and 18b. The voice input PCBs 17 and 18 convert the input voice of the user into sound wave signals that can be recognized by the voice recognition PCB 40a and the voice recognition PCB 40a inputs voice signals from the voice input PCBs 17 and 18 The user's voice can be recognized by analyzing the sound wave signal.

The audio input PCBs 17 and 18 are installed in the window support 12. A plurality of audio input PCBs 17 and 18 may be provided and the plurality of audio input PCBs 17 and 18 may be disposed symmetrically with respect to the openings formed in the window support 12. In this embodiment, the audio input PCBs 17 and 18 are provided in two and include a first audio input PCB 17 and a second audio input PCB 18.

The first audio input PCB 17 includes a first audio input PCB 17 positioned in front of the opening formed in the window support 12 and a second audio input PCB 17 located in the rear of the opening, 18). The first audio input PCB 17 is disposed in the second area SE2 of the window support plate 121 and the second audio input PCB 18 is disposed in the first area SE1 of the window support plate 121. [

The first voice input PCB 17 is disposed biased to the right with respect to the center of the window support plate 121 and the second voice input PCB 18 is disposed biased to the left with respect to the center of the window support plate 121.

On the other hand, one of the left and right sides of the main body 40 may be provided with a position control module 50c.

On the other hand, in the window support 12, PCB accommodating grooves 12a and 12b are formed on the upper surface opposite to the window 11 to accommodate the audio input PCBs 17 and 18. The voice input PCBs 17 and 18 do not protrude outside the PCB receiving grooves 12a and 12b when the voice input PCBs 17 and 18 are housed in the PCB receiving grooves 12a and 12b. That is, the PCB receiving grooves 12a and 12b are formed on the upper surface of the window support 12 so as to have a depth corresponding to the upper and lower thicknesses of the voice input PCBs 17 and 18, respectively. The upper surfaces of the audio input PCBs 17 and 18 coincide with the upper surface of the window support 12 when the audio input PCBs 17 and 18 are accommodated in the PCB accommodating grooves 12a and 12b.

The PCB receiving grooves 12a and 12b include a first PCB receiving groove 12a in which the first voice input PCB 17 is accommodated and a second voice loud power PCB inserting portion 12b in which the second voice input PCB 18 is accommodated ). The first PCB receiving groove 12a is disposed on the front side of the opening 12h formed in the window support 12 and the second PCB receiving groove 12b is disposed on the rear side of the opening 12h. The first PCB receiving groove 12a is formed in the second region SE2 of the window supporting plate 121 and the second PCB receiving groove 12b is formed in the first region SE1 of the window supporting plate 121. [

The first PCB receiving groove 12a is formed to be biased to the right with respect to the center of the window supporting plate 121 and the second PCB receiving groove 12b is formed to be biased to the left with respect to the center of the window supporting plate 121.

The window support 12 further includes positioning projections 12c and 12d protruding from the bottom of the PCB receiving grooves 12a and 12b. The voice input PCBs 17 and 18 are formed with positioning holes 17a and 18a into which the positioning projections 12c and 12d are inserted. One of the positioning projections 12c and 12d is formed at the corner of the rectangular PCB receiving grooves 12a and 12b and the positioning holes 17a and 18a are formed at the corners of the rectangular voice input PCB 17 and 18 . The operator inserts the positioning holes 17a and 18a into the positioning projections 12c and 12d when the audio input PCB 17 and 18 are received in the PCB receiving grooves 12a and 12b, 18 can be received at the correct positions of the PCB receiving grooves 12a, 12b.

The positioning projections 12c and 12d include a first positioning protrusion 12c in which the first PCB receiving groove 12a protrudes upward from the bottom and a second positioning protrusion 12b in which the second PCB receiving groove 12b protrudes upward And a second positioning protrusion 12d. The positioning holes 17a and 18a are formed in the first audio input PCB 17 and have a first positioning hole 17a into which the first positioning projection 12c is inserted and a second positioning hole 17b through which the second audio input PCB 18 And a second positioning hole 18a formed in the second positioning projection 12d for receiving the second positioning projection 12d.

Openings 12e and 12f are formed at the bottoms of the PCB receiving grooves 12a and 12b. The openings 12e and 12f serve as holes through which the harness cables 17b and 18b pass when the voice input PCBs 17 and 18 are connected to the voice recognition PCB 40a. The openings 12e and 12f include a first opening 12e formed at the bottom of the first PCB receiving groove 12a and a second opening 12f formed at the bottom of the second PCB receiving groove 12b .

The openings 12e and 12f constitute at least a part of the slits 121a and 121b formed in the first area SE1 and the second area SE2, respectively. The first PCB receiving groove 12a is disposed at a position shifted to the right of the second slit 121b formed in the second region SE2 and is connected to the third supporting boss 122c formed in the second region SE2, And is formed on the side of the fourth support boss 122d between the support bosses 122d. The second PCB receiving groove 12b is disposed at a position shifted to the left of the first slit 121a formed in the first region SE1 and has a first supporting boss 122a formed in the first region SE1, And is formed on the side of the first support boss 122a between the second support bosses 122b. Therefore, when the user presses the window 11, the window support plate 121 is easily resiliently deformed, so that the contact switch can be easily operated.

The openings 12e and 12f are formed to be wider in the front-back direction than the slits 121a and 121b. The voice input PCBs 17 and 18 are connected to the voice recognition PCB 40a via the harness cables 17b and 18b and connected to the voice recognition PCB 40a at the lower ends of the harness cables 17b and 18b 17c and 18c. The connectors 17c and 18c must pass through the openings 12e and 12f and down the window support 12 so that the connectors 17c and 18c are formed to be wider in the forward and backward direction than the slits 121a and 121b.

The harness cables 17b and 18b include a first harness cable 17b connecting the first audio input PCB 17 and the voice recognition PCB 40a and a second harness cable 17b connecting the second audio input PCB 18 and the voice recognition PCB 40a, And a second harness cable 18b connecting the first harness cable 18a and the second harness cable 18b. The connectors 17c and 18c include a first connector 17c coupled to the lower end of the first harness cable 17b and connected to the voice recognition PCB 40a and a second connector 17c coupled to the lower end of the second harness cable 18b And a second connector 18c connected to the voice recognition PCB 40a.

The voice passage holes 11c and 11d through which the user's voice passes are formed in the window 11 from the upper side of the window 11 to the lower side of the window 11. [ The voice passing holes 11c and 11d are formed in the first voice passing hole 11c formed in the front region with respect to the center of the window 11 and the second voice passing hole 11c formed in the rear region with respect to the center of the window 11, And includes a through hole 11d. The first voice passage hole 11c guides the voice to the first voice input PCB 17 and the second voice passage hole 11d guides the voice to the second voice input PCB 18. [

Depending on the embodiment, there is a fear that the appearance design may be reduced depending on the asymmetric structure. In the window 11 of this embodiment, two first decor holes 11e are formed on the side of the first sound passage hole 11c in the front-left and right direction with respect to the center of the window 11, Two second decoupling holes 11f are formed on the side of the second voice passage hole 11d from the rear side to the left and right with respect to the center of the second voice passage hole 11d.

In the window 11, the center portion is formed of the transparent region 11b, and the region other than the transparent region 11b is formed of the opaque region 11a.

It is preferable that the voice passage holes 11c and 11d are formed at positions corresponding to the voice input PCBs 17 and 18 so that the user's voice can be easily inputted to the voice input PCBs 17 and 18. [ The voice input PCBs 17 and 18 are formed with voice input holes 17d and 18d through which the voice passing through the voice pass holes 11c and 11d is inputted at a position corresponding to the voice pass holes 11c and 11d. It is preferable that a microphone (not shown) is provided on the bottom surface of the voice input PCB 17, 18 where the voice input holes 17d, 18d are formed. The microphone amplifies the voice inputted into the voice input holes 17d and 18d and inputs the amplified voice to the sound wave converting circuit of the voice input PCB 17 or 18. That is, the user's voice that has passed through the voice passage holes 11c and 11d from outside the window 11 is input to the voice input holes 17d and 18d formed in the voice input PCBs 17 and 18, Amplified and inputted to the sound wave converting circuit of the sound input PCB 17, 18 and converted into a sound wave signal readable by the voice recognition PCB 40a.

Between the window 11 and the window support 12, gaskets 17e and 18e are provided. The gaskets 17e and 18e do not enter the audio input holes 17d and 18d of the audio input PCBs 17 and 18 and enter the window 11 and the window support (12). To this end, the gaskets 17e and 18e are brought into close contact with the upper surface of the window 11, and the lower surfaces are brought into close contact with the upper surfaces of the voice input PCBs 17 and 18, respectively. The gaskets 17e and 18e are provided with communication holes 17f and 18f for communicating the voice passage holes 11c and 11d and the voice input holes 17d and 18d. Therefore, since all of the voice that has passed through the voice passage holes 11c and 11d is input to the voice input holes 17d and 18d through the communication holes 17f and 18f, it is possible to accurately recognize the voice.

Since the window 11, the window support 12 and the partition plate 152 are arranged in such a manner that the height of the front portion is lower and the height of the rear portion is higher than the height of the window 11, A space S (see FIG. 5) in which the voice recognition PCB 40a can be installed is secured between the rear portion of the upper surface of the main body 40, which is formed in a horizontal plane, and the partition plate 152.

9, a pair of fastening bosses 153c and 153d can be formed in the front portion of the upper end holding portion 153 of the cover housing 15 so as to be recessed rearward in the left and right directions, A pair of fastening bosses (not shown) 153b can be formed to be recessed forward.

3 to 9, a volume button 16 may be installed on the side wall 151 of the cover housing 15. [ The volume button 16 may include a dome 161 and an elastic pad (not shown). The dome 161 is made of synthetic resin and has a groove formed on one surface thereof and may include a volume increase operation portion 161a or a volume decrease operation portion 161b.

12 is a view schematically illustrating a smart home system including a voice recognition server system and a voice recognition apparatus according to an embodiment of the present invention.

12, a smart home system 10 according to an embodiment of the present invention includes a voice recognition device 1 having a communication module (not shown) and capable of communicating with other devices or connecting to a network, And a voice recognition server system 1100 including a plurality of servers for home appliance control.

On the other hand, the speech recognition apparatus 1 is a speech recognition apparatus.

In addition, the smart home system 10 according to an embodiment of the present invention may include a mobile terminal (not shown) such as a smart phone, a tablet PC, and the like.

The voice recognition device 1 may include a communication module therein to communicate with electronic devices in the smart home system 10 or outside.

The smart home system 10 according to an embodiment of the present invention may further include an access point (AP) device 7, It can access the Internet network and communicate with other devices.

The access point apparatus 7 can allocate a wireless channel according to a predetermined communication method to the electronic devices in the smart home system 10 and perform wireless data communication through the corresponding channel.

Here, the predetermined communication method may be a Wi-Fi communication method. Correspondingly, the communication module included in the voice recognition device 1 may be a Wi-Fi communication module, but the present invention is not limited to the communication method.

Alternatively, the voice recognition device 1 may include other types of communication modules or a plurality of communication modules. For example, the speech recognition device 1 may include an NFC module, a zigbee communication module, a Bluetooth (TM) communication module, and the like.

The voice recognition device 1 can be connected to a server included in the voice recognition server system 1100 or a predetermined external server or a user's mobile terminal through a wi-fi communication module or the like, Smart function can be supported.

The user can confirm the information on the voice recognition device 1 in the smart home system 10 or the voice recognition device 1 through the mobile terminal.

On the other hand, it may be inconvenient for the user to use the mobile terminal even when the user wants to control the voice recognition device 1 or confirm certain information in the home.

For example, it is more efficient to have a means of controlling the speech recognition device 1 in a different way if the user does not know the current location of the mobile terminal or is in another place.

The speech recognition apparatus 1 according to an embodiment of the present invention can receive a user's speech input and the speech recognition server system 1100 recognizes and analyzes the speech input of the user to control the speech recognition apparatus 1 can do.

Accordingly, the user can control the voice recognition device 1 without operating the mobile terminal or the remote control device.

At least some of the servers included in the voice recognition server system 1100 may be a server operated by a manufacturer of a voice recognition apparatus or a sales company or a server operated by a company entrusted with a service by a manufacturer or a sales company have.

13A is an example of a speech recognition server system according to an embodiment of the present invention.

13A, a speech recognition server system according to an embodiment of the present invention includes a voice server 1110 for receiving voice data from the voice recognition device 1, analyzing the received voice data to discriminate voice commands, . &Lt; / RTI &gt;

The voice server 1110 receives voice data from the voice recognition device 1, converts the received voice data into text data, and analyzes the text data to determine a voice command.

Further, the voice server 1110 can transmit a signal corresponding to the voice command discriminated to the predetermined server.

For example, the speech recognition server system according to an embodiment of the present invention receives a signal corresponding to the identified voice command from the voice server 1110, and generates a request signal corresponding to the determined voice command And a home appliance control server 1130 that transmits a control signal based on a request signal received from the linkage service server 1120 and the linkage service server 1120 to the voice recognition device 1. [

The voice recognition apparatus 1 can receive voice command input by the user and transmit voice data based on the received voice command input to the voice server 1110. [

The voice server 1110 includes an Automatic Speech Recognition (ASR) server 1111 that receives voice data from the voice recognition device 1 and converts the received voice data into text data, The speech recognition apparatus 1 receives the text data from the automatic speech recognition server 1111, analyzes the received text data to determine a voice command, and transmits a response signal based on the determined voice command to the voice recognition apparatus 1 (NLP) server 1112, and a signal including text corresponding to the response signal from the speech recognition apparatus 1, converts the text included in the received signal into speech data And a text to speech (TTS) server 1113 for transmitting the voice message to the voice recognition apparatus 1. [

The automatic speech recognition server 1111 may perform speech recognition on the speech data received from the speech recognition apparatus 1 to generate text data and transmit the text data to the natural language processing server 1112. [

The natural language processing server 1112 can analyze the text data received from the automatic speech recognition server 1111 according to a natural language processing algorithm to discriminate a voice command.

The natural language processing server 1112 can process a natural language that is a language that a person routinely uses according to a natural language processing algorithm, and can analyze an intent of the user. The natural language processing server 1112 can perform natural language processing on the text data received from the automatic speech recognition server 1111 to determine a voice command that matches the user's intention.

Accordingly, the natural language processing server 1112 can determine a voice command that matches the user's intention even if the user inputs a voice command in a normal use language.

The natural language processing server 1112 can transmit a signal corresponding to the natural language processing result, that is, a signal corresponding to the discriminated voice command to the linked service server 1120. [

The connection service server 1120 can receive a signal corresponding to the voice command determined by the natural language processing server 1112. [

The connection service server 1120 can perform the corresponding operation by communicating with the home appliance control server 1130 if the voice command is related to the voice recognition device 1. [

Alternatively, if the determined voice command is not related to the voice recognition device 1, the connection service server 1120 can communicate with the external service 1121 to perform a corresponding operation.

For example, if the determined voice command is a command for requesting information such as weather, stock, news, etc., the connection service server 1120 requests and receives the information from the server providing the service corresponding to the requested information .

The connection service server 1120 can transmit the received information to the voice server 1110 and the natural language processing server 1112 can transmit the received information to the voice recognition device 1. [

The connection service server 1120 may generate a request signal corresponding to the identified voice command and transmit the request signal to the home appliance control server 1130 if the voice command is related to the voice recognition device 1. [

The home appliance control server 1130 can transmit a control signal based on the request signal received from the linked service server 1120 to the voice recognition device 1. [

For example, when the music reproduction request of the speech recognition apparatus 1 is received, the household control server 1130 may transmit a control signal for music reproduction to the speech recognition apparatus 1. [

On the other hand, the speech recognition apparatus 1 can perform a corresponding operation according to the control signal received from the home appliance control server 1130.

Also, the speech recognition apparatus 1 may transmit a signal to the home appliance control server 1130 indicating that the operation is performed after performing the requested operation.

The home appliance control server 1130 receives a response signal to the control signal from the speech recognition device 1 and transmits processing result information corresponding to the response signal to the linkage service server 1120 .

The voice server 1110 can transmit a response signal including the processing result information to the voice recognition device 1. [

The voice server 1110 receives a signal including an output text corresponding to the processing result information from the voice recognition device 1, converts the received output text into voice data, To the recognizing device (1).

In this case, the response signal based on the voice command determined by the natural language processing server 1112 to be transmitted to the voice recognition device 1 may include the processing result information.

On the other hand, the speech recognition apparatus 1 can receive a response signal based on the determined voice command from the natural language processing server 1112. [ Here, the response signal may include text data of a response corresponding to the identified voice command.

For example, when the user inputs a voice command for requesting music reproduction, the response signal may include text data indicating that the music has been reproduced.

On the other hand, the speech recognition apparatus 1 can transmit a signal including the text corresponding to the received response signal to the text-to-speech conversion server 1113. Here, the signal including the text corresponding to the response signal may include an output text corresponding to the processing result information.

On the other hand, the text-to-speech conversion server 1113 can convert the text included in the received signal into voice data and transmit the voice data to the voice recognition device 1. [ The converted voice data may include a sound source file.

The voice recognition apparatus 1 can output a voice guidance message based on voice data received through a speaker.

On the other hand, the connection service server 1120 requests the home appliance control server 1130 of the state information of the voice recognition device 1 based on the signal corresponding to the determined voice command, 1130 may transmit the status information of the voice recognition device 1 to the linked service server 1120. [ When the state information of the voice recognition device 1 is not secured, the home appliance control server 1130 can request and receive the state information from the voice recognition device 1. [

On the other hand, when it is possible to support the determined voice command based on the state information of the voice recognition device 1, the connection service server 1120 transmits the voice command to the home appliance control server 1130 It is possible to transmit a corresponding request signal.

Alternatively, when it is impossible to support the determined voice command based on the status information of the voice recognition apparatus 1, the connection service server 1120 transmits the voice command to the natural language processing server 1112, It is possible to transmit a signal indicating the function.

In this case, the speech recognition apparatus 1 can request and receive the voice data to the text-to-speech conversion server 1113, and output a voice guidance message indicating that the voice data is not supported in the current state.

According to an embodiment, the voice server 1110 can determine whether or not the identified voice command can be supported. For example, the natural language processing server 1112 that has analyzed the intention of the user's voice command can determine whether or not the determined voice command can be supported.

In this case, when the determined voice command includes an instruction that can not be supported, the response signal based on the determined voice command transmitted by the natural language processing server 1112 is a voice command, May be a signal indicating that the function (1) is not supported.

The voice server 1110 and the voice recognition server system 1100 including the voice server 1100 according to an embodiment of the present invention can organically connect servers having various roles for natural voice processing.

The voice recognition apparatus 1 performs operations from reception of voice commands to preprocessing and server transmission, and the voice server 1110 can perform natural language processing such as voice / text conversion, intent analysis, and command identification.

By performing the natural language processing by the voice server 1110, it is possible to reduce the burden on the CPU, memory, and the like of the embedded module in the voice recognition apparatus.

Meanwhile, the connection service server 1120 can communicate with the external service and home appliance control server 1130 to perform operations based on the voice command of the user.

On the other hand, the voice recognition device 1 can receive voice data including a sound source file from the voice server 1110 and output a voice guidance message to the voice, so that the voice recognition device 1 can respond to voice input of the user through auditory feedback.

The voice recognition apparatus 1 receives the voice file from the voice server 1110 through streaming, and can reproduce and output the voice guidance message to the user. Accordingly, the speech recognition apparatus 1 need not store various sound source files.

Meanwhile, it is possible to link with various external services without colliding with other servers through the linked service server 1120. In addition, through the external service interworking server, it is possible to increase the success rate of the intention analysis by reflecting external information in the intention analysis.

The voice recognition server system 1100 according to an embodiment of the present invention assures compatibility and connectivity through a plurality of servers and utilizes the home appliance control server 1130 as a final control command, The collision between the home appliance control using the Wi-Fi communication and the home appliance control through the mobile terminal and the home appliance control by the voice input through the voice recognition device 1 can be prevented through the server 1130. [

The speech recognition server system 1100 according to an embodiment of the present invention can reduce a load on a specific server depending on a certain server through an organic connection between the servers, If there is a problem with the server, it can be easily responded to by linking with other servers having the same role.

In addition, a plurality of servers can be independently updated from time to time, which is advantageous for performance improvement.

13B is an example of a speech recognition server system according to an embodiment of the present invention.

The voice recognition server system illustrated in FIG. 13B improves the process of transmitting voice data for voice announcement message output to the voice recognition apparatus 1 by the voice recognition server system illustrated in FIG. 13A to improve the voice control response time will be.

Therefore, the speech recognition server system illustrated in FIGS. 13A and 13B can perform substantially the same operations other than the above-described differences, and the same portions will be briefly described below.

13B, a speech recognition server system according to an embodiment of the present invention includes a voice server 1110 for receiving voice data from the voice recognition device 1, analyzing the received voice data to discriminate voice commands, . &Lt; / RTI &gt;

In addition, the voice recognition server system according to an embodiment of the present invention may include a voice recognition server 1110 for receiving a signal corresponding to the voice command discriminated from the voice server 1110 and generating a request signal corresponding to the discriminated voice command And a home appliance control server 1130 that transmits a control signal based on a request signal received from the service server 1120 and the connection service server 1120 to the voice recognition device 1. [

The speech recognition server system illustrated in FIG. 13B is a system in which the speech server 1110 transmits speech data including processing result information based on the speech command to the speech recognition apparatus 1, without requesting the speech recognition apparatus 1. [ As shown in FIG.

The voice server 1110 includes an automatic voice recognition server 1111 that receives voice data from the voice recognition device 1 and converts the received voice data into text data, A natural language processing server 1112 for receiving the text data from the voice recognition device 1112 and analyzing the received text data to determine a voice command, and a speech recognition server 1112 for converting the response signal based on the voice command into voice data, And a text-to-speech conversion server 1113 for transmitting the text-

Also in this embodiment, the household appliance control server 1130 receives a response signal to the control signal from the speech recognition device 1, and transmits processing result information corresponding to the response signal to the connection service server 1120, As shown in FIG.

The linked service server 1120 can transmit the processing result information to the voice server 1110, and more specifically, to the natural language processing server 1112.

In this case, the speech data transmitted from the text-to-speech conversion server 1113 to the speech recognition apparatus 1 may include the processing result information.

The connection service server 1120 also requests the appliance control server for the status information of the voice recognition apparatus 1 based on the signal corresponding to the voice command discriminated, And can transmit the status information of the recognition device 1 to the linked service server 1120.

When the determined voice command can be supported on the basis of the state information of the voice recognition device 1, the connection service server 1120 transmits the voice command to the home appliance control server 1130 It is possible to transmit a corresponding request signal.

Alternatively, when it is impossible to support the determined voice command based on the status information of the voice recognition device 1, the connection service server 1120 transmits a voice message to the voice server 1110, A signal indicating that the signal is transmitted can be transmitted.

For example, the connection service server 1120 can transmit a signal to the natural language processing server 1112 to inform the natural language processing server 1112 of the unsupported function in the current state.

In addition, the natural language processing server 1112 transmits a signal indicating that it is an unsupported function in the current state to the text-to-speech conversion server 1113, and the text-to-speech conversion server 1113 generates corresponding voice data, It is possible to transmit it to the recognition device 1. [

The speech recognition apparatus 1 may receive the voice data from the text-to-speech conversion server 1113 and output a voice guidance message indicating that the voice data is not supported in the current state.

According to an embodiment, the voice server 1110 can determine whether or not the identified voice command can be supported. For example, the natural language processing server 1112 that has analyzed the intention of the user's voice command can determine whether or not the determined voice command can be supported.

In this case, when the determined voice command includes an instruction that can not be supported, the response signal based on the determined voice command transmitted by the natural language processing server 1112 is a voice command, May be a signal indicating that the function (1) is not supported.

When a speech announcement about the corresponding operation is requested to the text-to-speech conversion server 1113 for speech guidance at the earliest after the speech recognition apparatus 1 operates, a time difference may occur between the operation and the announcement.

However, according to the embodiment of the present invention, when the intention analysis is completed and the operation request is transmitted from the natural language processing server 1112, the text voice conversion server 1113 can provide the information to the text voice conversion server 1113 at the same time.

In addition, the text-to-speech conversion server 1113 can provide the announcement to the speech recognition device 1 on the basis of a time point when the home appliance control server 1130 issues a control command to the speech recognition device 1. [

Accordingly, the announcement can be made at the same time or immediately after the operation of the speech recognition apparatus 1.

According to the present embodiment, by directly connecting the natural language processing server 1112 and the text-to-speech conversion server 1113, the time difference between the control command and the announcement via the home appliance control server 1130 can be minimized.

13A and 13B, the speech recognition apparatus performing the hub function by speech recognition apparatus 1 has been exemplified and explained, but the present invention is not limited thereto. For example, the speech recognition device 1 can be employed in a robot cleaner, a refrigerator, a washing machine, a cooking device, a TV, a mobile terminal (smart phone, wearable device), a vehicle, a lighting device, have.

13A and 13B, the automatic speech recognition server 1111, the natural language processing server 1112, and the text-to-speech conversion server 1113 for speech recognition and processing are integrated Server.

In addition, according to an embodiment, the linkage service server 1120 and the home appliance control server 1130 may be configured as one integrated server.

According to the present invention, since the voice recognition device operates according to the voice input, the user does not need to operate the remote control device such as the remote controller, the mobile terminal, and the like, and the user convenience can be increased.

13A and 13B, the present invention recognizes a user's natural language voice command using a plurality of servers and performs a corresponding control operation, thereby enabling a voice recognition apparatus, a system resource of each server It is possible to efficiently recognize and process natural language.

FIG. 14 is a block diagram of an internal block diagram of a speech recognition apparatus according to an embodiment of the present invention. Referring to FIG.

14, a speech recognition apparatus 1 according to an embodiment of the present invention includes an audio input unit 220 for receiving a voice command of a user, a memory 250 for storing various data, A communication module 50 for communication, an output unit 290 for displaying predetermined information as an image or outputting audio information, and a control unit 240 for controlling overall operation.

The audio input unit 220 can receive external audio signals and user voice commands. To this end, the audio input unit 220 may include one or more microphones (MICs). In addition, the audio input unit 220 may include a plurality of microphones 221 and 222 in order to more accurately receive a user's voice command. The plurality of microphones 221 and 222 can be disposed at different positions and can acquire an external audio signal and process it as an electrical signal.

14 illustrates an example in which the audio input unit 220 includes two microphones including a first microphone 221 and a second microphone 222, the present invention is not limited thereto.

The audio input unit 220 may include a processor for converting analog sound into digital data, or may be connected to a processing unit to convert user input voice commands into data to be recognized by the controller 240 or a predetermined server.

Meanwhile, the audio input unit 220 may use various noise reduction algorithms for eliminating noise generated in the process of receiving a voice command of a user.

The audio input unit 220 may include a filter for removing noise from an audio signal received from each of the microphones 221 and 222 and an amplifier for amplifying and outputting a signal output from the filter, .

The memory 250 records various information necessary for the operation of the voice recognition device 1, and may include a volatile or nonvolatile recording medium. The storage medium stores data that can be read by a microprocessor, and includes a hard disk drive (HDD), a solid state disk (SSD), a silicon disk drive (SDD), a ROM, a RAM, a CD- Tape, floppy disk, optical data storage, and the like.

Meanwhile, the memory 250 may store data for voice recognition, and the control unit 240 may process the voice input signal of the user received through the audio input unit 220 and perform a voice recognition process.

Meanwhile, a simple speech recognition may be performed by the speech recognition apparatus 1, and a high-level speech recognition such as a natural language processing may be performed by the speech recognition server system 1100.

For example, when a wake up voice signal including a preset caller is received, the voice recognition apparatus 1 can be switched to a state for receiving a voice command. In this case, the speech recognition apparatus 1 performs only a speech recognition process up to whether or not a caller speech is input, and voice recognition for a subsequent user speech input can be performed through the speech recognition server system 1100.

Since the system resources of the speech recognition apparatus 1 are limited, complex natural language recognition and processing can be performed through the speech recognition server system 1100. [

Limited data may be stored in the memory 250. For example, in the memory 250, data for recognizing a wake up voice signal including a predetermined caller may be stored. In this case, the controller 240 can recognize a wake-up voice signal including a preset caller from the voice input signal received through the audio input unit 220.

On the other hand, the caller can be set by the manufacturer. For example, the " LG Hub " can be set to an alias.

The caller can be changed by the user.

The control unit 240 may control the voice recognition server system 1100 to transmit the voice command of the user inputted after recognizing the wake up voice signal to the voice recognition server system 1100 through the communication module 50. [

The communication module 50 can perform wireless communication with other electronic devices to exchange various signals. For example, the communication module 50 can communicate with electronic devices inside / outside the smart home system 10. [

In addition, the communication module 50 can communicate with the access point device 7, and can access the wireless Internet network through the access point device 7 to communicate with other devices.

The control unit 240 can also transmit the status information of the voice recognition device 1, the voice command of the user, and the like to the voice recognition server system 1100 or the like through the communication module 50.

Meanwhile, when a control signal is received through the communication module 50, the control unit 240 may control the voice recognition device 1 to operate according to the received control signal.

The output unit 290 may include a display 13 for displaying information corresponding to a command input by a user, a processing result corresponding to a command input by the user, an operation mode, an operation state, an error state,

According to an embodiment, the display 13 may be configured as a touch screen by forming a mutual layer structure with a touch pad. In this case, the display 13 may be used as an input device capable of inputting information by a user's touch in addition to the output device.

The output unit 290 may further include an audio output unit 291 for outputting an audio signal. The audio output unit 291 outputs a notification message such as a warning sound, an operation mode, an operation state, an error state, etc., information corresponding to a command input by the user, a processing result corresponding to a command input by the user, Audio can be output. The audio output unit 291 can convert an electric signal from the control unit 240 into an audio signal and output it. For this purpose, a speaker or the like may be provided.

The speech recognition apparatus 1 may further include an operation unit 181 for user input and a camera 210 capable of photographing a predetermined range around the speech recognition apparatus 1. [

The operation unit 181 includes a plurality of operation buttons, and can transmit a signal corresponding to the input button to the control unit 240.

The camera 210 photographs the surroundings of the voice recognition apparatus 1, the external environment, and the like.

For example, the camera 210 may include at least one optical lens and an image sensor (e.g., an optical sensor) configured to include a plurality of photodiodes (e.g., pixels) that are imaged by light passing through the optical lens. A CMOS image sensor, and a digital signal processor (DSP) that forms an image based on signals output from the photodiodes. The digital signal processor is capable of generating moving images composed of still frames as well as still images.

Meanwhile, the image captured by the camera 210 may be stored in the memory 250.

15 is a flowchart illustrating an operation method of a speech recognition apparatus according to an embodiment of the present invention.

Referring to Fig. 15, the speech recognition apparatus 1 can activate the speech recognition function by user input (S1510). The control unit 240 of the voice recognition device 1 can activate the microphones 221 and 222 according to user input.

For example, when one of the contact switches 181a, 181b, 181c, and 181d is activated, the voice recognition function can be activated.

Alternatively, the speech recognition apparatus 1 can automatically activate the speech recognition function in accordance with the setting for activating the speech recognition function.

For example, when the power is turned on, the speech recognition apparatus 1 can automatically activate the microphones 221 and 222 and activate the speech recognition function.

The speech recognition apparatus 1 according to an embodiment of the present invention can provide a user experience (UX) considering various situations that may occur during a speech recognition process and a home appliance control process.

The controller 240 controls the display 13 to provide visual information corresponding to each step of the speech recognition process and the home appliance control process.

In addition, the controller 240 controls the audio output unit 291 to provide auditory information corresponding to each step of the speech recognition process and the home appliance control process.

 The display 13 can display the operation mode, the current state, and the setting items of the voice recognition device 1 in various visual images. The display 13 can display the operation mode, the current state, and the setting items of the voice recognition device 1 in a character, number, and symbol, and can display a graphic image such as an icon.

Also, the display 13 can display corresponding information for each process of voice input.

Referring to FIG. 15, when the voice recognition function is activated, the display 13 lights and displays a microphone icon, and the control unit 240 displays a wake-up voice signal input (Step S1515). &Lt; / RTI &gt;

On the other hand, if the voice recognition function is disabled, the display of the microphone icon may be terminated.

In addition, the audio output unit 291 may output a voice guidance message for guiding the voice including the caller. For example, if the caller is set to " LG Hub ", the audio output unit 291 can output a voice guidance message such as "

On the other hand, when a voice input is received from the user while the microphones 221 and 222 of the voice recognition device 1 are activated, the microphones 221 and 222 receive the input voice and transmit it to the control unit 240 .

On the other hand, when a wake-up voice signal including a predetermined call word is received through the microphones 221 and 222 (S1520), the voice recognition apparatus 1 receives a voice command word for receiving a voice command (S1525). When a wake-up voice signal including a predetermined call word is received through the microphones 221 and 222 in step S1520, the control unit 240 can control the entry into the command standby mode (step S1525) .

If the voice recognition apparatus 1 always waits for a natural language command, the power consumed in the voice recognition function, the CPU occupation rate, and the load on the server of the voice recognition server system are burdensome.

Therefore, the speech recognition apparatus 1 can transmit only the voice signal incoming from the command standby state to the server.

In addition, the speech recognition apparatus 1 gives a condition at a time of waiting for a command, and when receiving a command within a time, it can receive a sound and deliver it to the server until the completion of the command.

According to one aspect of the present invention, the speech recognition apparatus 1 performs only a speech recognition process up to whether a caller speech is input, and the speech recognition for a subsequent user speech input can be performed through the speech recognition server system 1100 have.

Alternatively, the determination as to whether or not the caller voice is input may be performed in duplicate in the voice recognition apparatus 1 and the voice recognition server system 1100. [

Meanwhile, the caller may be set by a manufacturer, and another caller may be set for each voice recognition device. For example, it can be set to "LG Hub". The caller can be changed by the user.

Meanwhile, the control unit 240 may control the voice recognition server system 1100 to transmit a user's voice command, which is input after recognizing the wake up voice signal, through the communication module 50 (S1530).

On the other hand, waiting indefinitely for a user's voice command is wasteful of system resources and does not match the intention of a user who does not input a command. Therefore, in the command standby mode, only voice commands input during a predetermined elapsed time can be set to be processed ).

In this case, the control unit 240 may control the voice server 1110 to transmit voice data including voice commands received within the predetermined time elapsed time (S1530).

On the other hand, if the voice command is not received within the preset elapsed time (S1527), the controller 240 can control to switch to the wake-up signal standby mode again.

According to an embodiment, if the communication module 50 does not receive a response signal based on the voice command from the voice server 1110 within the first time, the audio output unit 291 sends a voice guidance message to wait Can be output. In this case, if the communication module 50 does not receive a response signal based on the voice command even within the second time after the first time, the audio output unit 291 outputs a voice request for re- A guidance message can be output.

If the communication module 50 does not receive the response signal based on the voice command even within the second time, the control unit 240 can control to switch to the command standby mode.

The speech recognition apparatus 1 according to the embodiment of the present invention can provide not only the operation of the output unit 290 but also predetermined information to the user in other ways as well.

For example, when a voice command is received within a preset elapsed time, the voice recognition apparatus 1 according to an embodiment of the present invention performs a feedback operation corresponding to reception of a command to inform the user that the command has been received .

According to an embodiment of the present invention, when the voice command is received within the preset elapsed time (S1527), the control unit 240 notifies the user that the voice recognition apparatus 1 has received the command, So as to perform an operation.

In this case, the controller 240 can perform the predetermined operation under the control of the controller 240. For example, it is possible to output a predetermined sound.

Further, when the user inputs a voice command to the voice recognition apparatus 1, since most of the cases are intended to operate the voice recognition apparatus 1, a predetermined sound is output in advance and a voice command corresponding to the voice command So that the user can respond to the voice command more quickly.

12 to 14, the voice server 1110 and the voice recognition server system 1100 including the voice server 1110 recognize and process voice data including voice commands received from the voice recognition device 1 can do.

Accordingly, when the communication module 50 receives the control signal based on the voice command from the home appliance control server 1130 (S1540), the control unit 240 controls the voice recognition device 1 to operate in accordance with the received control signal. (S1545).

When the communication module 50 receives a response signal based on the voice command from the voice server 1110 (S1550), the control unit 240 controls the audio output unit 1110 to output a voice guidance message corresponding to the received response signal (Step S1570).

The control unit 240 requests the voice server 1110 to send voice data in step S1565 and transmits the voice data requested by the voice server 1110 to the voice server 1110 in step S1560. It is possible to control to receive voice data.

On the other hand, according to the result of speech recognition, a signal corresponding to the speech recognition failure can be received from the speech server 1110 through the communication module 50. [ In this case, the control unit 240 may control the audio output unit 291 to output the voice guidance message requesting re-input of the voice command, and control to switch to the command standby mode again.

In accordance with the discrimination of supportability of the discriminated voice command, a signal indicating that the voice command is not supported from the voice server 1110 through the communication module 50 can be received (S1535) . In this case, the control unit 240 may control the audio output unit 291 to output a voice guidance message indicating that the voice command is related to the unsupported function (S1537).

In this case, the control unit 240 can control to switch to the wake-up signal standby mode.

Alternatively, the control unit 240 may control to switch to the command standby mode again.

According to the present invention, visual information and / or voice guidance messages suitable for each situation are provided, so that it is possible to accurately grasp at which stage a voice control command is not normally controlled. In addition, the degree of freedom of commands for controlling the product is high, and usability is improved.

16 is a flowchart illustrating an operation method of a speech recognition apparatus according to an embodiment of the present invention.

Referring to FIG. 16, the speech recognition apparatus 1 can receive a user's voice input signal through the microphones 221 and 222 (S1610).

In this case, the received voice input signal may include a predetermined voice call for voice processing of the voice recognition device 1, or may include a voice command of the user input in the voice standby mode after recognizing the caller.

The audio input unit 220 may include a processing unit for converting the analog voice into digital data or may be connected to the processing unit to convert the user input voice into data to be recognized by the control unit 240 and the voice recognition server system 1110.

 Meanwhile, the audio input unit 220 may include a filter for removing noise from an audio signal received from the microphones 221 and 222, and an amplifier for amplifying and outputting a signal output from the filter.

The communication module 50 of the voice recognition apparatus 1 may transmit the voice data corresponding to the voice input signal to the voice recognition server system 1110. [

On the other hand, the speech recognition apparatus 1 can identify the user based on the frequency and the intensity of the speech input signal, separately from the transmission of speech data (S1620) (S1630).

The voice of a person can be varied in frequency by sex and age. Also, the frequency and intensity of voice may be different for each individual.

For example, in the case of men, the basic frequency of voice becomes lower than 10 in 20s, and the stable state can be maintained until 40s. In the case of females, a gradual decrease in the fundamental frequency is shown with increasing age from the teenage to the thirties, but it can be markedly decreased after the fifties. Infants under two years of age have a high fundamental frequency of 300 to 500 Hz. Infants and children from 3 to 14 years of age can have a fundamental frequency of 250 to 300 Hz.

Therefore, the speech recognition apparatus 1 can extract frequency and intensity information from the user's voice, and can use it as user recognition and user's tone color data.

To this end, the speech recognition apparatus 1 collects the user's speech signal in advance, analyzes frequency and intensity of the collected speech signal, discriminates the user's tone color, 250).

That is, in the speech recognition apparatus 1 according to the embodiment of the present invention, a user-specific tone color database based on the collected speech signals can be constructed.

The database may store natural language sound files included in the voice data.

In addition, the database may store voice data classified according to one or more classification factors.

For example, the database may classify and store voice data for individual users, and the analyzed voice data may be stored based on frequency and intensity analysis of the voice signal.

The tone color data may include frequency and intensity data of the identified user's voice.

In addition, the tone color data may further include at least one of pitch, vibration rate, and shimmer data of the voice of the identified user.

The speech recognition apparatus 1 may perform machine learning such as deep learning on the received speech input signal and the memory 250 may store data used for machine learning, Can be stored.

Deep Learning, a type of machine learning, is a multi-level, deep learning process based on data.

Deep learning can represent a set of machine learning algorithms that extract key data from multiple sets of data as they step up.

The deep learning structure may include an artificial neural network (ANN). For example, the deep learning structure may include a deep neural network (DNN) such as CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), DBN .

The deep running structure according to the present invention can utilize various known structures. For example, the deep learning structure according to the present invention may be a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), a Deep Belief Network (DBN), or the like.

Recurrent Neural Network (RNN) is widely used in natural language processing, etc., and it is possible to construct an artificial neural network structure by stacking layers every moment with an effective structure for time-series data processing that varies with time .

Deep Belief Network (DBN) is a deep-run structure consisting of multiple layers of deep-running RBM (Restricted Boltzman Machine). The Restricted Boltzman Machine (RBM) learning is repeated, and a DBN (Deep Belief Network) having a corresponding number of layers can be constituted by a certain number of layers.

CNN (Convolutional Neural Network) is a model that simulates a person's brain function based on the assumption that when a person recognizes an object, it extracts the basic features of the object, then undergoes complicated calculations in the brain and recognizes the object based on the result to be.

On the other hand, the artificial neural network learning can be done by adjusting the weight of the inter-node interconnections (adjusting the bias value if necessary) so that the desired output is obtained for a given input. Also, the artificial neural network can continuously update the weight value by learning. Back propagation can be used for artificial neural network learning.

On the other hand, an artificial neural network may be installed in the speech recognition apparatus 1, and a machine learning based user recognition in which a received speech input signal is used as input data, can do.

The control unit 240 may include an artificial neural network such as a CNN (Convolutional Neural Network), an RNN (Recurrent Neural Network), a DBN (Deep Belief Network), and a Deep Neural Network You can learn.

As the machine learning method of the artificial neural network, both unsupervised learning and supervised learning can be used.

The control unit 240 may control to update the post-learning tone recognition artificial neural network structure according to the setting.

On the other hand, the voice recognition apparatus 1 can receive voice input signals uttered by the user and transmit the voice data based on the received voice input signals to the voice server 1110, 1) (S1620).

Meanwhile, the voice recognition server system 1110 can recognize the voice input of the user by analyzing the received voice data.

The voice server 1110 of the voice recognition server system 1110 may convert the voice data into text data. The text data conversion of the voice data can be performed in the automatic voice recognition server 1111.

The voice server 1110 may analyze the text data and recognize a user command included in the voice data. The natural language processing server 1112 of the voice server 1110 can perform the natural language processing on the first text data by using the natural language processing algorithm to discriminate the user command corresponding to the user's intention.

The voice recognition server system 1110 can transmit a response signal based on the voice input signal to the voice recognition device 1. [ Accordingly, the speech recognition apparatus 1 can receive a response signal based on the speech input signal from the speech recognition server system 1110 (S1640).

The response signal may include text data corresponding to the speech recognition result of the speech input signal, and the control unit 240 may map the text data to the tone color data stored in the database corresponding to the identified user (not shown).

The voice recognition apparatus 1 can output a voice guidance message corresponding to the received response signal through the audio output unit 291 (S1650).

The control unit 240 may control to output the voice guidance message with voice based on the tone color data stored in the database corresponding to the identified user.

Currently, many speech recognition devices are in a mechanical mono tone, and the speech recognition device 1 outputs a text-to-speech (TTS) voice guidance message.

As a result, the user can feel a sense of rejection in the machine sound, and it is difficult to feel that the user is conversing.

However, the voice recognition device 1 may analyze the tone of the user inputted through the microphones 221 and 222, treble and bass (frequency) and intensity, and re-process the analyzed data.

The speech recognition apparatus 1 can use a speaker recognition technology to map a speech frequency (low frequency and high frequency) and intensity information to stored data such as STT (Speech To Text) data in a database of a speaker.

Depending on the embodiment, the mapping data may be passed to the speech recognition server system 1110, which may store deep running and store big data. In this case, the speech recognition apparatus 1 can receive the tone color data of the user from the speech recognition server system 1110, receive the sound source file to which the user's tone color data is applied, and output the voice guidance message based thereon have.

The speech recognition apparatus 1 according to the embodiment of the present invention can output the voice guidance message similar to the tone and tone of the user beyond the rigid mechanical tone by applying the previously stored tone color data.

In other words, the present invention can provide customized voice guidance familiar to the user by reusing speech data of the speaker not only for voice recognition but also for user recognition.

Further, as the user uses the voice recognition function more frequently, the voice recognition apparatus 1 or the voice recognition server system 1100 can continuously improve the performance by using deep learning as the accumulated data and accumulated data.

The speech recognition apparatus 1 according to the embodiment of the present invention may include two or more speakers 43 and 44 for outputting different frequency ranges.

For example, the speaker 43 disposed on the upper side may be a tweeter for outputting a high frequency band, and the speaker 44 disposed on the lower side may be a woofer for outputting a low frequency band.

Based on the tone color data, the control unit 240 classifies the output speech into a high-frequency band and a low-frequency band, analog-encodes it, and outputs a high-frequency band sound through the high- And controls the low-band speaker 44 to output low-band speech.

Therefore, the speech recognition apparatus 1 according to the embodiment of the present invention can reproduce two speakers 43 and 44 again by using the deep learning and the tone data (frequency and intensity) of the user accumulated on the basis of the big data (Tone and tone) of the user as much as possible.

In addition, the speech recognition apparatus 1 according to the embodiment of the present invention can reflect the local dialect of the user or provide the individual TTS voice data differentiated from other users.

Accordingly, the user can own or share his own TTS voice data. Also, the data reflecting the tone of the user may be sent to another device or other person for utilization.

17 is a flowchart illustrating an operation method of a speech recognition apparatus according to an embodiment of the present invention.

Referring to FIG. 17, the speech recognition apparatus 1 can receive a user's voice input signal through the microphones 221 and 222 (S1710).

In this case, the received voice input signal may include a predetermined voice call for voice processing of the voice recognition device 1, or may include a voice command of the user input in the voice standby mode after recognizing the caller.

Alternatively, the received voice input signal may include voice of a user who has uttered a voice in the course of life, regardless of the voice command of the user.

The audio input unit 220 receives the user's voice through the microphones 221 and 222 and converts the input voice into data.

The control unit 240 may extract the frequency and intensity data from the voice data according to the speaker recognition technique to distinguish the speaker (S1720).

The speech recognition apparatus 1 can identify the user based on the frequency and the intensity of the speech input signal (S1720), and can output a speech announcement message based on the tone color data stored in the database corresponding to the identified user (S1730).

For example, when the user receives a voice saying home speech, the speech recognition apparatus 1 can speak a greeting with a familiar voice reflecting the tone of the user.

Further, the speech recognition apparatus 1 may recommend guidance of music reproduction, control of an air conditioner, and the like.

In this embodiment, the control unit 240 also classifies the output speech into a high-frequency band and a low-frequency band on the basis of the tone color data, analog-encodes the output speech, It is possible to output a high-frequency band voice and to output a low-frequency band voice through the low-frequency band speaker 44. [

The tone color data may include frequency and intensity data of the identified user's voice.

In some embodiments, the speech recognition apparatus 1 may perform a speech recognition process on its own.

In this case, the control unit 240 can discriminate a voice command included in the voice input signal, and transmits text data corresponding to the determined voice command to the memory 250 or voice recognition After mapping the voice data stored in the database of the server system 1100 to voice data, the voice guidance message to which the user's voice is applied can be controlled to be generated.

The control unit 240 maps the voice data to the tone color data stored in the database, fetches the voice for the character required for voice response from the database, and performs analog (ANALOG) encoding on the high and low tones from the imported data .

The control unit 240 can provide the REAL TTS voice service by controlling the treble to be reproduced through the tweeter speaker 43 and the low tone to be reproduced through the woofer speaker 44. [

18 is a flowchart illustrating an operation method of a speech recognition apparatus according to an embodiment of the present invention. FIG. 18 includes a process of collecting user's voice in parallel with and independent of speech recognition and databaseing.

Referring to FIG. 18, the voice recognition apparatus 1 can enter the voice acquisition mode by setting or agreement of the user (S1810). Here, the voice acquisition mode generates text data by collecting voice data and analyzing the frequency and intensity of the collected voice. Therefore, the voice acquisition mode is also referred to as STT (Speech To Text) data acquisition mode or Only STT DATA acquisition mode. .

Such a voice acquisition mode can be performed separately from the voice recognition process, and thus a fast voice database can be constructed.

Meanwhile, in order to collect voice independently of the voice recognition process, the microphones 221 and 222 may be activated independently of the voice recognition process (S1815).

That is, the microphones 221 and 222 can operate in the voice recognition standby state (mic always on state) without the voice recognition device 1 being called or the predetermined button being operated.

The voice recognition device 1 can collect voice data using the microphones 221 and 222 that are always turned on. Accordingly, the speech recognition device 1 can listen to and collect all the conversations of the family members in the installed space.

On the other hand, the speech recognition device 1 performs speaker recognition technology software processing on the collected voice (S1820), divides the voice of the person (S1825), and stores the voice in the database (S1840).

In this case, the speech recognition apparatus 1 may map the frequency and intensity data of the user's voice to STT data that has been collected and analyzed (S1830), and may store the STT data in the database (S1840).

On the other hand, in the database, only data frequently referred to by the user can be stored, or the weighted data can be stored based on the frequency of the vocabulary used by the user.

Also, the database may receive additional data related to the word in the external network based on the word having a high frequency.

Particularly, if a word used in a voice recognition function by a user is a frequently used word in a database and a lot of information is stored in the database, it is possible to provide a voice guidance specialized for the user and provide a voice guidance service reflecting the user's preference.

For example, if the database contains a lot of voice for sports, the voice recognition device 1 can develop into a voice recognition device 1 specialized for sports.

For example, when a user utteres a lot of words related to 'baseball', the baseball-related data frequently referred to by the user can be received from the voice recognition server system 1100 and other servers (S1850) Using the data, detailed information related to 'baseball' may be provided to the user (S1860).

In this case, the control unit 240 encodes (S1870) and outputs the voice of the information to be provided to the tweeter speaker 43 and the low sound to the woofer speaker 44, thereby providing the REAL TTS voice service provision (S1880).

On the other hand, the speech recognition apparatus 1 or the speech recognition server system 1100 can continuously improve the performance by using the non-operating time as a time for performing deep learning.

FIG. 19 is a diagram for explaining an operation method of a speech recognition apparatus according to an embodiment.

The database according to the embodiment of the present invention may be stored in the storage medium of the speech recognition apparatus 1 or the speech recognition server system 1100. [

Referring to FIG. 19, data collected from the members of the living space in which the voice recognition apparatus 1 is placed and used may be stored in the database. For example, the database may store data for family members A, B, C, and D.

The voice recognition device 1 or other electronic device can use the necessary person's data among the persons A, B, C and D stored in the database.

For example, the speech recognition apparatus 1 recognizes a speaker on a received speech, and recognizes that the present A and C are in a predetermined space.

In this case, the speech recognition apparatus 1 can retrieve the data for A and C from the database.

Thereafter, when the person C inputs a voice command " Tell me the weather today ", the voice recognition apparatus 1 itself performs a voice recognition process by itself or transmits voice data to the voice recognition server system 1100, It is possible to receive the recognition result.

After receiving the recognition or recognition result of the voice command, the voice recognition device 1 can output the voice guidance message in which the voice of the person C and the voice tone similar to the voice tone are applied, in response to the voice recognition result.

In the present embodiment, the speech recognition apparatus 1 can guide today's weather with a tone similar to that of human C with tone.

Also, a database can be used for other electronic devices such as an air conditioner, a washing machine, a refrigerator, and a mobile terminal.

For example, the refrigerator can recognize the speaker as to the received voice, and recognize that the present B and D are present in the predetermined space.

In this case, the refrigerator can retrieve data for B and D from the database.

Thereafter, when the person B inputs a voice command " Tell me a schedule today ", the refrigerator itself performs a natural speech recognition process or transmits voice data to the voice recognition server system 1100 and receives a voice recognition result .

After receiving the recognition or recognition result of the voice command, the refrigerator can output the voice guidance message to which the voice of the person B and the tone similar to the tone are applied, corresponding to the voice recognition result.

In this embodiment, the refrigerator can guide today's schedule with a tone similar to that of human B's tone.

According to at least one embodiment of the present invention, it is possible to provide a user-customized voice guidance, thereby enhancing the user's convenience and emotional quality.

Further, according to at least one of the embodiments of the present invention, it is possible to identify the user and provide voice guidance similar to the tone of the identified user.

Further, according to at least one of the embodiments of the present invention, natural language speech recognition can be efficiently performed by communication with the server system.

Also, according to at least one of the embodiments of the present invention, non-skilled users can conveniently utilize various functions and services, thereby increasing convenience for the user.

In addition, according to at least one of the embodiments of the present invention, speech recognition performance can be improved by using data collected by product and individual.

The voice server, the voice recognition server system, and the voice recognition apparatus according to the present invention are not limited to the configuration and method of the embodiments described above, but the embodiments may be modified in various ways, All or a part of the above-described elements may be selectively combined.

Meanwhile, the speech recognition method, the voice server, the speech recognition server system, and the method of operating the speech recognition apparatus according to the embodiments of the present invention can be implemented as a code that can be read by a processor on a processor-readable recording medium. The processor-readable recording medium includes all kinds of recording apparatuses in which data that can be read by the processor is stored. Examples of the recording medium that can be read by the processor include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and may also be implemented in the form of a carrier wave such as transmission over the Internet . In addition, the processor-readable recording medium may be distributed over network-connected computer systems so that code readable by the processor in a distributed fashion can be stored and executed.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It should be understood that various modifications may be made by those skilled in the art without departing from the spirit and scope of the present invention.

Speech Recognition Device: 1
Voice recognition server system: 1100
Voice server: 1110
ASR server: 1111
NLP server: 1112
TTS server: 1113
Linked service server: 1120
Home appliance control server: 1130

Claims (11)

Receiving a voice input signal of a user through a microphone;
Transmitting voice data corresponding to the voice input signal to a voice recognition server system;
Identifying a user based on frequency and intensity of the speech input signal;
Receiving a response signal based on the speech input signal from the speech recognition server system; And
And outputting a voice guidance message corresponding to the received response signal,
Wherein the voice guidance message output step outputs the voice guidance message with voice based on the tone color data stored in the database corresponding to the identified user. The method according to claim 1,
The voice announcement message output step may include:
The output voice is divided into a high-frequency band and a low-frequency band and is analog-encoded based on the tone data, a high-frequency band sound is output through a high-frequency band speaker provided in the speech recognition device, and a low- Wherein the speech recognition apparatus comprises: The method according to claim 1,
Wherein the tone color data includes frequency and intensity data of the identified user's voice. The method of claim 3,
Wherein the tone color data further includes at least one of a pitch, a jitter, and shimmer data. The method according to claim 1,
Wherein the response signal includes text data corresponding to a speech recognition result for the speech input signal. 6. The method of claim 5,
Mapping the text data to tone color data stored in a database corresponding to the identified user. &Lt; RTI ID = 0.0 &gt; 31. &lt; / RTI &gt; Receiving a voice input signal of a user through a microphone;
Identifying a user based on frequency and intensity of the speech input signal; And
And outputting a voice guidance message based on the tone color data stored in the database in correspondence with the identified user. 8. The method of claim 7,
The voice announcement message output step may include:
The output voice is divided into a high-frequency band and a low-frequency band and is analog-encoded based on the tone data, a high-frequency band sound is output through a high-frequency band speaker provided in the speech recognition device, and a low- Wherein the speech recognition apparatus comprises: 8. The method of claim 7,
Wherein the tone color data includes frequency and intensity data of the identified user's voice. 8. The method of claim 7,
And discriminating a voice command included in the voice input signal. 11. The method of claim 10,
And mapping text data corresponding to the identified voice command to tone color data stored in a database corresponding to the identified user.

Images