JP6518096B2 - Speech recognition system and speech recognition method - Google Patents

Description

  The present invention relates to a technology for selecting a speech dictionary used for speech recognition.

  In recent years, the word "while smartphone" has been widely distributed. This word means that the user takes some action and at the same time uses a portable terminal device such as a smartphone, and the alarm is sounded as a dangerous act. Such a problem is caused by the fact that the user needs to look at the mobile terminal device when using (viewing and operating) the mobile terminal device, which reduces the user's visual power with respect to the surroundings. In many portable terminal devices, the user must browse the display to receive information, and must gaze at a touch panel or key to input (manipulate) information.

  2. Description of the Related Art Conventionally, as a method for a portable terminal device to provide information to a user, voice guidance such as streaming voice from a speaker is known. In addition, there is known a technology of performing speech recognition processing on words (voice) uttered by a user and inputting information to a portable terminal device. As described above, by adopting voice guidance and voice recognition, it is possible to use the portable terminal device without requiring the user's gaze. That is, a user interface using speech, in particular, a technology for comfortably using speech recognition is becoming an indispensable technology for the recent portable terminal devices.

  In the technology of speech recognition, in order to improve the speech recognition rate of the input speech, a technology is proposed that selects and switches the optimum speech recognition dictionary from among a plurality of speech recognition dictionaries according to the position information of the user It is done. For example, in Patent Document 1, the relative position of the operator is calculated based on the outputs of the 3-axis gyro sensor and the 3-axis acceleration sensor, and voice recognition is performed according to the determined relative position of the operator. Techniques for selecting a dictionary are described.

  Further, as a technology for suppressing power consumption, there is known a technology in which a microcomputer with small power consumption is mounted separately from the main CPU. The microcomputer controls a sensor or the like that needs to be regularly monitored, and in the meantime, the main CPU is turned off as needed to suppress power consumption as a whole. Power consumption control techniques are particularly important in portable terminal devices with poor power supply capabilities.

JP, 2010-19122, A

  However, in the technology described in Patent Document 1, if the worker (user) does not input a voice indicating the content of the work to be done from now on, such as "I am entering an account," switching of the voice recognition dictionary is required. There was a problem that it was not done. That is, there is a problem that the user has to execute the trigger for switching the speech recognition dictionary consciously and surely.

  In order not to make the user aware of the switching of the speech recognition dictionary, the occurrence of an event that triggers the switching of the speech recognition dictionary has to be continuously monitored, resulting in a problem of increased power consumption. That is, the technology described in Patent Document 1 focuses only on improving the recognition rate of the input speech, and lacks the viewpoint of achieving both improvement of the recognition rate and suppression of power consumption. is there.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a technology for suppressing power consumption without reducing the recognition accuracy of speech recognition.

  In order to solve the above-mentioned problems, the invention according to claim 1 is a voice recognition system for recognizing voice by a voice dictionary, and a power saving mode in which power consumption is suppressed as compared with a normal operation mode and the normal operation mode. A first storage device capable of switching an operation mode between the first storage device, a first storage device associating and storing a plurality of events assumed in advance and a plurality of speech dictionary candidates serving as candidates for the speech dictionary; Monitoring means for acquiring physical quantities for detecting the detection information as observation information, a microphone for acquiring the speech as speech information, a second storage device for storing the speech dictionary, and a second operation for accessing the second storage device And the second computing device detects an event that is currently occurring from among the plurality of events assumed in advance, based on the observation information acquired by the observation means. Event detection means, and speech recognition means for performing speech recognition based on the speech information acquired by the microphone and the speech dictionary stored in the second storage device; The system further comprises selection means for selecting one speech dictionary candidate from among a plurality of speech dictionary candidates stored in the first storage device according to an event detected as an occurring event, the selection by the selection means Power consumption when one voice dictionary candidate is stored as the voice dictionary in the second storage device and the first computing device is operated in the power saving mode and the second computing device is operated; It is smaller than the power consumption when the first arithmetic device is operated in the normal operation mode.

  The invention according to claim 2 is the speech recognition system according to the invention according to claim 1, wherein the observation means acquires a physical quantity resulting from the movement of the user as observation information, and the event detection means currently The behavior of the user is estimated as an event that has occurred.

  The invention of claim 3 is the speech recognition system according to the invention of claim 2, wherein the event detection means estimates the action of the user by estimating the posture of the user.

  The invention according to claim 4 is the speech recognition system according to any one of claims 1 to 3, wherein the plurality of speech dictionary candidates are selected according to the event to be associated with. It is done.

  The invention according to claim 5 is the speech recognition system according to any one of claims 1 to 4, wherein the first arithmetic device, the second arithmetic device, and the second storage device are carried by a user. And a server device connected in a state in which data communication can be performed between the mobile terminal device and the mobile terminal device and including the first storage device and the selection unit.

  The invention according to claim 6 is the speech recognition system according to any one of claims 1 to 5, wherein the second storage device stores past history information, and the event detection means The present invention estimates the currently occurring event based on the history information stored in the second storage device.

  The invention according to claim 7 is a speech recognition method for recognizing speech by means of a speech dictionary, wherein a plurality of events assumed in advance are associated with a plurality of speech dictionary candidates serving as candidates for the speech dictionary to store the first memory. A step of storing in the device, a step of switching the operation mode of the first arithmetic device between the normal operation mode and a power saving mode in which the power consumption is reduced compared to the normal operation mode, physical quantities for detecting an event Detecting a currently occurring event from the plurality of events assumed in advance by the second computing device based on the step of acquiring the observation information as observation information by the observation means, and the observation information acquired by the observation means One speech dictionary candidate out of a plurality of speech dictionary candidates stored in the first storage device in accordance with the process and the event detected by the second arithmetic device as the currently occurring event. Selecting the selected voice dictionary candidate as the voice dictionary and storing the selected voice dictionary candidate in the second storage device accessed by the second computing device; and obtaining the voice as voice information by the microphone Performing the voice recognition by the second computing device based on the process, and the voice information acquired by the microphone and the voice dictionary stored in the second storage device, the first computing device The power consumption when operating the second arithmetic device while operating in the power saving mode is smaller than the power consumption when operating the first arithmetic device in the normal operation mode.

  The invention according to any one of claims 1 to 7 operates the first computing device in the power saving mode by performing voice recognition by the second computing device while operating the first computing device in the power saving mode. Since the power consumption when the arithmetic device is operated is smaller than the power consumption when the first arithmetic device is operated in the normal operation mode, the power consumption can be suppressed. Further, according to an event detected as a currently occurring event, by selecting one speech dictionary candidate from among a plurality of speech dictionary candidates stored in the first storage device, recognition accuracy of speech recognition can be increased. It is possible to suppress the information capacity of the voice dictionary without reducing it.

FIG. 1 shows a speech recognition system. It is a block diagram of a server apparatus. It is a figure which illustrates the structure of a database. It is a figure which shows the functional block with which a server apparatus is equipped with the flow of data. It is a block diagram showing a personal digital assistant device. It is a figure which shows the functional block with which a portable terminal device is equipped with the flow of data. It is a flowchart which shows operation | movement of a server apparatus. It is a flowchart which shows operation | movement of a portable terminal device. It is a flowchart which shows the update request | requirement process which a portable terminal device performs.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings. However, unless otherwise specified in the following description, the description regarding the direction or the direction corresponds to the drawing for the convenience of the description, and does not limit, for example, the implemented product, the product or the scope of rights.

<1. Embodiment>
FIG. 1 is a diagram showing a speech recognition system 1. The voice recognition system 1 includes a server device 2 and a portable terminal device 3 carried by a user. The server device 2 and the portable terminal device 3 are connected by a network (not shown) so as to enable data communication. For example, a mobile telephone network can be used as such a network.

  The number of server devices 2 and portable terminal devices 3 provided in the speech recognition system 1 is not limited to one as shown in FIG. That is, the speech recognition system 1 may be configured of a plurality of server devices 2 and a plurality of mobile terminal devices 3. Further, the configuration and function of the server device 2 described later may be shared and realized by a plurality of devices. Furthermore, the form of the network connecting the server device 2 and the portable terminal device 3 is not limited to a single network, and may be a complex network. Further, in the following description, a person who operates the server device 2 is referred to as an “operator”, and a person who operates the mobile terminal device 3 is referred to as a “user”.

  FIG. 2 is a block diagram of the server device 2. The server device 2 includes a CPU 20, a storage device 21, an operation unit 22, a display unit 23, and a communication unit 24.

  The CPU 20 reads and executes the program 210 stored in the storage device 21 to calculate various data and generate control signals. Thus, the CPU 20 has a function of controlling each component of the server device 2 and calculating and creating various data. That is, the server device 2 is configured as a general computer.

  The storage device 21 provides a function of storing various data in the server device 2. In other words, the storage device 21 stores the electronically fixed information in the server device 2.

  The storage device 21 may be a RAM or buffer used as a temporary working area of the CPU 20, a read only ROM, a non-volatile memory (for example, a NAND memory), a hard disk for storing relatively large capacity data, a dedicated memory A portable storage medium (such as a CD-ROM, a DVD-ROM, a PC card, an SD card, a USB memory, etc.) attached to the reading device corresponds to this. In FIG. 2, the storage device 21 is illustrated as if it were a single structure. However, in general, the storage device 21 is configured of a plurality of types of devices adopted as needed among the various devices (or media) exemplified above. That is, the storage device 21 is a generic name of a device group having a function of storing data.

  The actual CPU 20 is an electronic circuit internally provided with a RAM that can be accessed at high speed. However, for convenience of explanation, the storage device provided with such a CPU 20 is also included in the storage device 21 and described. That is, the data temporarily stored by the CPU 20 itself is also described as being stored by the storage device 21. As shown in FIG. 2, the storage device 21 is used to store a program 210, a database 211, selected dictionary information 212, update request information 311 (event information 372), and the like.

  FIG. 3 is a diagram illustrating the structure of the database 211. As shown in FIG. 3, the database 211 is information of a table structure in which one record is created for each voice dictionary candidate. In each record of the database 211, a record number, an event, and an audio dictionary candidate are associated with each other and stored.

  The record number is an identifier for identifying each record of the database 211 individually. In the example shown in FIG. 3, n records are recorded in the database 211 (n is a natural number of 2 or more).

  The event is assumed in advance, and for example, the user's condition type (sex, age, etc.), the user's action type (cooking, learning, commuting, etc.), surrounding situation type (weather, season, time, outdoor) And indoors, scenes etc.). Of course, these types may be combined in combination to be one event. As the event, it is preferable to assume that the vocabulary of the language used by the user can be predicted characteristically. In the example of the database 211 shown in FIG. 3, “shopping”, “jogging” and “default” are registered as events.

  The speech dictionary candidates are individual speech dictionaries prepared corresponding to a plurality of events assumed in advance and registered in the database 211. For example, the first voice dictionary prepared and associated with "shopping" is a dictionary mainly including words (such as product name, store name, price, usage, etc.) expected to be used in shopping It is. Also, for example, the second audio dictionary associated with "jogging" is a dictionary mainly including vocabulary (for example, words such as pace, pulse, course, advice, consumed calories, etc.) expected to be used in jogging .

  As described above, as for the plurality of speech dictionary candidates registered in advance in the database 211, the vocabulary to be recorded is selected according to the events to be associated. As a result, it is possible to suppress the volume of information used as a speech dictionary without reducing the recognition accuracy of each vocabulary. A general voice dictionary (general-purpose dictionary) has an information capacity of about several megabytes. However, by specifying the event and limiting the vocabulary, the speech recognition system 1 can suppress the information volume of the speech dictionary candidate to, for example, several kilobytes.

  The "n-th speech dictionary" associated with the event "default" is an event currently occurring (event as a search key, details will be described later) from record numbers "001" to "n-1". The phonetic dictionary candidate is also selected when it does not correspond to an event. The n-th speech dictionary shown here is a speech dictionary in which general-purpose vocabulary is registered, but the accuracy (for example, sampling frequency) is suppressed. That is, when the n-th speech dictionary is selected in the server device 2, the information capacity of the speech dictionary to be used is equal to that of the other speech dictionary candidates, but the recognition accuracy is sacrificed.

  As described above, one record of the database 211 stores one event and one speech dictionary candidate. Then, a plurality (n) of records are created in the database 211. That is, by storing the database 211, the storage device 21 associates and stores a plurality of events assumed in advance and a plurality of speech dictionary candidates serving as speech dictionary candidates. Therefore, the storage device 21 corresponds to a first storage device.

  Although details will be described later, the selected dictionary information 212 shown in FIG. 2 is information including one speech dictionary candidate selected by the CPU 20 from among a plurality of speech dictionary candidates registered in the database 211. The selected dictionary information 212 is transmitted from the server device 2 to the portable terminal device 3 that has transmitted the update request information 311 (event information 372).

  The update request information 311 is information generated by the mobile terminal device 3 and received by the server device 2. The update request information 311 is information including an identifier (for example, a network address or the like) for individually identifying the mobile terminal device 3 and event information 372. Although the details will be described later, the mobile terminal device 3 requests the server device 2 to update the voice dictionary by transmitting the update request information 311 to the server device 2.

  The event information 372 is information included in the update request information 311 as described above. Although details will be described later, the event information 372 is information indicating an event currently occurring in the portable terminal device 3 that has transmitted the update request information 311 including the event information 372.

  The operation unit 22 is hardware operated by an operator or the like to input an instruction to the server device 2. The operation unit 22 corresponds to, for example, various keys and buttons, a switch, a touch panel, a pointing device, or a jog dial. The operation unit 22 is operated, for example, when an operator constructs the database 211.

  The display unit 23 is hardware having a function of outputting various data to an operator or the like. The display unit 23 corresponds to, for example, a lamp, an LED, a CRT, a liquid crystal display, a liquid crystal panel, or the like.

  The communication unit 24 provides a function for the server device 2 to perform data communication with the mobile terminal device 3. That is, the server device 2 causes the communication unit 24 to receive the information transmitted from the mobile terminal device 3 and transmits the information to the mobile terminal device 3. As information that the server device 2 receives from the portable terminal device 3, there is, for example, update request information 311. Further, as information that the server device 2 transmits to the mobile terminal device 3, for example, there is selection dictionary information 212. When transmitting the selected dictionary information 212, the communication unit 24 refers to the update request information 311 to identify the mobile terminal device 3 that has transmitted the update request information 311.

  FIG. 4 is a diagram showing functional blocks included in the server device 2 together with the flow of data. The selection unit 200 illustrated in FIG. 4 is a functional block implemented by the CPU 20 operating according to the program 210.

  The selection unit 200 searches the database 211 stored in the storage device 21 according to the event information 372 included in the update request information 311, and selects one speech dictionary candidate from among a plurality of speech dictionary candidates. In addition, the selection unit 200 creates selection dictionary information 212 including one selected speech dictionary candidate.

  The event information 372 is information created in the mobile terminal device 3 and is information transmitted from the mobile terminal device 3 to the server device 2. When it is necessary to update the voice dictionary, the mobile terminal device 3 creates update request information 311 including the event information 372 and transmits it to the server device 2. As described above, the event information 372 is information indicating an event detected as an event that is currently occurring in the mobile terminal device 3. The selection unit 200 has a function of searching the database 211 using an event indicated in the event information 372 as a search key, and identifying a speech dictionary candidate associated with the event.

  Therefore, when the event information 372 indicates, for example, “shopping” as an event occurring at present, in the database 211 illustrated in FIG. 3, the first voice dictionary associated with “shopping” is an audio of 1. Selected as a dictionary candidate. In this case, the selected dictionary information 212 created by the selecting unit 200 is the first speech dictionary.

  The selected dictionary information 212 thus created is transmitted by the communication unit 24 to the portable terminal device 3 that has issued the update request as a response to the update request (update request information 311) from the portable terminal device 3.

  FIG. 5 is a block diagram showing the mobile terminal device 3. The mobile terminal device 3 includes a CPU 30, a storage device 31, an operation unit 32, a display unit 33, a communication unit 34, and a speaker 35.

  The CPU 30 reads and executes the program 310 stored in the storage device 31 to calculate various data and generate control signals. Thus, the CPU 30 has functions of controlling each component of the mobile terminal 3 and calculating and creating various data. That is, the portable terminal device 3 is configured as a general computer.

  In addition, the CPU 30 has, as operation modes, a normal operation mode in which all functions can be used, and a power saving mode in which part or all of the functions are limited. The power saving mode is a so-called sleep mode, and is an operation mode in which power consumption in the CPU 30 can be suppressed instead of limiting some or all of the functions.

  A plurality of modes may be defined stepwise in the power saving mode. Further, in the following description, “the CPU 30 operates in the power saving mode” is used to include the case where the CPU 30 is completely stopped.

  The storage device 31 provides a function of storing various data in the mobile terminal device 3. The storage device 31 may be a RAM or buffer used as a temporary working area of the CPU 30, a read only ROM, a non-volatile memory (for example, a NAND memory), a hard disk for storing relatively large capacity data, a dedicated memory This corresponds to a portable storage medium (PC card, SD card, USB memory, etc.) attached to the reading device. In FIG. 5, the storage device 31 is illustrated as if it were a single structure. However, in general, the storage device 31 is configured by a plurality of types of devices adopted as needed among the various devices (or media) exemplified above. That is, the storage device 31 has a function of storing data, and is a generic name of devices accessed by the CPU 30.

  The actual CPU 30 is an electronic circuit internally provided with a RAM that can be accessed at high speed. However, for convenience of explanation, the storage device included in such a CPU 30 will be described by being included in the storage device 31. That is, the data temporarily stored by the CPU 30 is also described as being stored by the storage device 31. As shown in FIG. 5, the storage device 31 is used to store the program 310, the update request information 311, and the like.

  The operation unit 32 is hardware operated by the user to input an instruction to the mobile terminal device 3. The operation unit 32 corresponds to, for example, various keys and buttons, a switch, a touch panel, a pointing device, or a jog dial.

  The display unit 33 is hardware having a function of outputting various data to the user. The display unit 33 corresponds to, for example, a lamp, an LED, a liquid crystal display, a liquid crystal panel, or the like.

  The communication unit 34 provides a function for the portable terminal device 3 to perform data communication with the server device 2. That is, the mobile terminal device 3 causes the communication unit 34 to receive the information transmitted from the server device 2 and transmit the information to the server device 2. As information that the mobile terminal device 3 receives from the server device 2, for example, there is selection dictionary information 212. Further, as information transmitted to the server device 2 by the mobile terminal device 3, for example, there is the update request information 311.

  The speaker 35 is hardware having a function of outputting voice based on voice information (generally different from the voice information 374 but may reproduce the voice information 374). The speaker 35 is used to give voice guidance to the user and to provide music, a broadcast program, a receiving function, and the like.

  Furthermore, the portable terminal device 3 includes an MPU 36, a storage device 37, an observation device group 38, and a microphone 39.

  The MPU 36 reads and executes the program 370 stored in the storage device 37, and performs operations of various data, generation of control signals, and the like. Thus, the MPU 36 has functions of controlling each component of the mobile terminal 3 and calculating and creating various data.

  Further, the MPU 36 is configured as an arithmetic device that consumes less power at the time of operation. Specifically, it is designed that operating the MPU 36 while operating the CPU 30 in the power saving mode consumes less power in the mobile terminal device 3 than when the CPU 30 operates in the normal operation mode. There is. That is, the MPU 36 is a so-called built-in LSI whose processing capacity is lower than that of the CPU 30 which is the main arithmetic unit in the portable terminal device 3.

  The storage device 37 provides a function of storing various data in the mobile terminal device 3. In other words, the storage device 37 stores the information electronically fixed in the portable terminal device 3 as the storage device 31 does.

  As the storage device 37, a RAM or buffer used as a temporary working area of the MPU 36, a read only ROM, a non-volatile memory (for example, a NAND memory or the like) or the like corresponds. In FIG. 5, the storage device 37 is illustrated as if it were a single structure. However, in general, the storage device 37 is configured of a plurality of types of devices adopted as needed among the various devices (or media) exemplified above. That is, the storage device 37 has a function of storing data, and is a generic name of devices accessed by the MPU 36.

  The actual MPU 36 is an electronic circuit internally provided with a RAM that can be accessed at high speed. However, for convenience of explanation, the storage device included in such MPU 36 is also included in the storage device 37 and described. That is, the data temporarily stored by the MPU 36 itself will be described as being stored by the storage device 37. As shown in FIG. 5, the storage device 37 is used to store a program 370, observation information 371, event information 372, history information 373, voice information 374, selected dictionary information 212 and the like.

  The observation device group 38 includes a plurality of detection devices that obtain observation information 371 by detecting information indicating the surrounding environment, information on the movement of the mobile terminal 3 (a user carrying the mobile terminal 3), and the like. Ru. The observation device group 38 includes a temperature sensor, an air pressure sensor, a humidity sensor, an illuminance sensor, a vibration sensor, a GPS for specifying a position, an imaging device for acquiring an image of the surroundings, a gyro sensor, an acceleration sensor, a magnetic sensor, a pulse sensor, A blood pressure sensor etc. are assumed.

  In addition, the gyro sensor, the acceleration sensor, the magnetic sensor, and the like acquire physical quantities resulting from the movement of the user as observation information 371. The method of estimating the posture of the user carrying the portable terminal device 3 or estimating the action based on the information acquired by the gyro sensor, the acceleration sensor, the magnetic sensor or the like appropriately applies the conventional technique. Here, the detailed description will be omitted. Note that audio information 374 acquired by the microphone 39 described later may be regarded as part of the observation information 371.

  The observation device group 38 is controlled by the MPU 36, and acquisition of observation information 371 is enabled not only in the normal operation mode of the CPU 30 but also in the power saving mode. However, part of the observation device group 38 may be stopped by the MPU 36 as needed.

  The microphone 39 is hardware having a function of converting ambient sound into an electrical signal and acquiring audio information 374. The microphone 39 is controlled by the MPU 36 in the same manner as the observation device group 38, and the voice information 374 can be obtained not only in the normal operation mode of the CPU 30 but also in the power saving mode. By providing the microphone 39 to the portable terminal device 3, the voice (language) emitted by the user is converted into voice information 374 and stored in the storage device 37.

  FIG. 6 is a diagram showing functional blocks included in the mobile terminal device 3 together with the flow of data. The interface unit 360, the event detection unit 361, and the voice recognition unit 362 illustrated in FIG. 6 are functional blocks implemented by the MPU 36 operating according to the program 370.

  The interface unit 360 has a function of controlling input and output of signals between the CPU 30 and the MPU 36. The interface unit 360 causes the storage device 37 to store the selected dictionary information 212 transferred from the CPU 30. Also, event information 372 created by the event detection unit 361 is transferred to the CPU 30. Furthermore, it also has a function of transferring the recognition result of the speech recognition unit 362 to the CPU 30.

  The interface unit 360 also has a function of switching the operation mode of the CPU 30 operating in the power saving mode to the normal operation mode, as necessary. Thus, the MPU 36 can return the CPU 30 to the normal operation mode and process it, for example, when complicated processing is required.

  The event detection unit 361 detects a currently occurring event from among a plurality of events assumed in advance, based on the observation information 371 and the history information 373 acquired by the observation device group 38. The event detection unit 361, when detecting a specific event (event currently occurring), creates event information 372 indicating the event. More specifically, the event detection unit 361 detects a change in a currently occurring event while monitoring a currently occurring event based on observation information 371 constantly acquired. Then, when a change in an event is detected, event information 372 indicating an event that has become a newly occurring event is created.

  The event detection unit 361 also has a function of creating history information 373. The history information 373 is information such as which event has been detected based on what kind of observation information 371 in the past. Specific examples of the history information 373 include, for example, information that the user jogged on Sunday morning, information that the user cooked at a predetermined time in the evening, or shopping at a predetermined position (shop location) And other information. Thereby, since an event can be detected based on a user's action pattern etc., detection accuracy of an event improves.

  The history information 373 may include the recognition rate by the speech recognition unit 362 or the like. By this configuration, it is possible to feed back the recognition rate of the voice dictionary candidate (selected dictionary information 212) selected based on the event information 372, and thereafter, a more appropriate voice dictionary candidate is selected. It becomes. Also, the history information 373 may be created based on the input information when the operation unit 32 is operated by the user.

  The speech recognition unit 362 executes speech recognition based on the speech information 374 acquired by the microphone 39 and the selected dictionary information 212 stored in the storage device 37. A specific technique of speech recognition by the speech recognition unit 362 can appropriately adopt a conventional technique, and thus the detailed description is omitted.

  The recognition result by the speech recognition unit 362 will be described as being transmitted to the CPU 30 via the interface unit 360. However, the recognition result does not necessarily have to be transmitted to the CPU 30. For example, the information may be information processed only by the MPU 36.

  The above is the description of the configuration and functions of the speech recognition system 1. Next, the speech recognition method will be described.

  FIG. 7 is a flow chart showing the operation of the server device 2. Each step shown in FIG. 7 mainly shows steps performed by the server device 2 in realizing the speech recognition method according to the present invention. Further, it is assumed that the database 211 is created in advance and stored in the storage device 21 before each process shown in FIG. 7 is started. That is, it is assumed that the process of storing the plurality of events assumed on the portable terminal 3 side and the plurality of speech dictionary candidates in the storage device 21 in association with each other has already been completed.

  When the operation is started, the server device 2 monitors whether the update request information 311 from the portable terminal device 3 has been received (step S1).

  When the update request information 311 is received (Yes in step S1), the selection unit 200 searches the database 211 based on the event information 372 included in the received update request information 311.

  As described above, the event information 372 included in the update request information 311 indicates “currently occurring event” in the mobile terminal device 3. Therefore, the selection unit 200 selects the speech dictionary candidate associated with the currently occurring event from the database 211 by searching the database 211 using the currently occurring event as a search key (step S2).

  The speech dictionary candidates associated with each event in the database 211 are speech dictionaries optimized and registered corresponding to each event. Therefore, the selection unit 200 selects a speech dictionary candidate associated with an event currently occurring in the portable terminal 3 to select the speech dictionary candidate most suitable for the event occurring currently. . For example, when “jogging” is indicated as a currently occurring event, the selection unit 200 can select the second speech dictionary created as the speech dictionary suitable for the user who is jogging.

  When step S2 is executed, the selection unit 200 creates selection dictionary information 212 including the selected speech dictionary candidate and an identifier for identifying the update request information 311 including the event information 372 used for selection (step S3). ).

  Thus, when the selected dictionary information 212 is newly created, the communication unit 24 specifies the update request information 311 based on the identifier included in the selected dictionary information 212. Then, the communication unit 24 specifies the mobile terminal device 3 that has transmitted the specified update request information 311, and transmits the update request information 311 to the mobile terminal device 3 (step S4). Thus, the server device 2 transmits the selected dictionary information 212 as a response to the update request (update request information 311) from the mobile terminal device 3.

  FIG. 8 is a flowchart showing the operation of the mobile terminal device 3. The steps shown in FIG. 8 mainly show steps performed by the mobile terminal device 3 in realizing the speech recognition method according to the present invention. In addition, it is assumed that the n-th speech dictionary is stored in the storage device 37 as the selected dictionary information 212 before each process shown in FIG. 8 is started. That is, it is assumed that the process of storing the default speech dictionary (nth speech dictionary) in the storage device 37 in the portable terminal device 3 has already been completed.

  Note that FIG. 8 does not illustrate the process of switching the CPU 30 from the normal operation mode to the power saving mode. Such operation mode switching can be performed, for example, when no operation by the user is detected for a predetermined period, when a direct instruction by the user (instruction to switch to the power saving mode) is given, or by an application in use. It is executed when it is judged. However, the triggers for switching to the power saving mode are not limited to these.

  When the mobile terminal device 3 is powered on, the mobile terminal device 3 performs a predetermined initial setting and then transitions to a state in which voice recognition can be performed. This state is hereinafter referred to as "operation start state". In the operation start state, the operation mode of the CPU 30 is a normal operation mode or a power saving mode. Further, in the operation start state, the portable terminal device 3 is assumed to be carried by the user.

  In the operation start state, the mobile terminal device 3 creates observation information 371 by the observation device group 38 (step S11). Step S11 is periodically and continuously executed without an instruction from the user. The observation information 371 created in step S11 is stored in the storage device 37.

  When the observation information 371 is stored in the storage device 37, the event detection unit 361 detects an event that is currently occurring based on the observation information 371 and the history information 373. More specifically, the event detection unit 361 analyzes the observation information 371 acquired by the observation device group 38 to grasp the situation, and at the present time, by referring to the user's action pattern or the like based on the history information 373 Estimate the event that is Furthermore, the event detection unit 361 determines whether the event has changed by comparing it with the event information 372 already stored (step S12).

  As described above, in the operation start state, the mobile terminal device 3 constantly acquires the observation information 371 and monitors whether or not a currently occurring event has changed. That is, as a matter of course in the normal operation mode, the process of step S12 is also performed in the power saving mode. Furthermore, since the special instruction from the user is not required in the process of step S12, the process of step S12 is executed even if the user is not particularly aware. The initial value of the event shown in the event information 372 is “default”.

  If a change has occurred in the event and it is determined as Yes in step S12, the portable terminal device 3 executes an update request process (step S13).

  FIG. 9 is a flowchart showing the update request process performed by the mobile terminal device 3. The update request process is a process in which the mobile terminal device 3 requests the server device 2 to transmit a new voice dictionary.

  When the update request process is started, the event detection unit 361 creates new event information 372 indicating the detected event (event newly generated) (step S31). Also, the event detection unit 361 updates the event information 372 already stored in the storage device 37 based on the newly created event information 372.

  Furthermore, the event detection unit 361 is based on the newly created event information 372 (detection result), the observation information 371 referred to when creating the event information 372 (information based on the detection result), etc. The history information 373 is updated (step S32). As a result, detection results of events in the past are accumulated.

  Next, the interface unit 360 detects that the event information 372 has been updated, and determines whether the operation mode of the CPU 30 is the power saving mode (step S33). Then, if the operation mode is the power saving mode (Yes in step S33), the operation mode is switched to the normal operation mode (step S34). On the other hand, when it is not the power saving mode (No in step S33), the interface unit 360 skips the process of step S34.

  The processes of steps S33 and S34 will be described more specifically. When the event information 372 is updated and it is necessary to transmit the event information 372 to the CPU 30, the interface unit 360 notifies the CPU 30 of an interrupt signal. When an interrupt signal is notified from the MPU 36 (interface unit 360) to the CPU 30, if the CPU 30 is in the power saving mode, this interrupt signal becomes a signal for returning the CPU 30 to the normal operation mode. On the other hand, when the interrupt signal is notified from the communication unit 34 to the CPU 30, if the CPU 30 is in the normal operation mode, this interrupt signal is not regarded as a signal for returning the CPU 30 to the normal operation mode. It is processed as a typical interrupt signal. Therefore, the actual interface unit 360 does not determine in step S33 whether the CPU 30 is in the power saving mode.

  After steps S33 and S34 are executed, the CPU 30 always enters the normal operation mode and can receive event information 372. Therefore, the interface unit 360 transfers the newly created event information 372 to the CPU 30 (step S35).

  When the event information 372 is received from the MPU 36, the CPU 30 creates the update request information 311 based on the event information 372 (step S36). The update request information 311 is information including the event information 372 and the identifier of the mobile terminal 3 (for example, a network address etc.) as described above.

  When the update request information 311 is created, the CPU 30 controls the communication unit 34 to transmit the update request information 311 to the server device 2. Thus, the communication unit 34 transmits the update request information 311 to the server device 2 (step S37).

  When step S37 is executed and the communication unit 34 transmits the update request information, the portable terminal device 3 ends the update request process, and returns to the process shown in FIG.

  Referring back to FIG. 8, when it is determined No in step S12, the mobile terminal device 3 skips step S13. Therefore, the portable terminal device 3 does not execute the update request process (step S13) unless the currently occurring event changes, and the update request information 311 is not transmitted to the server device 2. .

  In the operation start state, the portable terminal device 3 creates the voice information 374 by the microphone 39 (step S14). Step S14 is periodically and continuously executed in the operation start state even without the user's instruction. The voice information 374 created in step S14 is stored in the storage device 37.

  When the speech information 374 is stored in the storage device 37, the speech recognition unit 362 executes speech recognition based on the speech information 374 and the selected dictionary information 212 (step S15), and whether or not the recognition is successful. It is determined (step S16).

  As described above, in the mobile terminal device 3, the processing in steps S14 to S16 is realized by the MPU 36, and can be executed even when the CPU 30 is in the power saving mode. That is, the voice recognition system 1 is configured to be able to always perform voice recognition while suppressing power consumption.

  In addition, the processing in steps S14 to S16 is configured as processing that is executed without the user's special instruction in the operation start state of the portable terminal device 3. Therefore, the user can use speech recognition without being particularly aware, and the burden on the user is reduced.

  If the speech recognition unit 362 succeeds in recognition (Yes in step S16), the MPU 36 executes the recognition result (step S17).

  The execution of the recognition result in step S17 means that the MPU 36 transfers the recognition result to the CPU 30. Specifically, first, the speech recognition unit 362 transmits the recognition result to the interface unit 360. Next, the interface unit 360 transfers the recognition result transmitted from the speech recognition unit 362 to the CPU 30.

  When the recognition result is transferred to the CPU 30 and the CPU 30 is in the power saving mode, the interface unit 360 transfers the recognition result after switching the operation mode of the CPU 30 to the normal operation mode.

  The process executed by the CPU 30 will be described by taking an example in which “jogging” is detected as a currently occurring event and the second speech dictionary candidate is stored in the storage device 37 as the selected dictionary information 212 as an example. In such a state, for example, when the user utters "pulse", the voice recognition unit 362 performs voice recognition using the second voice dictionary candidate, and causes the CPU 30 to use the word "pulse" (text information) as a recognition result. introduce.

  The CPU 30 that has received the recognition result from the MPU 36 executes a process according to the recognition result.

  In the example shown above, the CPU 30 controls the speaker 35 to measure the user's pulse rate and provide voice guidance as processing for the user's utterance of "pulse". As a result, sound such as "120" is reproduced from the speaker 35, for example. Therefore, the user can use the portable terminal device 3 without visually observing and viewing the portable terminal device 3 and operating it.

  In the operation start state, when the communication unit 34 receives the selected dictionary information 212 transmitted from the server device 2 (Yes in step S18), the portable terminal device 3 determines whether the operation mode is the power saving mode. (Step S19). Then, if the operation mode is the power saving mode (Yes in step S19), the operation mode is switched to the normal operation mode (step S20). On the other hand, when it is not the power saving mode (No in step S19), the mobile terminal device 3 skips the process of step S20.

  The processes of steps S18 to S20 will be described more specifically. The communication unit 34 monitors the network in the operation start state, and monitors whether the mobile terminal device 3 receives a call via the network. When the communication unit 34 detects an incoming call, an interrupt signal is notified from the communication unit 34 to the CPU 30. Therefore, the actual communication unit 34 does not determine whether the received information is the selected dictionary information 212 in step S18.

  When an interrupt signal is notified from the communication unit 34 to the CPU 30, if the CPU 30 is in the power saving mode, this interrupt signal becomes a signal for returning the CPU 30 to the normal operation mode. On the other hand, when the interrupt signal is notified from the communication unit 34 to the CPU 30, if the CPU 30 is in the normal operation mode, this interrupt signal is not regarded as a signal for returning the CPU 30 to the normal operation mode. It is processed as a typical interrupt signal. Therefore, the CPU 30 does not determine whether or not the power saving mode is set in step S19.

  If No in step S19 or if step S20 is executed, the CPU 30 transfers the selected dictionary information 212 received by the communication unit 34 toward the MPU 36 (interface unit 360). When the selected dictionary information 212 is transferred from the CPU 30, the interface unit 360 causes the storage device 37 to store the selected dictionary information 212 (step S21). As a result, in the portable terminal device 3, the selected dictionary information 212 already stored is updated to the newly received selected dictionary information 212.

  As described above, in the initial state, the n-th speech dictionary is stored as the selected dictionary information 212 in the storage device 37 of the mobile terminal device 3. In this state, for example, when the second speech dictionary associated with the event "jogging" is received as the selected dictionary information 212, step S21 is executed to update the n-th speech dictionary to the second speech dictionary It will be done.

  For example, it can be expected that words used by the user during jogging (words input to the mobile terminal device 3) are limited to the vocabulary related to jogging. Therefore, when "jogging" is detected as a currently occurring event, the second speech dictionary whose vocabulary is selected corresponding to the jogging is used for speech recognition, without lowering the speech recognition accuracy. It is possible to use an audio dictionary whose information capacity (size) is reduced compared to a normal audio dictionary.

  Thus, the speech recognition system 1 has an advantage that the response performance is good because the size of the speech dictionary is small. As described above, in the voice recognition system 1, whenever a new event is detected in the portable terminal device 3, a corresponding voice dictionary is downloaded from the server device 2 to the portable terminal device 3. If the size of the speech dictionary to be downloaded is large, it takes time to download, the time to prepare the speech dictionary will increase, and the response performance will deteriorate. However, in the speech recognition system 1, since the size of the speech dictionary (selected dictionary information 212) to be downloaded is small, the time required for the download is short, and the response performance is not sacrificed.

  In addition, since the size of the voice dictionary is small, the storage capacity of the storage device 37 can be small, and voice recognition can be performed even with an arithmetic device having a relatively low processing capability such as the MPU 36. Therefore, the cost of the entire system can be reduced.

  Further, in the technology described in Japanese Patent Application Laid-Open No. 2010-191223, if a worker (user) does not input a voice indicating the content of work to be done from now on, such as "I am accounting," a voice dictionary Switching is not performed. In other words, the trigger for switching the speech dictionary must be consciously and surely executed by the user. However, the speech recognition system 1 automatically detects a currently occurring event based on the observation information 371 obtained continuously and without the user's awareness by the observation device group 38 and triggers this. As update request (update request processing) is performed. Therefore, the user does not have to be particularly aware of providing a trigger for switching the voice dictionary, and the burden on the user is reduced.

  Also, in the prior art, since the selection of the speech dictionary is performed based on the position information, the user can only give a trigger at the position where the target speech dictionary is selected. Therefore, the prior art may select an inappropriate voice dictionary if the user does not understand the correspondence between the user's location and the voice dictionary. However, the speech recognition system 1 can select an optimum speech dictionary without relying on the user because it detects events that are currently occurring based on various observation information 371 (and history information 373).

  Further, in the prior art, since the selection of the speech dictionary is performed based only on the position information, the speech dictionary can not be optimized based on the event unrelated to the position, and the versatility is low. There is a problem that narrowing is also insufficient. However, since the speech recognition system 1 detects a currently occurring event based on various observation information 371, it is possible to select a speech dictionary more suitable for the situation.

  As described above, the voice recognition system 1 that recognizes voice by means of a voice dictionary can switch the operation mode between the normal operation mode and the power saving mode in which power consumption is suppressed as compared to the normal operation mode. , A storage device 21 storing a database 211 associating a plurality of events assumed in advance with a plurality of speech dictionary candidates serving as candidates for a speech dictionary, and an observation device acquiring physical quantities for detecting events as observation information 371 A group 38, a microphone 39 for acquiring speech as speech information 374, a storage unit 37 for storing selected dictionary information 212, and an MPU 36 for accessing the storage unit 37 are provided. Then, based on the observation information 371 acquired by the observation device group 38, the MPU 36 is acquired by the event detection unit 361 that detects an event that is currently occurring out of a plurality of events assumed in advance, and the microphone 39. The voice recognition unit 362 performs voice recognition based on the voice information 374 and the selected dictionary information 212 stored in the storage device 37. Further, the speech recognition system 1 selects one speech dictionary candidate from among a plurality of speech dictionary candidates stored in the storage device 21 in accordance with the event detected as the event currently occurring by the event detection unit 361. The selection unit 200 is further provided, and the selected dictionary information 212 selected by the selection unit 200 is stored in the storage device 37 as a speech dictionary. Furthermore, in the voice recognition system 1, the power consumption when operating the MPU 36 while operating the CPU 30 in the power saving mode is designed to be smaller than the power consumption when operating the CPU 30 in the normal operation mode There is. Thus, power consumption can be suppressed by causing the MPU 36 with low power consumption to perform voice recognition. In addition, even if the small selection dictionary information 212 is used for speech recognition, the selection dictionary information 212 is optimized according to the event, so the recognition rate is not reduced.

  In addition, the observation device group 38 acquires a physical quantity resulting from the user's movement as the observation information 371, and the event detection unit 361 estimates the user's action as an event that is currently occurring. The voice (word) emitted by the user is highly relevant to the user's action. Therefore, by estimating the user's behavior, it is possible to select a more optimal speech dictionary. That is, the accuracy of speech recognition is improved.

  Further, the event detection unit 361 estimates the user's action by estimating the user's posture. The behavior of the user is highly related to the posture of the user, so that the accuracy of estimating the behavior of the user is improved.

  In addition, the plurality of speech dictionary candidates are sorted out according to the event to be associated with. Thereby, the size of the voice dictionary (selected dictionary information 212) can be suppressed without reducing the recognition accuracy.

  Moreover, the suppression effect of power consumption becomes more remarkable by applying to the portable terminal device 3 with low self-supporting ability of the speech recognition system 1 electric power.

  Further, the storage device 37 stores the past history information 373, and the event detection unit 361 estimates a currently occurring event based on the history information 373 stored in the storage device 37. Therefore, the estimation accuracy of the event is improved.

  In the above embodiment, only steps S20 and S34 have been described as the process of switching the CPU 30 from the power saving mode to the normal operation mode. However, what becomes a trigger which changes CPU30 to a normal operation mode is not necessarily limited to what becomes conditions which perform these processes.

<2. Modified example>
As mentioned above, although embodiment of this invention was described, this invention is not limited to the said embodiment, A various deformation | transformation is possible.

  For example, each process shown in the above-mentioned embodiment is an illustration to the last, and is not limited to the order and the contents which were shown above. That is, the order and content may be changed as appropriate, as long as the same effect is obtained. For example, the present invention can be realized even if the order of the step of updating event information 372 (step S31) and the step of updating history information 373 (step S32) is switched.

  In addition, it has been described that the selection unit 200 described in the above embodiment is realized as software by the CPU 20 operating according to the program 210. In addition, it has been described that the interface unit 360, the event detection unit 361, and the voice recognition unit 362 are realized as software by the MPU 36 operating according to the program 370. However, some or all of these functional blocks may be configured as dedicated logic circuits and implemented as hardware.

  In the above embodiment, the database 211 is stored in the server device 2 and the CPU 20 (selection unit 200) of the server device 2 selects the voice dictionary candidate. However, for example, information corresponding to the database 211 may be stored in the storage device 31 of the mobile terminal device 3 and the CPU 30 may be configured to select speech dictionary candidates and transmit them to the MPU 36.

1 speech recognition system 2 server device 20, 30 CPU
200 selection unit 21, 31, 37 storage device 210, 310, 370 program 211 database 212 selection dictionary information 22, 32 operation unit 23, 33 display unit 24, 34 communication unit 3 portable terminal device 311 update request information 35 speaker 36 MPU
360 interface unit 361 event detection unit 362 speech recognition unit 371 observation information 372 event information 373 history information 374 speech information 38 observation device group 39 microphone

Claims (7)

A speech recognition system for recognizing speech by means of a speech dictionary, comprising
A first arithmetic device capable of switching an operation mode between a normal operation mode and a power saving mode in which power consumption is suppressed as compared with the normal operation mode;
A first storage device that associates and stores a plurality of events assumed in advance and a plurality of speech dictionary candidates serving as speech dictionary candidates;
An observation means for acquiring a physical quantity for detecting an event as observation information;
A microphone for acquiring the voice as voice information;
A second storage device storing the speech dictionary;
A second computing device accessing the second storage device;
Equipped with
The second computing device is
An event detection unit that detects an event currently occurring from the plurality of events assumed in advance based on observation information acquired by the observation unit;
Voice recognition means for performing voice recognition based on the voice information acquired by the microphone and the voice dictionary stored in the second storage device;
Equipped with
Selecting means for selecting one speech dictionary candidate from among a plurality of speech dictionary candidates stored in the first storage device according to an event detected as an event currently occurring by the event detection means Equipped
Storing one speech dictionary candidate selected by the selection unit in the second storage device as the speech dictionary;
The power consumption when operating the second arithmetic device while operating the first arithmetic device in the power saving mode is smaller than the power consumption when operating the first arithmetic device in the normal operation mode Speech recognition system. The speech recognition system according to claim 1, wherein
The observation means acquires, as observation information, a physical quantity resulting from the movement of the user.
The speech recognition system, wherein the event detection means estimates the user's behavior as a currently occurring event. The speech recognition system according to claim 2, wherein
The voice recognition system which estimates the action of the user by the event detecting means estimating the posture of the user. The speech recognition system according to any one of claims 1 to 3, wherein
The speech recognition system according to claim 1, wherein the plurality of speech dictionary candidates are selected according to an event to be associated. The speech recognition system according to any one of claims 1 to 4, wherein
A portable terminal device carried by a user and comprising the first computing device, the second computing device, and the second storage device;
A server device connected to the portable terminal device in a data communication enabled state and including the first storage device and the selection unit;
Speech recognition system comprising: The speech recognition system according to any one of claims 1 to 5, wherein
The second storage device stores past history information,
The speech recognition system according to claim 1, wherein the event detection means estimates a currently occurring event based on history information stored in the second storage device. A speech recognition method for recognizing speech with a speech dictionary, comprising:
Storing a plurality of events assumed in advance and a plurality of speech dictionary candidates as candidates for the speech dictionary in the first storage device in association with each other;
Switching the operation mode of the first computing device between a normal operation mode and a power saving mode in which power consumption is suppressed as compared to the normal operation mode;
Acquiring a physical quantity for detecting an event as observation information by the observation means;
Detecting a currently occurring event from the plurality of events assumed in advance by the second arithmetic unit based on observation information acquired by the observation unit;
Selecting one speech dictionary candidate from among a plurality of speech dictionary candidates stored in the first storage device according to an event detected by the second arithmetic device as a currently occurring event;
Storing the selected one voice dictionary candidate as the voice dictionary in a second storage device accessed by the second arithmetic device;
Acquiring the voice as voice information by a microphone;
Executing voice recognition by the second computing device based on voice information acquired by the microphone and a voice dictionary stored in the second storage device;
Have
The power consumption when operating the second arithmetic device while operating the first arithmetic device in the power saving mode is smaller than the power consumption when operating the first arithmetic device in the normal operation mode Speech recognition method.

Images