JP2016164791A - Server device and search method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique for an information terminal that enables a user to easily utilize a high performance search function.SOLUTION: An information terminal comprises: voice input reception means which receives input of voice; communication means which performs communication with a predetermined server device via a network; output means; A Point Of Interest (POI) identification means which transmits voice information received by the voice input reception means to the server device and receives information for identifying POI candidates related to the voice information; POI candidates output means which outputs the information for identifying the POI candidates received by the POI identification means to the output means; and route search means which, after receiving input of selection from the information for identifying the POI candidates, searches for a route which leads to the selected POI.SELECTED DRAWING: Figure 6

Description

  The present invention relates to information terminal technology.

  2. Description of the Related Art Conventionally, information terminals such as navigation devices use a technique for receiving a voice input with a microphone or the like and searching for a geographical name that can be a destination or a waypoint. Patent Document 1 describes a technique regarding such a navigation device.

JP 2006-349427 A

  In the navigation device as described above, the navigation device recognizes speech and performs a dictionary search for candidate facilities and the like. In order to provide a higher search function, the processing load of the navigation device is considered. Therefore, it is necessary to use a very high performance device.

  An object of the present invention is to provide a technology of an information terminal that can use a high search function more easily.

  In order to solve the above problems, an information terminal according to the present invention includes a voice input receiving unit that receives voice input, a communication unit that communicates with a predetermined server device via a network, an output unit, and the voice input receiving unit. POI specifying means for transmitting voice information received by the means to the server device and receiving information for specifying POI (Point Of Interest) candidates related to the voice information; POI received by the POI specifying means A POI candidate output means for outputting information specifying the candidate to the output means, and a route search means for receiving a selection input of information specifying the POI candidate and searching for a route to the POI. Features.

  Further, the server device according to the present invention includes a voice information receiving unit that receives voice information from a predetermined information terminal via a network, a noise removing unit that removes noise information from the voice information, and the noise-removed unit Voice transmitting means for transmitting voice information to a predetermined voice recognition device via the network; POI information receiving means for receiving POI (Point Of Interest) information related to the character string via the network; And POI information transmitting means for transmitting POI information to the information terminal.

  Moreover, the search system according to the present invention is a search system having an information terminal and a server device that communicates with the information terminal via a network, wherein the information terminal receives voice input means. And a communication means for communicating with the server device via a network, an output means, and voice information received by the voice input receiving means to the server device to send a POI (Point) related to the voice information Of interest) POI specifying means for receiving information for specifying candidates, POI candidate output means for outputting information for specifying POI candidates received by the POI specifying means to the output means, and specifying the POI candidates Route search means for receiving a selection input of information to be searched and searching for a route to the POI, The server device includes: a voice information receiving unit that receives voice information from the information terminal via a network; a noise removing unit that removes noise information from the voice information; and a predetermined voice recognition for the voice information from which noise has been removed. Voice transmitting means for transmitting to the apparatus via the network; POI information receiving means for receiving POI (Point Of Interest) information related to the character string via the network; and the POI information for the information terminal And a POI information transmitting means for transmitting to.

  The search method according to the present invention is a search method for a search system having an information terminal and a server device that communicates with the information terminal via a network, the search system accepting voice input. A transmission step for transmitting voice information received by the voice input reception unit to the server device; and voice information for receiving voice information from the information terminal via the network. A reception step; a noise removal step for removing noise information from the voice information; a voice transmission step for transmitting the voice information from which noise has been removed to a predetermined voice recognition device via the network; and POI (Point Of Interest) information to be received from the voice recognition device via the network. A POI information receiving step for transmitting, a POI information transmitting step for transmitting the POI information to the information terminal, a POI specifying step for receiving the POI information, and the POI information received by the POI specifying step as the output means. And a POI candidate output step of outputting to the system.

  According to the present invention, it is possible to provide a technology of an information terminal that can use a higher search function more easily.

It is a schematic diagram of a search system. It is a hardware block diagram of a relay server apparatus. It is a schematic block diagram of a navigation apparatus. It is a figure which shows the structure of a link table. It is a functional block diagram of an arithmetic processing part. It is a figure which shows the flow of an information search process. It is a sequence diagram in an information search process. It is a flowchart of a POI search result integration process. It is a flowchart of a microphone selection process. It is a flowchart of the selection process at the time of microphone utterance. It is a figure which shows the flow of the modification of an information search process. It is a sequence diagram in another modification of information search processing.

  A navigation device and a search system to which a first embodiment of the present invention is applied will be described below with reference to the drawings.

  FIG. 1 shows an overall configuration diagram of the search system 1000. The search system 1000 includes a navigation device 100 mounted on a vehicle and a relay server device 500 via a network 30 such as a wide area network such as the Internet, a LAN (Local Area Network), a WAN (Wide Area Network), or a mobile phone network. The voice specifying server device 900 and the POI providing server device 950 are connectable.

  Here, the voice identification server device 900 is a device that realizes a voice recognition service provided by a predetermined operator or the like via the network 30. In the present embodiment, when receiving the transmitted voice information (waveform information specifying the characteristics of the voice), the voice identification server device 900 performs voice recognition and transmits the recognized word as a character string. Since the recognized words usually have ambiguity, recognition that allows ambiguity such as N-best search is performed, and one or more character strings that may be applicable are transmitted according to the accuracy of recognition. .

  The POI providing server device 950 is a device that realizes a POI (Point Of Interest) search service provided by a predetermined operator or the like via the network 30. In the present embodiment, when the POI providing server device 950 receives the transmitted character string, the POI providing server device 950 includes a POI corresponding to the character string, that is, one that includes the character string or a character string similar to the character string. Alternatively, a plurality of POIs are searched and specified, and a list of POIs is transmitted according to the probability of the corresponding. In the POI list, one or a plurality of POIs are associated with each transmitted character string in descending order of accuracy, and each POI has coordinate information such as the POI name and latitude / longitude specifying the POI position. , POI address, phone number for POI, etc.

  The navigation device 100 is a so-called navigation device capable of displaying map information and indicating a point indicating the current location of the navigation device 100 and information for guiding a route to a set destination.

  When the relay server device 500 receives the POI search request and the voice information from the navigation device 100, the relay server device 500 performs noise removal on the voice information, transmits it to the voice specifying server device 900, and transmits the character transmitted from the voice specifying server device 900. The column is transmitted to the POI providing server device 950, and the received POI list is transmitted to the navigation device 100.

  Here, the configuration of the relay server device 500 will be described in more detail. The relay server device 500 includes a storage unit 510, a control unit 520, and a transmission / reception unit 530. A server information table 511 is stored in the storage unit 510, and setting information for specifying the voice specifying server device 900 for specifying voice and the POI providing server device 950 for providing POI is stored. Stored.

  The control unit 520 includes a noise removal processing unit 521 and a POI presentation unit 522. The noise removal processing unit 521 performs noise removal corresponding to each algorithm by using one or a plurality of noise removal algorithms on the voice information received from the navigation device 100. That is, when the noise removal processing unit 521 can execute, for example, four types of noise removal algorithms, each type of algorithm is applied to the voice information received from the navigation device 100 to obtain four types of noise-removed voices. Output information. Such an algorithm includes an algorithm that removes noise by adaptive filtering, a spectrum subtraction that removes the spectrum of noise in the frequency domain, and a short-time spectrum that changes in time in the frequency domain (Running Spectrum). There is an algorithm such as Running Spectrum Filter that removes noise by passing a digital filter in the time axis direction every time.

  The POI presentation unit 522 receives voice information from the navigation device 100, causes the noise removal processing unit 521 to perform noise removal, and sets one or more pieces of voice information from which noise has been removed stored in the server information table 511. Based on the information, it is transmitted to the voice identification server device 900. Upon receiving one or more recognized character strings transmitted from the voice identification server device 900, the POI presentation unit 522 transmits the received recognized character strings to the navigation device 100, and is selected by the user of the navigation device 100. When the received character string is received, a POI search request including the character string is transmitted to the POI providing server device 950. Then, the POI list transmitted from the POI providing server device 950 is transmitted to the navigation device 100.

  The transmission / reception unit 530 transmits information to other devices via the network 30 and receives information from other devices. In the present embodiment, the transmission / reception unit 530 performs communication among the navigation device 100, the voice identification server device 900, and the POI providing server device 950.

  FIG. 2 is a hardware configuration diagram of the relay server device 500. The relay server device 500 includes an input device 551, an output device 552, a communication device 553, a calculation device 554, a main storage device 555, and an external storage device 556. Each device is connected by a bus 557. Note that the input device 551 and the output device 552 are not essential components, and may be provided as necessary.

  The input device 551 is a device that receives input from a keyboard, a mouse, a touch pen, and other pointing devices. The output device 552 is a device that performs display such as a display. The communication device 553 is a device that communicates with other devices via a network such as the network 30. The communication device 553 of the relay server device 500 can communicate with the voice specifying server device 900, the POI providing server device 950, the communication device 12 of the navigation device 100, and the like through the network 30. The arithmetic device 554 is an arithmetic device such as a CPU (Central Processing Unit). The main storage device 555 is a memory device such as a RAM (Random Access Memory). The external storage device 556 is a non-volatile storage device such as a hard disk device or an SSD (Solid State Drive).

  Note that the instruction code expanded in the main storage device 555 may be stored in the external storage device 556, or on another network (not shown) on the network 30 or a network such as the Internet via the communication device 553. It may be obtained from the apparatus. The main storage device 555 has an area where the instruction code executed by the arithmetic device 554 is expanded. The external storage device 556 is a so-called normal storage device, and records in advance software for operating the relay server device 500, initial values of data required by the software, other data, and the like.

  The noise removal processing unit 521 and the POI presentation unit 522 of the control unit 520 of the relay server device 500 described above are constructed by the arithmetic device 554 reading and executing a predetermined program. Therefore, the main storage device 555 stores a program for realizing processing of each functional unit.

  Note that the components of the relay server device 500 described above are classified according to main processing contents in order to facilitate understanding of the configuration. Therefore, the present invention is not limited by the way of classifying the components and their names. The configuration of the relay server device 500 can be classified into more components depending on the processing content. Moreover, it can also classify | categorize so that one component may perform more processes.

  Further, the control unit 520 of the relay server device 500 may be constructed by hardware (ASIC, GPU, etc.). Further, the processing of each functional unit may be executed by one hardware or may be executed by a plurality of hardware.

  FIG. 3 shows an overall configuration diagram of the navigation device 100. The navigation device 100 includes an arithmetic processing unit 1, a display 2, a storage device 3, a voice input / output device 4 (including a microphone 41 as a voice input device and a speaker 42 as a voice output device), an input device 5, and a ROM. The apparatus 6 includes a vehicle speed sensor 7, a gyro sensor 8, a GPS (Global Positioning System) receiver 9, an FM multiplex broadcast receiver 10, a beacon receiver 11, and a communication device 12.

  The arithmetic processing unit 1 is a central unit that performs various processes. For example, the current location is calculated based on information output from the various sensors 7 and 8, the GPS receiver 9, the FM multiplex broadcast receiver 10, and the like. Further, map data necessary for display is read from the storage device 3 or the ROM device 6 based on the obtained current location information.

  The arithmetic processing unit 1 develops the read map data in graphics, and displays a mark indicating the current location on the display 2 in a superimposed manner. In addition, an optimal route (or a route or stopover) that connects the departure point or current location instructed by the user with the map data or the like stored in the storage device 3 or the ROM device 6 is used. Search for the recommended route. Further, the user is guided using the speaker 42 and the display 2.

  The arithmetic processing unit 1 of the navigation device 100 has a configuration in which each device is connected by a bus 25. The arithmetic processing unit 1 includes a CPU (Central Processing Unit) 21 that executes various processes such as numerical calculation and control of each device, and a RAM (Random Access Memory) that stores map data, arithmetic data, and the like read from the storage device 3. ) 22, a ROM (Read Only Memory) 23 for storing programs and data, and an I / F (interface) 24 for connecting various types of hardware to the arithmetic processing unit 1.

  The display 2 is a unit that displays graphics information generated by the arithmetic processing unit 1 or the like. The display 2 is configured by a liquid crystal display, an organic EL display, or the like.

  The storage device 3 is composed of at least a readable / writable storage medium such as an HDD (Hard Disk Drive) or a nonvolatile memory card.

  This storage medium stores a link table 200 that is map data (including link data of links constituting roads on a map) necessary for a normal route search device.

  FIG. 4 is a diagram illustrating the configuration of the link table 200. The link table 200 includes, for each mesh identification code (mesh ID) 201, which is a partitioned area on the map, link data 202 of each link constituting a road included in the mesh area.

  For each link ID 211 that is a link identifier, the link data 202 includes coordinate information 222 of two nodes (start node and end node) constituting the link, a road type 223 indicating the type of road including the link, and a link length. Link length 224 indicating the link travel time 225 stored in advance, a start connection link that is a link connected to the start node of the link, and an end connection link that is a link connected to the end node of the link A start connection link, an end connection link 226, a speed limit 227 indicating a speed limit of the road including the link, and the like are included.

  Here, by distinguishing the start node and the end node for the two nodes constituting the link, the upward direction and the downward direction of the same road are managed as different links.

  Returning to FIG. The voice input / output device 4 includes a built-in microphone 41 as a voice input device and a speaker 42 as a voice output device. The microphone 41 acquires sound outside the navigation device 100 such as a voice uttered by a user or another passenger.

  The voice input / output device 4 also has a connection unit that accepts connection of the extension microphone 43. That is, since the voice input / output device 4 can accept the connection of the extension microphone 43 such as a headset having generally higher performance sound collection performance, it can accept voice information with higher accuracy.

  The speaker 42 outputs a message to the user generated by the arithmetic processing unit 1 as voice. The microphone 41 and the speaker 42 are separately arranged at a predetermined part of the vehicle. However, it may be housed in an integral housing. The navigation device 100 can include a plurality of microphones 41 and speakers 42.

  The input device 5 is a device that receives an instruction from the user through an operation by the user. The input device 5 includes a touch panel 51, a dial switch 52, and other hardware switches (not shown) such as scroll keys and scale change keys. In addition, the input device 5 includes a remote controller that can perform operation instructions to the navigation device 100 remotely. The remote controller includes a dial switch, a scroll key, a scale change key, and the like, and can send information on operation of each key or switch to the navigation device 100.

  The touch panel 51 is mounted on the display surface side of the display 2 and can see through the display screen. The touch panel 51 specifies a touch position corresponding to the XY coordinates of the image displayed on the display 2, converts the touch position into coordinates, and outputs the coordinate. The touch panel 51 includes a pressure-sensitive or electrostatic input detection element.

  The dial switch 52 is configured to be rotatable clockwise and counterclockwise, generates a pulse signal for every rotation of a predetermined angle, and outputs the pulse signal to the arithmetic processing unit 1. The arithmetic processing unit 1 obtains the rotation angle from the number of pulse signals.

  The ROM device 6 includes at least a readable storage medium such as a ROM (Read Only Memory) such as a CD-ROM or a DVD-ROM, or an IC (Integrated Circuit) card. In this storage medium, for example, moving image data, audio data, and the like are stored.

  The vehicle speed sensor 7, the gyro sensor 8, and the GPS receiver 9 are used by the navigation device 100 to detect the current location (own vehicle position). The vehicle speed sensor 7 is a sensor that outputs a value used to calculate the vehicle speed. The gyro sensor 8 is composed of an optical fiber gyro, a vibration gyro, or the like, and detects an angular velocity due to the rotation of the moving body. The GPS receiver 9 receives a signal from a GPS satellite and measures the distance between the mobile body and the GPS satellite and the rate of change of the distance with respect to three or more satellites to thereby determine the current location, travel speed, and travel of the mobile body. It measures the direction.

  The FM multiplex broadcast receiver 10 receives an FM multiplex broadcast signal transmitted from an FM broadcast station. FM multiplex broadcasting includes VICS (Vehicle Information Communication System: Registered Trademark) information, current traffic information, regulatory information, SA / PA (service area / parking area) information, parking information, weather information, and FM multiplex general information. As text information provided by radio stations.

  The beacon receiving device 11 receives rough current traffic information such as VICS information, regulation information, SA / PA (service area / parking area) information, parking lot information, weather information, emergency alerts, and the like. For example, it is a receiving device such as an optical beacon that communicates by light and a radio beacon that communicates by radio waves.

  The communication device 12 is a device that connects the navigation device 100 to the network 30 or the like and communicates with other devices such as the relay server device 500 connected to the network. The communication device 12 may be built in the navigation device 100 or may be mounted as an external device such as a communication module or a mobile phone using a mobile phone network. Good. In addition, information is transmitted and received between the navigation apparatus 100 and the communication apparatus 12 according to a predetermined communication standard such as USB (Universal Serial Bus) or Bluetooth (registered trademark).

  FIG. 5 is a functional block diagram of the arithmetic processing unit 1. As illustrated, the arithmetic processing unit 1 includes a basic control unit 101, an input reception unit 102, an output processing unit 103, a noise level determination unit 104, a relay server communication unit 105, and a POI presentation information creation unit 106. And microphone recognition unit 107.

  The basic control unit 101 is a central functional unit that performs various processes, and controls other processing units according to the processing content. In addition, information on various sensors, the GPS receiver 9 and the like is acquired, and a map matching process is performed to identify the current location. In addition, the travel history is stored in the storage device 3 for each link by associating the travel date and time with the position as needed. Further, the current time is output in response to a request from each processing unit.

  The input receiving unit 102 receives an instruction from the user input via the input device 5 or the microphone 41, and controls each unit of the arithmetic processing unit 1 so as to execute processing corresponding to the requested content. For example, when the user requests a search for a recommended route, the output processing unit 103 is requested to display a map on the display 2 in order to set a destination.

  The output processing unit 103 receives screen information to be displayed such as polygon information, for example, converts it into a signal for drawing on the display 2, and instructs the display 2 to draw.

  The noise level determination unit 104 determines the noise level of the voice information input from the microphone 41 or the extension microphone 43 of the voice input / output device 4. Specifically, the noise level determination unit 104 extracts a noise component included in a waveform of a predetermined silence portion of the received audio information, preferably a silence portion waveform corresponding to the first 100 ms of the audio information, and the noise amount It is determined whether or not the noise level is equal to or higher than a predetermined level according to the amount of the error.

  The relay server communication unit 105 transmits the input voice information to the relay server device 500. Further, the relay server communication unit 105 receives a character string obtained as a result of speech recognition from the relay server device 500. Also, the relay server communication unit 105 transmits the selected POI information to the relay server device 500 and receives the POI list information.

  The POI presentation information creation unit 106 integrates the received POI list information to create screen information or the like for presentation to the user so that it can be selected as a POI list, and requests the output processing unit 103 to output it.

  The microphone recognition unit 107 recognizes a microphone connected to the navigation device 100. Specifically, the microphone recognition unit 107 detects that the extension microphone 43 is connected, and uses the microphone according to the user's selection as to which microphone to use with the built-in microphone 41. Identify the microphone.

  Each functional unit of the arithmetic processing unit 1 described above, that is, the basic control unit 101, the input reception unit 102, the output processing unit 103, the noise level determination unit 104, the relay server communication unit 105, the POI presentation information creation unit 106, and the microphone recognition unit 107. Is constructed by the CPU 21 reading and executing a predetermined program. Therefore, the RAM 22 stores a program for realizing the processing of each functional unit.

  In addition, each above-mentioned component is classified according to the main processing content, in order to make an understanding easy the structure of the navigation apparatus 100. FIG. Therefore, the present invention is not limited by the way of classifying the components and their names. The configuration of the navigation device 100 can be classified into more components depending on the processing content. Moreover, it can also classify | categorize so that one component may perform more processes.

  Each functional unit may be constructed by hardware (ASIC, GPU, etc.). Further, the processing of each functional unit may be executed by one hardware or may be executed by a plurality of hardware.

  [Description of Operation] Next, the operation of the POI search process performed in the search system 1000 including the navigation device 100, the relay server device 500, the voice specifying server device 900, and the POI providing server device 950 will be described. FIG. 6 is a flowchart showing the POI search process. This flow is started by receiving a voice input start instruction using a predetermined PTT (Push To Talk) button or the like while the navigation device 100 is activated.

  First, the input receiving unit 102 starts waiting for voice input (step S001). Then, the input receiving unit 102 detects the voice section until the voice standby is completed by releasing the PTT button or the like (step S003), and compresses the input voice information to create voice information (step S002). Here, the noise level determination unit 104 determines the noise level of the input voice information. When the noise level is higher than a predetermined level, that is, in an environment where there is a lot of noise, the input receiving unit 102 may set the compression rate to be low and perform compression to minimize sound quality deterioration due to compression. . In addition, in a case where the noise level is higher and exceeds a predetermined threshold, that is, in an environment where the noise is too large to be able to perform speech recognition normally, the input receiving unit 102 does not create voice information. The subsequent processing may not be performed.

  Then, the input reception unit 102 transmits voice information to the relay server device 500 via the relay server communication unit 105. And the noise removal process part 521 of the relay server apparatus 500 implements the noise removal process which implement | achieves a predetermined algorithm with respect to the received audio | voice information (step S004). Specifically, the noise removal processing unit 521 performs noise removal processing on the received voice information according to one or a plurality of algorithms determined in advance to be applied at the time of noise removal processing, and noise is removed. Generating one or more audio information.

  Then, the POI presentation unit 522 transmits one or more pieces of voice information from which noise has been removed to the voice identification server device 900. Then, the voice identifying server device 900 transmits one or more candidate character string information, which is a result of performing recognition by performing predetermined voice recognition processing on each voice information, to the relay server device 500 (step S005). In the speech recognition processing, existing speech recognition processing or the like is performed, and character strings that are candidates for one or a plurality of recognition results are output together with the accuracy by N-best search or the like. For example, if the voice information spoken by the user corresponds to “pizza”, character strings that are candidates for “pizza”, “Pizza”, “knee”, “squid”, etc. are output for each voice information. The

  Upon receiving the output character string information of the recognition result, the POI presentation unit 522 weights the recognition result (step S006). Specifically, the output recognition result string information includes one or more candidates according to the noise removal algorithm, and if there are duplicate candidates among them, they are integrated into one. The candidates are corrected with higher accuracy (for example, a predetermined ratio is added to the accuracy), and the candidate character strings are ranked in the order of accuracy. Note that the POI presenting unit 522 may perform weighting in the weighting process according to a noise removal algorithm applied to the voice information. That is, since speech information to which an appropriate noise removal algorithm is applied is considered to have high recognition accuracy, priority may be given to candidates that are considered to have high recognition accuracy. In addition, if there is a candidate for a POI that is highly unlikely to be a facility, it may be removed.

  The POI presenting unit 522 transmits the ranked candidate character strings to the navigation device 100. Then, the POI presentation information creation unit 106 of the navigation device 100 creates screen information that allows the received list of recognized recognition result candidate character strings to be selected, and instructs the output processing unit 103 to display the screen information. 2 is displayed (step S007). Here, when the number of ranked character string candidates is less than a predetermined number, or when the level of noise included in the audio information received in step S002 is lower than a predetermined level, that is, speech recognition. When there is little ambiguity in the result, the process of step S008 described later may be omitted and the process of transmitting a POI search request in step S009 may be performed.

  Then, the input receiving unit 102 receives a candidate selection input designated by the user on the displayed screen, and transmits it to the relay server device 500 via the relay server communication unit 105 (step S008).

  The POI presentation unit 522 transmits the transmitted candidate character string to the POI providing server device 950, and transmits a POI search request (S009).

  The POI providing server device 950 fluctuates and searches for a POI that includes the transmitted candidate character string in the facility name or address, or includes a character string similar to the transmitted candidate character string in the facility name or address. Then, a plurality of POI candidates are searched by accuracy, and a POI list including information including the name, coordinates, telephone number, address, etc. of the POI is transmitted to the relay server device 500 (step S010).

  The POI presentation unit 522 integrates the transmitted POI list by performing POI search result integration processing described later (step S011). Then, the POI presentation unit 522 transmits the integrated POI search result to the navigation device 100.

  The POI presentation information creation unit 106 creates a display screen for selectively displaying each POI using the received POI search result, and instructs the output processing unit 103 to display it on the display 2 (step S012). . For example, when the selected candidate is “pizza”, the POI presentation information creation unit 106 displays a list of restaurants that provide pizza foods in a selectable manner, and according to the coordinate position of the restaurant, Create a screen that displays the icon of the restaurant at the appropriate location on the map.

  Then, the basic control unit 101 receives selection of the displayed POI via the input reception unit 102, and performs a route search using the selected POI as a destination or waypoint (step S013). During the route search, the basic control unit 101 displays a route search message including the name of the selected POI. For example, if the name of the selected POI is “Tokyo Pizza”, the basic control unit 101 displays a message “Searching for a route to Tokyo Pizza” and performs a route search to the POI. .

  The above is the flow of the POI search process. According to the POI search process, a higher search function can be used more easily. Specifically, even if the processing capability of the navigation device is not particularly high, it can be said that highly accurate voice recognition and a highly functional POI search function can be used.

  FIG. 7 is a sequence diagram showing the relationship between processes in steps S002 to S007 in the POI search process shown in FIG.

  First, the relay server communication unit 105 of the navigation device 100 starts transmitting voice information to the transmission / reception process (controlled by the POI presenting unit 522) of the relay server device 500 (step S101).

  Then, the relay server communication unit 105 continues to transmit the audio information until the transmission of all the audio information is completed (step S102).

  In the transmission / reception process of the relay server device 500, after the transmission of the audio information from the relay server communication unit 105 is finished, the transmission of the audio information is started to the noise removal process controlled by the noise removal processing unit 521 (step S103). When the transmission of all audio information is completed, the transmission / reception process of the relay server device 500 ends the transmission of the audio information to the noise removal process (step S104).

  In the noise removal process, the noise removal processing unit 521 performs a predetermined noise removal process on the transmitted voice information (step S105). The noise removal processing unit 521 of the relay server device 500 starts transmitting the voice information from which noise has been removed to the server communication process controlled by the POI presenting unit 522 (step S106). When the transmission of the voice information from which noise has been removed is completed, the transmission of the voice information to the server communication process is completed in the noise removal process of the relay server device 500 (step S107).

  When all the speech information from which noise has been removed is received, in the server communication process, the POI presenting unit 522 starts transmission of the speech information from which noise has been removed from the noise removal process to the speech identification server device 900. (Step S108). Here, when there is audio information for each noise removal algorithm, a plurality of audio information are all transmitted.

  When the transmission of the voice information from which noise has been removed is completed, the transmission of the voice information to the voice specifying server apparatus 900 is completed in the server communication process of the relay server apparatus 500 (step S109).

  Then, the voice identification server device 900 performs a predetermined voice recognition process on the received noise-removed voice information, and specifies one or a plurality of candidate character strings obtained as a result of the recognition by N-best search. (Step S110).

  Then, in the server communication process of the relay server device 500, the POI presentation unit 522 receives all candidate character strings transmitted from the voice identification server device 900 (step S111).

  In the server communication process of the relay server device 500, the POI presentation unit 522 sends all the character strings received in the transmission / reception process to the transmission / reception process (step S112).

  In the transmission / reception process of the relay server device 500, the POI presentation unit 522 transmits the character string of the recognition result to the relay server communication unit 105 of the navigation device 100 (step S113).

  The above is the relationship between processes in steps S002 to S007 in the POI search process shown in FIG. Note that detailed explanations of processes such as noise removal (step S004) and recognition result weighting (step S006) are omitted in the description of the relationship between the processes.

  FIG. 8 is a diagram showing a flow of POI search result integration processing. The POI search result integration process is a process executed by the relay server device 500 in step S011 of the POI search process of FIG.

  First, if the POI list is the same among the recognized character strings, the POI presenting unit 522 deletes the recognized character string and the POI list with low accuracy (step S201). Specifically, the POI presenting unit 522 compares each POI list with respect to one or a plurality of recognized character strings received from the POI providing server device 950, their accuracy, and the corresponding POI list, and constructs a POI list. When there is a POI list in which the POIs to be completely matched, the POI list for the recognized character string with low accuracy is deleted, and the recognized character string and the accuracy information are also deleted. This is repeated until there is no overlap between all POI lists.

  Then, if there are overlapping POIs in the POI list, the POI presenting unit 522 deletes the POI with low accuracy from the list (step S202). Here, it is assumed that the POI list includes POI information and information on the accuracy of the POI with respect to the recognized character string. When a plurality of POIs having the same POI name are included in the POI list for one recognized character string, the POI presenting unit 522 deletes the POI with low accuracy from the POI list and eliminates duplication.

  Then, if there is a POI common to the recognized character strings, the POI presenting unit 522 deletes the POI of the recognized character string with low accuracy (step S203). Specifically, the POI presenting unit 522 compares the POI lists corresponding to the recognized character strings between the recognized character strings, and when the POIs having the same POI name are included in each POI list, the accuracy is low. The POI included in the POI list associated with the recognized character string is deleted from the POI list to eliminate duplication.

  Next, the POI presenting unit 522 moves the POI of the recognized character string with low accuracy to the POI list of the recognized character string with high accuracy between similar recognized character strings (step S204). Specifically, the POI presenting unit 522 specifies a combination in which the degree of similarity between character strings is greater than or equal to a predetermined value between the recognized character strings, and the POI associated with the recognized character string with low accuracy in the combination. The POI list is integrated by moving the POI information included in the list to the lower level of the POI list associated with the recognized character string with high accuracy.

  The above is the flow of the POI search result integration process. According to the POI search result integration process, it is possible to obtain a POI list from which duplication is eliminated while integrating the POI lists searched by the search character strings having similar relationships with the recognized character strings.

  FIG. 9 is a processing flow of microphone selection processing performed in the navigation device 100. The microphone selection process is performed when the basic control unit 101 or the like newly detects the connection of the extension microphone 43 in the navigation device 100.

  First, the microphone recognizing unit 107 determines whether or not a newly connected microphone (hereinafter referred to as a new microphone) is closer to the already connected microphone (step S301). Specifically, the microphone recognizing unit 107 determines that the new microphone is “closest” if the new microphone is a headset microphone that is assumed to be located near the mouth of the speaker. If the new microphone is an external microphone, it is determined to be “closer” than the built-in microphone 41. However, the microphone recognition unit 107 does not determine that the new microphone is “close” when the headset microphone is already connected and the new microphone is an external microphone.

  When the new microphone is not closer than the already connected microphone (in the case of “No” in step S301), the microphone recognizing unit 107 ends the microphone selection process.

  When the new microphone is closer to the already connected microphone (in the case of “Yes” in step S301), the microphone recognizing unit 107 performs display for inquiring about the change of the microphone to be used (step S302). Specifically, the microphone recognizing unit 107 outputs a dialog box or the like that accepts a change instruction that becomes a response such as affirmative / negative for the message, along with a message such as “Would you like to set a new microphone as a normal microphone?” The output processing unit 103 is instructed to do so.

  When the microphone recognizing unit 107 receives a change instruction that becomes a response such as affirmative / negative for the displayed inquiry, the microphone recognizing unit 107 determines whether the received instruction is affirmative (step S303). If the content is not affirmative, the microphone recognition unit 107 ends the microphone selection process.

  When the content to be affirmed is received (in the case of “Yes” in step S303), the microphone recognizing unit 107 changes the microphone used for voice recognition to a new microphone (step S304). Specifically, the microphone recognizing unit 107 associates the setting of the microphone that receives input during the voice recognition processing with the new microphone.

  The above is the flow of the microphone selection process. According to the microphone selection process, when a new microphone is recognized, if the microphone is closer to the already connected microphone, whether or not to use the microphone for voice recognition depends on the user's instruction. Can be set. Note that the microphone selection process may be set with priority when there is a microphone whose designation has been received in advance by the user.

  Note that the microphone recognized as a new microphone in the microphone selection process is not limited to a microphone such as a headset microphone. For example, the navigation device 100 may be a microphone connected to a mobile phone or the like (Bluetooth: Bluetooth), an FM transmitter, or the like. Any device that can accept voice input, such as a microphone that communicates with the device, may be used.

  Next, the microphone utterance selection process performed by the microphone recognition unit 107 when the user speaks will be described with reference to FIG. The microphone utterance selection process is a process of selectively adopting an input from a microphone with good sound quality and maintaining high input sound quality when input can be received by a plurality of microphones.

  First, the basic control unit 101 receives sound in all input microphones (step S401). Note that the microphones that receive audio may not be all microphones, but may be a plurality of microphones designated in advance, or a predetermined number of microphones selected in descending order of gain.

  Next, the microphone recognizing unit 107 identifies the noise level for each microphone and adopts the sound received by the low-level microphone (step S402). Specifically, the microphone recognizing unit 107 specifies the noise level (S / N ratio) for the input audio information for each microphone via the noise level determining unit 104, and the audio information with the lowest noise ratio is obtained. Is used to identify the input voice.

  The above is the processing flow of the microphone utterance selection process. According to the microphone utterance selection process, it is possible to adopt voice information with good sound quality among the voice information actually spoken, so the best sound quality for each utterance when the acoustic environment such as the vehicle changes from time to time. It can be said that voice input can be performed.

  The first embodiment of the present invention has been described above. According to the first embodiment of the present invention, the navigation device 100 can use a higher search function more easily. Note that the microphone utterance selection process may be preferentially adopted when there is a microphone that has been designated by the user.

  The present invention is not limited to the above embodiment. The above embodiment can be variously modified within the scope of the technical idea of the present invention. For example, as shown in FIG. 11, a second embodiment may be adopted in which the flow of POI search processing is changed so that a POI search is performed without waiting for a user's instruction for a character string obtained as a result of speech recognition.

  A second embodiment will be described below. The second embodiment is a search system basically having substantially the same configuration as the first embodiment, and there is a difference in POI search processing. The following description will be focused on the difference.

  First, the input receiving unit 102 starts waiting for voice input (step S501). Then, the input receiving unit 102 detects a voice interval and compresses the input voice information to create voice information (step S502) until the voice waiting is ended by releasing the PTT button or the like (step S503). Here, the noise level determination unit 104 determines the noise level of the input voice information. When the noise level is higher than a predetermined level, that is, in an environment where there is a lot of noise, the input receiving unit 102 may set the compression rate to be low and perform compression to minimize sound quality deterioration due to compression. . In addition, in a case where the noise level is higher and exceeds a predetermined threshold, that is, in an environment where the noise is too large to be able to perform speech recognition normally, the input receiving unit 102 does not create voice information. The subsequent processing may not be performed.

  Then, the input reception unit 102 transmits voice information to the relay server device 500 via the relay server communication unit 105. And the noise removal process part 521 of the relay server apparatus 500 implements the noise removal process which implement | achieves a predetermined algorithm with respect to the received audio | voice information (step S504). Specifically, the noise removal processing unit 521 performs noise removal processing on the received voice information according to one or a plurality of algorithms determined in advance to be applied at the time of noise removal processing, and noise is removed. Generating one or more audio information.

  Then, the POI presentation unit 522 transmits one or more pieces of voice information from which noise has been removed to the voice identification server device 900. The voice identification server device 900 transmits one or a plurality of character string information that is a result of recognition by performing predetermined voice recognition processing on each voice information to the relay server device 500 (step S505). In the speech recognition processing, existing speech recognition processing or the like is performed, and character strings that are candidates for one or a plurality of recognition results are output together with the accuracy by N-best search or the like. For example, if the voice information spoken by the user corresponds to “pizza”, character strings that are candidates for “pizza”, “Pizza”, “knee”, “squid”, etc. are output for each voice information. The

  Then, the POI presenting unit 522 receives the output character string information of the recognition result, and weights the recognition result (step S506). Specifically, the output character string information of the recognition result includes one or a plurality of candidates according to the noise removal algorithm, and the POI presenting unit 522 determines that there is a duplicate candidate. The accuracy of the integrated candidates is corrected to be higher (for example, a predetermined ratio is added to the accuracy), and the candidate character strings are ranked in the order of accuracy.

  The POI presentation unit 522 transmits the ranked candidate character strings (plurality) to the POI providing server device 950, and transmits a POI search request (S0507).

  The POI providing server device 950 includes each of the transmitted candidate character strings in the facility name or address, or includes a character string similar to each of the transmitted candidate character strings in the facility name or address. A relay server apparatus that searches for fluctuations in POI, searches for one or more POI candidates by accuracy, and associates a POI list including information including the name, coordinates, telephone number, address, etc. of the POI with a candidate character string It transmits to 500 (step S508).

  The POI presentation unit 522 transmits the transmitted POI list for each candidate character string to the navigation device 100 (step S509).

  The POI presentation information creation unit 106 uses the received POI search result to create a display screen that displays each POI for each candidate character string so as to be selectable, and displays it on the display 2 on the output processing unit 103. (Step S510). For example, when the candidate character strings are “pizza”, “Pizza”, and “knee”, the POI presentation information creation unit 106 selects a list of restaurants that provide pizza food for each candidate character string. A screen for displaying the icon of the restaurant at a corresponding position on the map is created according to the coordinate position of the restaurant.

  Then, the basic control unit 101 receives selection of the displayed POI via the input reception unit 102, and performs a route search using the selected POI as a destination or waypoint (step S511). During the route search, the basic control unit 101 displays a route search message including the name of the selected POI. For example, if the name of the selected POI is “Tokyo Pizza”, the basic control unit 101 displays a message “Searching for a route to Tokyo Pizza” and performs a route search to the POI. .

  The above is the flow of the POI search process according to the second embodiment. According to the POI search process according to the second embodiment, the POI can be searched and presented for each of one or more candidates of the character string recognized based on the voice information, so that the user's input operation is further reduced, You can easily use the high search function. Specifically, even if the processing capability of the navigation device is not particularly high, it can be said that highly accurate voice recognition and a highly functional POI search function can be used.

  Further, for example, in the first embodiment, as shown in FIG. 7, the voice information is collectively transmitted from the navigation device 100 to the relay server device 500 or the like after the utterance is finished, as shown in FIG. Not limited to. That is, voice information may be transmitted from the navigation device 100 to the relay server device 500 or the like during speech, and noise removal may be sequentially performed. Such deformation processing will be described with reference to FIG.

  First, the relay server communication unit 105 of the navigation device 100 starts transmitting voice information to the transmission / reception process (controlled by the POI presenting unit 522) of the relay server device 500 (step S601).

  Note that the relay server communication unit 105 continues to transmit the audio information until the transmission of all the audio information is completed (step S607).

  In the transmission / reception process of the relay server device 500, when the transmission of the voice information is started, the POI presentation unit 522 does not wait for the transmission of the voice information from the relay server communication unit 105 to end, and the noise removal processing unit 521 Transmission of voice information is started to the controlled noise removal process (step S602). When the transmission of all audio information is completed, the transmission / reception process of the relay server device 500 ends the transmission of the audio information to the noise removal process (step S608).

  In the noise removal process, the noise removal processing unit 521 performs predetermined noise removal processing on the transmitted voice information. First, the noise removal processing unit 521 of the relay server device 500 determines a noise level for a silent part at the beginning of the received voice information for a predetermined time (for example, 100 milliseconds), and selects an appropriate level according to the noise level. One or more denoising algorithms are determined (step S603). Then, the noise removal processing unit 521 of the relay server device 500 applies the determined noise removal algorithm to perform noise removal on the received voice information (step S604), and sequentially from the part from which the noise has been removed. Then, transmission to the server communication process controlled by the POI presenting unit 522 is started (step S605). When transmission of all audio information from which noise has been removed is completed, transmission of audio information to the server communication process ends in the noise removal process of relay server device 500 (step S609).

  When the voice information from which the noise has been removed is received, in the server communication process, the POI presenting unit 522 starts transmitting the voice information from which the noise has been removed from the noise removal process to the voice identification server device 900 in sequence (step). S606). Here, when voice information exists for each noise removal algorithm, POI presenting section 522 transmits all of the plurality of voice information to which different noise removal algorithms are applied.

  When the transmission of the voice information from which noise has been removed is completed, the transmission of the voice information to the voice specifying server apparatus 900 is completed in the server communication process of the relay server apparatus 500 (step S610).

  Then, the voice specifying server device 900 performs a predetermined voice recognition process on the received noise-removed one or more pieces of voice information, and performs N-best search for candidate character strings obtained as a result of the recognition. One or more are specified (step S611).

  Then, in the server communication process of the relay server device 500, the POI presentation unit 522 receives all candidate character strings transmitted from the voice identification server device 900 (step S612).

  In the server communication process of relay server device 500, POI presentation unit 522 sends all received character strings to the transmission / reception process (step S613).

  In the transmission / reception process of the relay server device 500, the POI presentation unit 522 transmits the character string of the recognition result to the relay server communication unit 105 of the navigation device 100 (step S614).

  The above is a modified example of the relationship between processes in steps S002 to S007 in the POI search process. Note that detailed explanations of processes such as noise removal (step S004) and recognition result weighting (step S006) are omitted in the description of the relationship between the processes.

  By transforming in this way, it is possible to perform in real time from the speech of speech information to the start of speech recognition, so that the start timing of speech recognition processing can be advanced and responsiveness can be enhanced.

  In the above, this invention was demonstrated centering on embodiment. In each of the above-described embodiments, examples in which the present invention is applied to a navigation device and the like have been described. However, the present invention is not limited to a navigation device and can be applied to all information terminals. In each embodiment, the processing performed by the navigation device 100 and the processing unit used to perform the processing are provided in another terminal device (for example, a mobile phone or a smartphone via the communication device 12). A part of the processing may be distributed between the navigation device 100 and other terminal devices.

DESCRIPTION OF SYMBOLS 1 ... Arithmetic processing part, 2 ... Display, 3 ... Memory | storage device, 4 ... Voice output device, 5 ... Input device, 6 ... ROM device, 7 ... Vehicle speed sensor , 8 ... Gyro sensor, 9 ... GPS receiver, 10 ... FM multiplex broadcast receiver, 11 ... Beacon receiver, 12 ... Communication device, 21 ... CPU, 22 ... RAM 23 ... ROM 24 ... I / F 25 ... bus 30 ... network 41 ... microphone 42 ... speaker 43 ... extension microphone 51 ..Touch panel, 52 ... Dial switch, 100 ... Navigation device, 101 ... Basic control unit, 102 ... Input reception unit, 103 ... Output processing unit, 104 ... Noise level determination unit 105 ... Relay server Communication unit 106 ... POI presentation information creation unit 107 ... Microphone recognition unit 200 ... Link table 500 ... Relay server device 510 ... Storage unit 520 ... Control unit 530 ... Transmission / reception unit, 1000 ... Search system

In order to solve the above problems, a server device according to the present invention includes a voice information receiving unit that receives voice information from a predetermined information terminal via a network, and a plurality of noise removal algorithms to apply noise information from the voice information. Noise removal means for generating a plurality of pieces of speech information from which noise has been removed in accordance with the applied noise removal algorithm, and the network from the speech information from which noise has been removed to a predetermined speech recognition device. Voice transmission means for transmitting via the network, character string reception means for receiving a plurality of character string candidates that are recognition results of the voice information by the voice recognition device via the network, and an applied noise removal algorithm One or a plurality of the above-described character string candidates that are weighted in accordance with the predetermined information that provides POI (Point Of Interest) information Character string transmitting means for transmitting to the OI providing apparatus via the network, POI information receiving means for receiving the POI information related to the character string candidate via the network, and the POI information for the information terminal And a POI information transmitting means for transmitting to .

Claims (18)

Voice input receiving means for receiving voice input;
Communication means for communicating with a predetermined server device via a network;
Output means;
POI identifying means for transmitting the voice information accepted by the voice input accepting means to the server device and receiving information for identifying a POI (Point Of Interest) candidate related to the voice information;
POI candidate output means for outputting information specifying the POI candidates received by the POI specifying means to the output means;
Route search means for receiving a selection input of information for identifying the POI candidate and searching for a route to the POI;
An information terminal comprising: The information terminal according to claim 1,
The POI specifying means further includes:
When a plurality of character string candidates obtained as a result of recognizing the speech information are received from the server device, the received character strings are selectably output to the output means, and one selection input of the character string candidates is received. Accept,
An information terminal characterized by that. The information terminal according to claim 1 or 2,
The POI specifying means is:
The voice received by the voice input accepting unit is information compressed at a predetermined compression rate and transmitted to the server device,
If the quality of the voice received by the voice input receiving means is less than or equal to a predetermined value, the predetermined compression rate is set to reduce deterioration
An information terminal characterized by that. The information terminal according to any one of claims 1 to 3,
The POI specifying unit does not transmit the audio information to the server device when the quality of the audio received by the audio input receiving unit is below a specified level.
An information terminal characterized by that. The information terminal according to any one of claims 1 to 4, wherein
The POI specifying means sequentially transmits the voice received by the voice input receiving means to the server device.
An information terminal characterized by that. The information terminal according to any one of claims 1 to 5,
The POI candidate output means includes:
In the process of outputting the information specifying the POI candidate received by the POI specifying means to the output means, when there are a plurality of character strings obtained as a result of recognizing the speech information, the POI candidate is Output to the output means in association with each character string,
An information terminal characterized by that. Comprising voice input receiving means for receiving one or more voice inputs;
The voice input receiving means identifies and adopts a voice input with better sound quality when receiving a plurality of voice inputs;
An information terminal characterized by that. Control means;
One or a plurality of voice input receiving means,
The voice input receiving means receives one or a plurality of voice inputs;
When the control means includes a plurality of the voice input receiving means, the control means adopts a voice input receiving means close to the user.
An information terminal characterized by that. Control means;
One or a plurality of voice input receiving means,
The voice input receiving means receives one or a plurality of voice inputs;
When the control means includes a plurality of voice input reception means, if there is a voice input reception means designated in advance by the user, the designated voice input reception means is adopted.
An information terminal characterized by that. Voice information receiving means for receiving voice information from a predetermined information terminal via a network;
Noise removing means for removing noise information from the voice information;
Voice transmission means for transmitting the voice information from which noise has been removed to a predetermined voice recognition device via the network;
POI information receiving means for receiving POI (Point Of Interest) information related to the character string via the network;
POI information transmitting means for transmitting the POI information to the information terminal;
A server apparatus comprising: The server device according to claim 10, further comprising:
A character string receiving means for receiving a character string, which is a recognition result of the voice information by the voice recognition device, via the network;
A character string transmitting means for transmitting the character string via the network to a predetermined POI providing device that provides POI (Point Of Interest) information related to the character string;
A server apparatus comprising: The server device according to claim 11, further comprising:
If the character string received by the character string receiving means includes a plurality of character string candidates, the character string candidate transmitting means for transmitting the character string candidates to the information terminal;
A selection character string receiving means for receiving, from the information terminal, one candidate character string selected from the character string candidates;
The character string transmitting unit transmits the one candidate character string via the network to the POI providing device when the selected character string receiving unit receives the one candidate character string.
A server device characterized by that. The server device according to claim 11, further comprising:
When the character string received by the character string receiving means includes a plurality of character string candidates, the character string transmitting means transmits the plurality of character strings via the network to the POI providing device,
The POI information receiving means receives information on a plurality of POIs associated with the plurality of character strings.
A server device characterized by that. The server device according to any one of claims 10 to 13, further comprising:
POI deduplication means for eliminating duplication when the POI information received by the POI information receiving means includes a plurality of POIs;
A server apparatus comprising: The server device according to any one of claims 10 to 14,
The noise removing means generates a plurality of speech information from which noise has been removed using a plurality of noise removing algorithms.
A server device characterized by that. The server device according to any one of claims 10 to 15, wherein
The noise removing means includes
When the voice information receiving means starts receiving voice information from a predetermined information terminal via a network, noise information is sequentially removed from the received information,
The voice transmission means sequentially transmits voice information from which noise has been removed by the noise removal means to a predetermined voice recognition device via the network.
A server device characterized by that. A search system comprising an information terminal and a server device that communicates with the information terminal via a network,
The information terminal
Voice input receiving means for receiving voice input;
Communication means for communicating with the server device via a network;
Output means;
POI identifying means for transmitting the voice information accepted by the voice input accepting means to the server device and receiving information for identifying a POI (Point Of Interest) candidate related to the voice information;
POI candidate output means for outputting information specifying the POI candidates received by the POI specifying means to the output means;
Route search means for receiving a selection input of information for identifying the POI candidate and searching for a route to the POI;
With
The server device is
Voice information receiving means for receiving voice information from the information terminal via a network;
Noise removing means for removing noise information from the voice information;
Voice transmission means for transmitting the voice information from which noise has been removed to a predetermined voice recognition device via the network;
POI information receiving means for receiving POI (Point Of Interest) information related to the character string via the network;
POI information transmitting means for transmitting the POI information to the information terminal;
A search system comprising: A search method for a search system, comprising: an information terminal; and a server device that communicates with the information terminal via a network,
The search system includes:
Voice input receiving means for receiving voice input;
An output means,
A transmitting step of transmitting information of the sound received by the sound input receiving means to the server device;
A voice information receiving step for receiving voice information from the information terminal via the network;
A noise removing step for removing noise information from the voice information;
A voice transmission step of transmitting the voice information from which noise has been removed to a predetermined voice recognition device via the network;
A POI information receiving step of receiving POI (Point Of Interest) information related to the character string from the voice recognition device via the network;
A POI information transmission step of transmitting the POI information to the information terminal;
A POI specifying step for receiving the POI information;
A POI candidate output step of outputting the POI information received in the POI specifying step to the output means;
The search method characterized by implementing.

Images