JP4791699B2 - Business support system and method - Google Patents

Description

  The present invention relates to a system and a method for supporting various operations such as inspection of equipment and the like, and more particularly, to a system and method for performing business support in two directions using voice.

  For example, in a substation of an electric power company, maintenance and inspection work is regularly performed to confirm that various facilities for transmission and substation work normally. Conventionally, such maintenance and inspection work has been done by dispatching workers to the inspection site, sending inspection work instructions from the monitoring center to the site workers by telephone or fax, and checking results from the inspection workers by phone or fax. It is done by sending to the center.

However, in a method using a communication means such as a telephone or FAX, it takes time to transmit inspection instructions and inspection results, and the contents are likely to be inaccurate. Therefore, for example, in the inspection work support system disclosed in Patent Document 1, it is possible to support the inspection work while exchanging information by voice interactively with the inspection worker by using voice recognition / synthesis technology. ing. Specifically, a wearable computer carried by an inspection operator is composed of a headset, a mobile phone, a voice recognition device, and a notebook personal computer. The voice of the inspection worker is input to the voice recognition device and recognized, and the voice recognition result is transmitted from the notebook personal computer to the host computer via the mobile phone. Is transmitted to the notebook personal computer, so that two-way information transmission by voice can be performed between the inspector and the host computer.
JP 2002-245188 A

  If the voice recognition is performed only on the portable terminal (wearable computer) side as in the system described in Patent Document 1, the processing load related to voice recognition on the portable terminal side becomes large. This is not preferable from the viewpoint of reducing the size and weight of the terminal. On the other hand, the present applicant has proposed a business support system having a configuration in which a voice recognition function is provided on the server side in Japanese Patent Application No. 2003-363716.

  However, in the configuration in which voice recognition is performed on the server side, it is assumed that a voice signal can be transmitted from the portable terminal to the server. For this reason, communication between the portable terminal and the server must be ensured. However, for example, when working in an environment where the radio wave condition is poor, such as in remote areas, mountainous areas, concrete buildings, or in a car running at high speed, communication between the mobile terminal and the server is interrupted. In such a case, since the speech recognition process cannot be performed, the business support must be stopped.

  The present invention has been made in view of the above points. In a situation where the portable terminal carried by the user and the server are both provided with a voice recognition function, communication cannot be performed between the portable terminal and the server. An object of the present invention is to provide a business support system and method capable of providing two-way voice work support to a user even if it exists.

In order to achieve the above object, the present invention is a system for supporting the business of a user, which is composed of a mobile terminal carried by the user and a server having communication means for communicating with the mobile terminal. And
The portable terminal is
A communication determination unit that determines whether or not communication with the server is possible;
A microphone,
When the communication determination unit determines that communication with the server is not possible, the mobile terminal is set to perform voice recognition based on a voice signal input from the microphone , and the communication determination unit A process switching unit configured to set the voice recognition to be performed by the server when it is determined that the server can communicate with the server ;
A terminal-side voice recognition unit that performs voice recognition based on the voice signal when the process switching unit is set to perform voice recognition on the portable terminal;
Based on the voice recognition result by the terminal-side voice recognition unit, a terminal-side support execution unit that outputs a message for the user as a voice signal;
A voice transmission unit configured to transmit voice information corresponding to a voice signal input from the microphone to the server when the processing switching unit is set to perform voice recognition in the server;
A recognition result storage unit for storing a speech recognition result by the terminal side speech recognition unit in a state where communication with the server is not possible;
A recognition result transmission unit that transmits the stored voice recognition result to the server when it is determined by the communication determination unit that communication with the server is possible ;
The server
A voice information receiving unit that receives the voice information transmitted from the portable terminal;
A server-side voice recognition unit that performs voice recognition based on the received voice information;
A server-side support execution unit that generates a message to be notified to the user based on a voice recognition result by the server-side voice recognition unit;
A message transmission unit that transmits audio information corresponding to the generated message to the mobile terminal;
And a storage unit for storing a speech recognition result by the server-side speech recognition unit and a speech recognition result received from the recognition result transmission unit .

  The portable terminal includes a dictionary acquisition unit that acquires dictionary information for speech recognition corresponding to the current business content from the server, and the terminal-side speech recognition unit performs speech recognition processing based on the acquired dictionary information. It is good also as performing.

  Further, the mobile terminal includes a support information acquisition unit that acquires support information corresponding to the current business content from the server, and the terminal-side work support unit sends a message to the user based on the acquired support information. It may be generated. In this case, the support information acquisition unit may acquire the support information from the server when it is predicted that the mobile terminal will be unable to communicate with the server.

The portable terminal further includes an audio processing unit that extracts feature data from the input audio signal by predetermined audio processing,
The voice transmission unit may transmit the extracted feature data as the voice information to the server.

  The server may further include an authentication unit that stores a voice print of each worker and authenticates the worker based on collation between the voice print of the received voice information and the stored voice print.

  The portable terminal includes an image acquisition unit, and the audio transmission unit receives the image data together with the audio information when the image data is acquired by the image acquisition unit corresponding to the corresponding audio. It is good also as transmitting to.

  In addition, the server includes an image acquisition unit, and the message transmission unit, when an image is acquired by the image acquisition unit corresponding to the generated message, includes the image data together with the audio information corresponding to the message. May be transmitted to the portable terminal.

  According to the present invention, both the portable terminal carried by the user and the server have a voice recognition function, and even if the communication between the portable terminal and the server cannot be performed, the voice to the user can be obtained. It is possible to provide interactive business support.

  FIG. 1 is an overall configuration diagram of a business support system according to an embodiment of the present invention. As shown in the figure, the system according to the present embodiment includes a work portable terminal 10 and a business support server 20.

  The business support server 20 is, for example, a server computer installed in a center for performing business monitoring, and is connected to the LAN 40. The business support server 20 is provided with a scanner 22, and various image data can be input to the business support server 20.

  On the other hand, the portable work terminal 10 is carried by a worker who performs various inspection work and the like, and the worker inputs and outputs voices by a headset 12 including a microphone and headphones (or earphones and speakers). And can communicate with the business support server 20 via the mobile phone 14 via the public network 30 (for example, the Internet) and the LAN 40. In addition, the work portable terminal 10 is connected to a digital camera 16 so that photographic image data can be input, and the current position can also be detected by a GPS function. Note that the work portable terminal 10 can be connected to the LAN 40 when not in use, and can directly communicate with the business support server 20 via the LAN 40.

  FIG. 2 is a block diagram illustrating functional configurations of the work mobile terminal 10 and the business support server 20.

  First, the work portable terminal 10 will be described. As illustrated in FIG. 2, the work mobile terminal 10 includes a voice analysis unit 102, a terminal-side voice recognition unit 104, a dictionary acquisition unit 105, a recognition result storage unit 106, a process switching unit 108, a communication determination unit 110, and a recognition result transmission. Unit 112, support information acquisition unit 114, terminal side support execution unit 116, audio output unit 118, image buffer 119, and the like.

  The voice analysis unit 102 analyzes the voice signal input from the microphone of the headset 12 and extracts feature amount data. This feature amount data is data indicating features of audio information. In the present embodiment, as one example, feature amount data is extracted using a so-called cepstrum. Calculation of feature data using a cepstrum is a well-known technique in the field of speech recognition, but here, the outline of this technique will be briefly described.

In the present embodiment, the audio signal is input at a sampling frequency of 16 kHz, for example. Therefore, the audio is expressed by 16,000 numerical data per second, and the numerical data is divided every 32 ms. . That is, continuous 512 numerical data is defined as one unit of division, and one divided unit is represented by s (t) (t = 0, 1,..., 511). Using this s (t), a 16-dimensional cepstrum C (u) (u = 0, 1,..., 15) is calculated by the following equation.

  Here, the function F is a 512-point discrete Fourier transform, and the domain of definition and the range of values are each composed of 512 points. The logarithmic function log calculates the logarithm for each of the 512-point domain. The 16-dimensional cepstrum thus determined is obtained by compressing 512 voice data into 16 data, and it is known that voice recognition can be performed using these 16 data.

  The above is an example, and it is possible to use other than cepstrum as the feature quantity used for speech recognition. In addition, when 512 pieces of numerical data c (t) are compressed into 16 pieces of numerical data C (u), they may not be divided equally as described above, but may be divided unevenly. Furthermore, operations such as noise removal may be performed on s (t).

  The calculation load required for the feature amount extraction processing described above is small, and the feature amount extraction processing is performed in real time by software processing even if the portable work terminal 10 is composed of a PDA or a notebook computer with relatively low processing capability. be able to. Alternatively, a dedicated IC chip that performs the feature amount extraction processing may be incorporated into the work portable terminal 10 and processed in hardware.

  The feature amount data extracted by the voice analysis unit 102 as described above is supplied to the terminal-side voice recognition unit 104 according to the setting by the processing switching unit 108 or from the mobile phone 14 via the network to the business support server. 20 is used for voice recognition processing. That is, in the present embodiment, the voice recognition process is performed by either the work portable terminal 10 or the business support server 20, and the process switching unit 108 sets which voice recognition process is performed.

  The communication determination unit 110 determines whether or not the mobile phone 14 is in a state where network connection is possible (that is, whether or not the work mobile terminal 10 and the business support server 20 can communicate). If the process switching unit 108 can communicate with the business support server 20 based on the determination result by the communication determination unit 110, the process switching unit 108 uses the feature data extracted by the voice analysis unit 102 from the mobile phone 120 to the business support server 20. If it is in a state where communication with the business support server 20 is impossible, the feature amount data is supplied to the terminal-side voice recognition unit 104 to carry out voice recognition on the business support server 20 side. Voice recognition processing is performed on the terminal 10 side.

  The terminal-side voice recognition unit 104 includes a dictionary storage unit 104a that stores a minimum necessary dictionary specialized for the current business contents of the work portable terminal 10, and the dictionary stored in the dictionary storage unit 104a is stored in the dictionary storage unit 104a. The speech recognition process is performed based on the feature amount data with reference to the above. The dictionary information is stored in the dictionary storage unit 104a, for example, by the dictionary acquisition unit 105 by the dictionary acquisition unit 105 from the work support server 20 before starting the inspection work using the work portable terminal 10. It shall be done by downloading. Alternatively, when the vehicle is moving by car or the like, the dictionary acquisition unit 105 may automatically download the dictionary information when it is detected by GPS that the vehicle is in an incommunicable area. Thus, since only the dictionary specialized for the use of the work portable terminal 10 is stored, the work portable terminal 10 can have a small storage capacity for storing the dictionary.

  The recognition result storage unit 106 stores the speech recognition result by the terminal side speech recognition unit 104 while the communication determination unit 110 determines that communication with the business support server 20 is not possible. Then, when the communication determination unit 110 determines that the recognition result transmission unit 112 has returned from the incommunicable state to the communicable state, the recognition result transmission unit 112 receives the voice recognition result stored in the recognition result storage unit 106 from the mobile phone 120 as a business support. Send to server 20.

  Image data of a photograph taken by the digital camera 16 is stored in the image buffer 119. When the feature amount data or the voice recognition result is transmitted to the business support server 20, if the corresponding image data is stored in the image buffer 119, the image data is also transmitted together.

  The support information acquisition unit 114 downloads and stores support information corresponding to the current job of the work mobile terminal 10 from the job support server 20. Here, the support information is information for giving appropriate instructions and advice in accordance with the work status of the worker. As will be described later, the business support server 20 stores support information corresponding to various business operations in a database, and responds to the corresponding business in response to a request from the support information acquisition unit 114 of the work mobile terminal 10. The corresponding support information is downloaded to the work portable terminal 10. Note that the download of the support information by the support information acquisition unit 114 can be performed manually by an inspection operator before starting the inspection operation, for example. Alternatively, as in the case of the dictionary acquisition unit 105, when the vehicle is moving in a car or the like, the support information acquisition unit 114 automatically obtains the support information when it is detected that the GPS has reached an incommunicable area. You may make it download.

  The terminal-side support execution unit 116 refers to the support information acquired by the support information acquisition unit 114 based on the voice recognition result by the terminal-side voice recognition unit 104, and the voice information of the message for supporting the work of the inspection worker Is generated.

  The voice output unit 118 outputs voice information generated by the terminal-side support execution unit 116 and voice information transmitted from the business support server 20 as will be described later.

Next, the business support server 20 will be described.
As shown in FIG. 2, the business support server 20 includes a server-side speech recognition unit 200, a server-side support execution unit 210, a speech synthesis unit 220, a communication unit 230, an authentication unit 240, a support information transmission unit 242, and a recognition result reception unit. 244, a support information database 250, a recognition result database 260, and the like.

  The communication unit 230 controls communication via the network, and transmits and receives various types of information to and from the mobile phone 14 on the work mobile terminal 10 side. Further, the authentication unit 240 stores voiceprint data of each inspection worker, and performs personal authentication on the work portable terminal 10 that has been accessed by using the voiceprint of the inspection worker.

  The server-side speech recognition unit 200 includes a recognition decoder 202, an acoustic model storage unit 204, a dictionary storage unit 206, and a language model storage unit 208. The acoustic model storage unit 204 stores phonetic unit acoustic information such as “a”, “i”, and “u”, and the dictionary storage unit 206 stores word unit dictionary information such as “electricity” and “switch”. Yes. Further, the language model storage unit 208 stores statistical information related to the connection relationship between words, for example, “thermal power” and “power generation” are connected in this order and are not connected in the reverse order. ing. The recognition decoder 202 performs speech recognition processing with reference to the acoustic model storage unit 204, the dictionary storage unit 206, and the language model storage unit 208 based on the feature amount data received by the communication unit 230, and performs the speech recognition. The result is supplied to the server side support execution unit 210. Note that the recognition decoder 200 may be implemented by software, or may be implemented as hardware by a dedicated IC chip or the like.

  As will be described in detail in the following embodiments, the server-side support execution unit 210 stores the support information database 250 based on the speech recognition result by the server-side speech recognition unit 200 (that is, the recognition result of the utterance speech of the inspection worker). The stored various business support information is searched to generate instructions, advice, etc. (hereinafter referred to as instruction information) to be given to the inspection operator. The generated instruction information is converted into a voice signal by the voice synthesizer 220 and transmitted from the communication unit 230 to the work portable terminal 10 via the network. The support information database 250 can also store drawing data such as a drawing of a facility to be inspected or a map read by the scanner 22.

  As described above, in the system of the present embodiment, bidirectional information transmission by voice can be performed between the work portable terminal 10 and the business support server 20. Note that the inspection result transmitted from the inspection operator by voice and recognized by the server-side voice recognition unit 200 is stored in the recognition result database 260. Further, the speech recognition result transmitted from the recognition result transmitting unit 112 of the work portable terminal 10 is received by the recognition result receiving unit 244, and is also stored in the recognition result database 260. The server side support execution unit 210 also has a function of creating various reports based on the information stored in the recognition result database 260 in this way.

  Further, the support information download unit 242 reads support information corresponding to the corresponding job from the support information database 250 in response to a request from the work mobile terminal 10 and downloads the support information to the work mobile terminal 10 via the communication unit 230. To do.

  As described above, according to the system of this embodiment, when the portable work terminal 10 can communicate with the business support server 20, the inspection operator performs a voice conversation with the business support server 20. Thus, work such as inspection can be performed while receiving instructions and advice from the work support server 20. For this reason, for example, it is possible to carry out the work that has conventionally been required by two or more people to ensure the accuracy of inspection, such as a substation of a power company, while ensuring accuracy. At the same time, since the work support server 20 can perform unattended work support, it is possible to save labor for work such as equipment inspection.

  In addition, for example, the portable work terminal 10 is used for work in an environment where the radio wave condition is poor, such as a power transmission tower operation in a mountainous area, a work in a concrete building, or a high-speed movement in a vehicle such as an automobile. When communication with the support server 20 cannot be performed, the process switching unit 108 switches to voice recognition processing in the work mobile terminal 10, so that the work mobile terminal 10 alone can be used for the inspection worker. Can give instructions and advice.

  Thus, in this embodiment, when the work portable terminal 10 can communicate with the work support server 20, the inspection operator can receive work support from the work support server 20, and work can be performed even when communication is impossible. The minimum necessary business support can be obtained by the processing of the mobile terminal 10 alone. That is, in the present embodiment, the portable work terminal 10 and the work support server are configured by combining the voice recognition process in the work support server 20 and the voice recognition process in the work mobile terminal 10 in a hybrid configuration. The work support for the inspection worker can be performed regardless of the state of communication with the terminal 20 (that is, regardless of the environment of the work place).

  Also, only necessary minimum information corresponding to the current work is downloaded to the work mobile terminal 10 side as dictionary information for performing speech recognition on the work mobile terminal 10 and support information for performing work support. However, the work portable terminal 10 side performs only the minimum voice recognition processing and business support processing with a narrowed application. For this reason, it is possible to reduce the storage capacity and processing capacity required for the work portable terminal 10, and the work portable terminal 10 can be reduced in size and weight to improve portability, and the price of the terminal can be reduced. .

  In addition, since the operator can input the inspection result and the like by voice from the headset 12, the inspection operation can be performed with both hands free to use. For this reason, a required work can be performed without difficulty even in a narrow place.

  Also, when a person provides work support as before, if the person in charge of support does not have sufficient knowledge in response to an inquiry from a worker, ask another person in charge or examine the materials. However, according to the present embodiment, since the business support server 20 can immediately generate instructions and advice by referring to the support information database 250, the business support Can be done smoothly.

  In the above-described embodiment, when the work mobile terminal 10 and the work support server 20 can communicate with each other, the feature data extracted from the audio signal by the work mobile terminal 10 is transmitted to the work support server 20. Therefore, compared with the case where the audio signal itself is transmitted, the amount of data is much smaller, the time required for information transmission is shortened, and the communication cost is reduced. In addition, when audio signals are sent as they are, some kind of compression processing is required, which may reduce the recognition accuracy. However, there is no need for data compression when sending feature data, so the recognition accuracy is reduced due to compression. No problem arises. However, the present invention includes not only the case where the feature amount data is extracted from the audio signal on the mobile terminal side and transmitted to the server, but also the case where the audio signal is digitized and transmitted to the server as it is.

  Hereinafter, more specific application examples of the work support system of the present embodiment will be described as Example 1 and Example 2.

<Example 1: Inspection work of substation>
It is an Example in the case of supporting the substation patrol work which patrols and inspects various facilities provided in the substation. In this embodiment, in the support information database 250, as support information corresponding to the substation patrol work, data such as a check list indicating inspection items for each device of the substation, an abnormal situation that may occur, and the abnormal situation Are stored, such as emergency response information representing countermeasures against the danger, dangerous work that may be performed by the worker, and dangerous work information representing warning contents to be issued in response to the dangerous work.

Further, the dictionary storage unit 206 and the language model storage unit 208 store, for example, the following dictionary and language model corresponding to the substation patrol work.
(Dictionary example)
SF6 (Esue Fukusukusu), number of operations (Beautiful), bushing (Bushing), oil pump (Aburaponpu), packing (Dakkin), dashpot (Dasyupotto), rust (Hasesei)
(Language model example)
SF6 → Gas → No → Leakage → No Pressure gauge → No → Instruction → Ha → Good Appearance → Inspection → Abnormal → No → Auxiliary → Switch → → → Abnormal → No →

  FIG. 3 is a flowchart showing an overview of the inspection work. First, an inspection worker (person in charge of inspection) specifies a substation to be inspected by connecting the work portable terminal 10 to the LAN 40 prior to inspection, and provides inspection route data and substation inspection work for the substation. Dictionary information and support information for voice recognition corresponding to the above are downloaded from the business support server 20 to the work portable terminal 10 (S100). However, the download may be performed via the mobile phone 14. As described above, the dictionary information and the support information may be downloaded when it is predicted that communication with the business support server 20 will be interrupted in the work portable terminal 10.

  After downloading the necessary information in this way, the patrolist attaches the headset 12 and the mobile phone 14 to the work portable terminal 10 and starts patrol.

  The patrol route data downloaded as described above is displayed on the screen of the work portable terminal 10, and the patrol person patrols the substation based on the patrol route (S102). The patrol person may be guided by voice based on the patrol route data. If the portable work terminal 10 and the work support server 20 can communicate with each other during the patrol, voice communication is performed between the patrol person and the work support server 20 as needed (S104). In the present embodiment, the work portable terminal 10 and the business support server 20 are described as being in a communicable state. However, when the two are incapable of communication, the inside of the work portable terminal 10 is described. When the business support process is performed, and the speech recognition result is returned to a communicable state, the business support server 20 is collectively transmitted.

  First, before the inspection of each facility starts, the patrol person inputs the facility name by voice, and the business support server 20 refers to the support information database 250 and reads the check list of the corresponding facility. Then, according to the check list, the instruction information is transmitted to the work portable terminal 10 as voice information. The patrol person performs an inspection according to the instruction information and inputs the inspection result by voice. Further, when the business support server 20 refers to the dangerous work information stored in the support information database 250 and senses some dangerous situation (for example, a situation where there is a risk of electric shock) from the voice of the patrol person, A warning sound such as “Is the ground installed” or “Is the power supply locked” is transmitted to the portable work terminal 10. Thereby, the patrol person can safely perform the patrol work.

  In addition, when the patrol person encounters any abnormality during the facility patrol process (S106), the patrol person notifies the work support server 20 by voice of the content of the abnormality. The business support server 20 refers to the response information in the support information database 250, searches for appropriate instruction information for the contacted abnormality, and transmits the advice to the work portable terminal 10 as voice information. (S108). For example, when a patrolman makes an inquiry asking "What should I do if the motor is overheated?", The business support server 20 searches for advice "Please do the processing ..." for this inquiry and use it for work. It transmits to the portable terminal 10. If it is difficult to register all trouble advices in the support information database 250, the work support server 20 is connected to an external expert system or the like, and no corresponding instruction information is found in the support information database 250. In such a case, an inquiry may be made to an external expert system or the like.

  In this embodiment, a system managed by the maintenance department is connected to the business support server 20, and the business support server 20 determines that an emergency measure is required for the abnormality notified from the patrol person. (S110) The contents of the first aid are transmitted to the work portable terminal 10 (S112), and the equipment requiring the first aid and the contents of the first action are transmitted to the maintenance department system by e-mail or the like (S114). At that time, a report in which dialogues with the patrol person until the trouble occurs may be created and transmitted to the maintenance department system.

  Information such as the inspection result transmitted from the work portable terminal 10 to the work support server 20 is stored in the recognition result database 260 (S116). If the inspection is not completed (S118), the process returns to S102 for the next inspection. The target equipment is inspected.

  Hereinafter, an example of a dialogue between the patrol person and the business support server 20 will be described by taking as an example a case of checking a circuit breaker in a substation. Before the inspection of the circuit breaker, the patrol person inputs the “circuit breaker” that is the inspection target device by voice input or operation input to the work portable terminal 10. Then, the business support server 20 refers to the check sheet (FIG. 4) stored in the support information database 250 corresponding to the circuit breaker, and performs the inspection work while performing a dialog with the patrol person as follows. Instruct and obtain inspection results.

Server> “Please check before inspection”
Patrol>"OK"
Server> “What is the number of operations before inspection?”
Patrol>"230times"
Server> “How many times does the oil pump operate?”
Patrol>"150times"
Server> “Check the appearance. Is there any damage on the exterior or loose bolts?”
Patrol>"None"
Server> “Are there any damage or cracks in the bushing?”
Patrol>"None"
Server> “Is the grounding terminal loose or the grounding wire damaged?”
Patrol> “There is a loose terminal and tighten.”
Server> “Is there any abnormality in the operation of the solenoid valve?”
Patrol>"None"
Server> “Perform an open / close test. Are you ready?”
Patrol> “Yoshi”
Server> “With circuit breaker (Iri)”
Patrol>"Circuit breaker, entering, ok"
Server> “Circuit breaker”
Patrol> “Circuit breaker, cut, good”
The inspection result transmitted from the patrol person through the above dialogue is accumulated in the recognition result database 260. The business support server 20 creates an inspection result report based on the inspection result data stored in the recognition result database 260. Thus, in this embodiment, the inspection record is automatically created, so that it is not necessary to manually create the inspection result report.

  In the course of the patrol work, the patrol person takes a field photograph with the digital camera 16 as necessary, and the photographed image data is the job support server together with the feature amount data or the voice recognition result as described above. 20 is transmitted. When the photographic image data is sent, the business support server 20 also stores the photographic image data in the recognition result database 260.

<Example 2: Electric consumption meter reading>
This is an embodiment in the case where a meter reading person of an electric power company supports a meter reading operation of visiting each household and measuring a monthly electricity usage amount. In this embodiment, in the support information database 250, as support information corresponding to the electricity usage meter reading work, each customer's name, address, meter number, past monthly usage, watt-hour meter indication value at the previous meter reading, Caution information (for example, drawings and photographs showing the installation location when the watt-hour meter is difficult to find, precautions such as a cautionary dog's attention), and other information (such as information on late payment of electricity charges) are stored. Further, the dictionary storage unit 206 and the language model storage unit 208 store, for example, the following dictionary and language model corresponding to the power usage meter reading work.

(Dictionary example)
1. Electric meter related meter (keiki), meter (meter), meter (meter key)
2. Contractual relationship Abolition (High), Total abolition (Renewal) (Zenpai Shin), Retirement (Zenshitsu), Reuse (Saiyo), Amount used (Ryoyo), Suspicious usage (Ryofushin), Electric light (Dentou), Electric power (Denryoku), metered electric light (Zuyori Dentou), outdoor light (Gaitou)
3. Meter-reading equipment Handy (handy), instructions (swords), number of instructions (swords)
(Language model example)
・ Instrument → Number → 358 → Number → Kanemura → Ms → → → This month → → → Number of instructions → 4495
・ Number → 358 → Instruction → 4495
・ 358 → Number → → → Kanamura → Number of instructions → 4495
・ Kanamura → San → Metered light → A → Instrument → Number → 358 →
→ Number of instructions → 4495
・ Instrument → Number → 359 → Hiraoka → san →
→ Day → Time → 3326 →
→ Family → Time → 2266 →
→ Night → Time → 1156

  FIG. 5 is a flowchart showing an outline of a watt-hour meter reading operation. In this embodiment, the description will be made assuming that the portable work terminal 10 and the business support server 20 are in a communicable state. As in the first embodiment, when the work mobile terminal 10 and the work support server 20 are in a state where communication is not possible, work support processing is performed inside the work mobile terminal 10, and the speech recognition result during the communication is communicated. When it returns to a possible state, it will be sent to the business support server 20 collectively.

  Before going to the meter reading work, the meter reader connects the work portable terminal 10 to the LAN 40, and provides the data on the visit route of the day and dictionary information and support information for voice recognition corresponding to the substation patrol work on the work support server. 20 is downloaded to the work portable terminal 10 (S200). This visit route is displayed on the screen of the work portable terminal 10, and the meter reader moves to each customer's house while referring to the displayed visit route, and reads the electricity meter (S202). At that time, the meter reader inputs the name of the customer to be metered from now on by voice or operation on the work portable terminal 10, and in response to this, the business support server 20 refers to the support information database 250, If there is corresponding customer's attention information, the content is transmitted to the portable work terminal 10 as voice information. As a result, the meter reader can easily find the position of the watt-hour meter even when it is difficult to understand, and can also perform the meter reading work smoothly, such as preparing for the dog.

  At each customer's home, the meter reader performs meter reading work while performing voice communication with the business support server 20 (S204). Specifically, first, the meter reader reads out the customer name and the meter number assigned to the watt-hour meter. The business support server 20 identifies the customer using the customer name and meter number read by the meter reader as keys, reads the instruction value of the customer at the previous meter reading from the support information database 250, and transmits it to the work portable terminal 10. . The meter reader reads out the watt-hour meter reading value, and the business support server 20 recognizes the value read out by the meter reader and transmits the recognized value to the work portable terminal 10 as a digital value and voice information. The work portable terminal 10 reproduces the voice information from the business support server 20, and the meter reader can thereby confirm that the instruction value of the watt hour meter has been correctly input. Also, the meter reader may find it difficult to find a watt-hour meter at the time of meter reading, or encounter a dog, but if the business support server 20 does not provide appropriate caution information, for example, “The meter is in the warehouse. Voice information such as “inside” or “careful about dogs” is input. Further, the situation of the site may be photographed with the digital camera 16. The caution information input by voice by the meter reader and the photographic image data taken by the digital camera 16 are transmitted to the business support server 20 and recorded in the support information database 250 as the corresponding customer caution information. The precautions recorded in this way are notified to the meter reader at the next and subsequent meter readings.

  Next, the business support server 20 calculates the usage amount for this month from the current instruction value read by the meter reader and the previous month's instruction value, and based on this value, there is no abnormality or suspicious point in the current meter reading value. Whether or not (S206). Specifically, the usage amounts of the current month and the previous month are compared, and if the amount of change is equal to or greater than a predetermined value, it is determined that there is a possibility of leakage or some other abnormality. Further, the usage amount of this month is compared with the history of the past usage amount, and it is determined whether there is any suspicious point such as a reading error or a watt hour meter failure in the usage amount of this month. If the meter reading value is abnormal or suspicious, a warning message is transmitted from the business support server 20 to the work portable terminal 10 (S208). As described above, when there is an abnormality or a suspicious point in the current meter reading value, by transmitting the fact to the mobile terminal 10 for work, the meter reading person does not return to the sales office, and an abnormal or suspicious point is detected at the time of meter reading. Knowing the existence, on the spot, it is possible to take necessary measures for abnormality such as investigation of the presence or absence of electric leakage, reconfirmation of the indicated value, inspection of the watt hour meter (S210).

  The work portable terminal 10 calculates the power usage amount and the electricity charge corresponding to the current month from the previous instruction value and the current instruction value transmitted from the work support server 20 as described above, and the work portable terminal 10 A notice of electricity usage as shown in FIG. 6 is created by the provided printer. However, if there is delinquency by the previous month due to the customer's delinquency information, the calculation of electricity charges and the creation of a notification form will not be performed. The meter reader inserts the information slip into the post box of the customer's house, the gap between the front doors, and the like.

  The information such as the meter reading value transmitted from the work portable terminal 10 to the business support server 20 and the information such as the amount used this month obtained from the meter reading value are stored in the recognition result database 260 (S212), and all customer homes are stored. If the meter reading has not been completed (S214), the process returns to S202 and the meter reading operation is performed at the next customer's house.

  Hereinafter, an example of a dialogue between the meter reader and the business support server 20 is shown. In addition, there are various menus for electricity contracts, but pay-per-use lighting contracts (the charge per unit usage is constant regardless of the season and time zone) and family time (per unit usage amount depending on the season and time zone) An example in the case of a change in the price of

・ Contract pattern A (in case of metered-light contract)
Meter reader> “Instrument number, 358”
"Kanamura-sama"
"Number of instructions this month 4495"
Server> “Instrument number, 358, Mr. Kanamura, this month is 4495.”
Meter reader> (Confirmation operation)
<The work portable terminal outputs a meter reading notice>
・ Contract pattern B (Family time)
Meter reader>"Mr.Hiraoka"
"Instrument number 359"
"Number of instructions this month"
"Daytime 3326"
"Family Time 2266"
"Nighttime 1156"
Server> “Mr. Hiraoka Instrument Number 359 Number of Instructions This Month Daytime 3326 Family Time 2266 Nighttime 1156”
Meter reader> (Confirmation operation)
<The work portable terminal outputs a meter reading notice>

  Although two embodiments have been described above, the application target of the present invention is not limited to these. For example, when a sales representative makes a product explanation to each customer while visiting a customer's home, he / she carries the portable mobile terminal 10 for which the customer's visit route has been registered in advance and voice-inputs the customer name. Corresponding customer information is transmitted from the business support server 20 to the work mobile terminal 10, and when a product name is input by voice, the product description information is transmitted from the business support server 20 to the work mobile terminal 10. This makes it possible for sales representatives to conduct sales activities efficiently.

  In addition, by exchanging information by voice between the work portable terminal 10 and the business support server 20, various business support such as destination guidance on a map, ticket reservation, and traffic congestion information can be performed. .

1 is an overall configuration diagram of a business support system according to an embodiment of the present invention. It is a block diagram showing the function structure of the portable terminal for work and a work support server. It is a flowchart which shows the outline | summary of patrol work. It is a figure which shows an example of the check sheet memorize | stored in the assistance information database. It is a flowchart which shows the outline | summary of a watt-hour meter meter-reading work. It is a figure which shows an example of the notice sheet of an electric usage-amount.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Mobile terminal for work 12 Headset 14 Mobile phone 16 Digital camera 20 Business support server 30 Public network 40 LAN
DESCRIPTION OF SYMBOLS 102 Speech analysis part 104 Terminal side speech recognition part 106 Recognition result memory | storage part 108 Process switching part 110 Communication determination part 112 Recognition result transmission part 114 Support information acquisition part 116 Terminal side support execution part 118 Voice output part 119 Image buffer 120 Mobile phone 200 Server side speech recognition unit 202 Recognition decoder 204 Acoustic model storage unit 206 Dictionary storage unit 208 Language model storage unit 210 Server side support execution unit 220 Speech synthesis unit 230 Communication unit 240 Authentication unit 242 Support information transmission unit 244 Recognition result reception unit 250 Support Information database 260 Recognition result database

Claims (10)

A system for supporting the user's business, comprising a mobile terminal carried by the user and a server having communication means for communicating with the mobile terminal,
The portable terminal is
A communication determination unit that determines whether or not communication with the server is possible;
A microphone,
When the communication determination unit determines that communication with the server is not possible, the mobile terminal is set to perform voice recognition based on a voice signal input from the microphone , and the communication determination unit A process switching unit configured to set the voice recognition to be performed by the server when it is determined that the server can communicate with the server ;
A terminal-side voice recognition unit that performs voice recognition based on the voice signal when the process switching unit is set to perform voice recognition on the portable terminal;
Based on the voice recognition result by the terminal-side voice recognition unit, a terminal-side support execution unit that outputs a message for the user as a voice signal;
A voice transmission unit configured to transmit voice information corresponding to a voice signal input from the microphone to the server when the processing switching unit is set to perform voice recognition in the server;
A recognition result storage unit for storing a speech recognition result by the terminal side speech recognition unit in a state where communication with the server is not possible;
A recognition result transmission unit that transmits the stored voice recognition result to the server when it is determined by the communication determination unit that communication with the server is possible ;
The server
A voice information receiving unit that receives the voice information transmitted from the portable terminal;
A server-side voice recognition unit that performs voice recognition based on the received voice information;
A server-side support execution unit that generates a message to be notified to the user based on a voice recognition result by the server-side voice recognition unit;
A message transmission unit that transmits audio information corresponding to the generated message to the mobile terminal;
A business support system comprising: a storage unit that stores a speech recognition result by the server-side speech recognition unit and a speech recognition result received from the recognition result transmission unit . The mobile terminal includes a dictionary acquisition unit that acquires dictionary information for speech recognition corresponding to the current business content from the server,
Business support system according to claim 1, wherein the performing speech recognition processing on the basis of the terminal-side speech recognition unit the acquired dictionary information. The mobile terminal includes a support information acquisition unit that acquires support information corresponding to the current business content from the server,
Business support system according to claim 1, wherein generating a message to the user based on the support information the terminal-side work support portion which the acquired. The business support system according to claim 3, wherein the support information acquisition unit acquires the support information from the server when it is predicted that the mobile terminal cannot communicate with the server. . The portable terminal further includes an audio processing unit that extracts feature data from the input audio signal by predetermined audio processing,
The voice transmitting unit, business support system according to any one of claims 1-4, characterized by transmitting to the server the feature data said extracted as the audio information. 2. The server according to claim 1, further comprising: an authentication unit that stores a voice print of each worker and authenticates the worker based on collation between the voice print of the received voice information and the stored voice print. The business support system according to any one of 5 to 5 . The portable terminal includes an image acquisition unit, and the audio transmission unit transmits the image data together with the audio information to the server when image data is acquired by the image acquisition unit corresponding to the corresponding audio. The business support system according to any one of claims 1 to 6 , wherein: The server includes an image acquisition unit, and when an image is acquired by the image acquisition unit in response to the generated message, the server transmits the image data together with audio information corresponding to the message. business support system according to any one of claims 1-7, characterized by transmitting to the portable terminal. A method of supporting the user's business by a portable terminal carried by the user and a server having a communication means for communicating with the portable terminal,
The mobile terminal is
A communication determination step for determining whether or not communication with the server is possible;
When it is determined that communication with the server is not possible, the mobile terminal is set to perform voice recognition based on a voice signal input from the microphone , and is determined to be communicable with the server. A process switching step for setting the voice recognition to be performed by the server ,
If it is set to perform at the portable terminal voice recognition by the processing switching step, a row Cormorant terminal side audio recognition step speech recognition based on the sound signal input from the microphone,
Based on the voice recognition result in the terminal side voice recognition step, a terminal side execution support step for outputting a message for the user as a voice signal ;
A voice transmission step of transmitting voice information corresponding to a voice signal input from the microphone to the server when voice recognition is set in the server in the process switching step ;
A recognition result storage step for storing a result of speech recognition in the terminal side execution support step in a state where communication with the server is not possible;
A recognition result transmission step of transmitting the stored voice recognition result to the server when it is determined in the communication determination step that communication with the server is possible;
The server is
A voice information receiving step for receiving the voice information transmitted from the portable terminal ;
A row Cormorant server-side speech recognition step of the speech recognition on the basis of the received audio information,
A server-side support execution step for generating a message to be notified to the user based on the voice recognition result ;
A message transmission step of transmitting voice information corresponding to the generated message to the mobile terminal;
And storing a voice recognition result obtained by the server-side voice recognition step and receiving and storing the voice recognition result transmitted in the recognition result transmission step . A communication means that is carried by a user and is a business support terminal for supporting the business of the user, and communicates with a predetermined server;
A communication determination unit for determining whether or not the communication means is in a state where communication with the server is possible;
A microphone,
When the communication determination unit determines that communication with the server is not possible, the mobile terminal is set to perform voice recognition based on a voice signal input from the microphone , and the communication determination unit A process switching unit configured to set the voice recognition to be performed by the server when it is determined that the server can communicate with the server ;
A terminal-side voice recognition unit that performs voice recognition based on the voice signal when the process switching unit is set to perform voice recognition on the portable terminal;
A terminal-side support execution unit that generates voice information of a message for the user based on a voice recognition result by the voice recognition unit;
A voice transmission unit configured to transmit voice information corresponding to a voice signal input from the microphone to the server when the processing switching unit is set to perform voice recognition in the server;
A recognition result storage unit for storing a speech recognition result by the terminal side speech recognition unit in a state where communication with the server is not possible;
A recognition result transmission unit that transmits the stored voice recognition result to the server when the communication determination unit determines that communication with the server is possible ;
A voice output unit that outputs voice information generated by the terminal-side support execution unit and voice information transmitted from the server according to the voice information transmitted by the voice transmission unit; Business support terminal.

Images