JP7001241B2 - Packet monitoring device, private branch exchange, virtual assistant calling method, and program - Google Patents

Description

The present disclosure relates to packet monitoring devices, private branch exchanges, virtual assistant calling methods, and programs.

A private branch exchange installed on the premises of a government office, a company, a school, etc. accommodates a plurality of private telephones and enables a call between the private telephones and a call with an external telephone.

Patent Document 1 describes a system for supplying a virtual assistant assisting a user in voice communication between users. In the system described in Patent Document 1, a virtual assistant may participate in a call, and when called by a user on the call, a task of identifying a voice instruction from the call according to the user and corresponding to the voice instruction is performed. Running.

Special Table 2016-533690 Gazette

However, in the technique described in Patent Document 1, when calling a virtual assistant, the user who is the caller needs to intentionally give a predetermined voice instruction on the system side, and the technique for calling is performed on the user's terminal. You will need a module.

Therefore, even in a telephone that does not have or cannot have a module for calling a virtual assistant, a technique for calling and participating in the virtual assistant during a call is desired.

An object of the present disclosure is to provide a packet monitoring device, a private branch exchange, a virtual assistant calling method, and a program capable of solving the above-mentioned problems. The challenge is to make it possible for a virtual assistant to join a call at the right time during a call, even if the phone does not have the ability to have the virtual assistant join the call.

The packet monitoring device according to the first aspect of the present disclosure is a call between telephones including at least one of the premises telephones in a network in which a plurality of premises telephones and a premises exchange accommodating the plurality of premises telephones are connected. The control unit includes a monitoring unit and a control unit that monitor the communication packet data of the above, and the control unit executes voice recognition processing on the communication packet data monitored by the monitoring unit, and performs voice recognition processing in the voice recognition processing. When a predetermined keyword is included in the recognition result, a call process for calling the virtual assistant is executed, and a mixing process for mixing the voice of the virtual assistant into the call is executed.

The premises exchange according to the second aspect of the present disclosure is a communication of a call between a connection unit for connecting a plurality of premises telephones to be accommodated via a network and a telephone including at least one of the premises telephones in the network. A monitoring unit for monitoring packet data and a control unit are provided, and the control unit executes voice recognition processing on the communication packet data monitored by the monitoring unit, and the recognition result in the voice recognition processing. If a predetermined keyword is included in the phone, a call process for calling the virtual assistant is executed, and a mixing process for mixing the voice of the virtual assistant into the call is executed.

In the virtual assistant calling method according to the third aspect of the present disclosure, in a network in which a packet monitoring device is connected to a plurality of premises telephones and a premises exchange accommodating the plurality of premises telephones, at least one of the premises telephones is used. A monitoring step for monitoring communication packet data of a call between telephones including the telephone, a step for the packet monitoring device to execute voice recognition processing for the communication packet data monitored in the monitoring step, and the packet monitoring device. However, when a predetermined keyword is included in the recognition result in the voice recognition process, the step of executing the call process for calling the virtual assistant and the packet monitoring device mix the voice of the virtual assistant into the call. It is provided with a step of executing a mixing process.

The program according to the fourth aspect of the present disclosure is between telephones including at least one of the private telephones in a network in which a computer is connected to a plurality of private telephones and a private exchange accommodating the plurality of private telephones. A monitoring step for monitoring the communication packet data of a call, a step for executing a voice recognition process for the communication packet data monitored in the monitoring step, and a recognition result in the voice recognition process include a predetermined keyword. If so, it is a program for executing a step of executing a call process for calling the virtual assistant and a step of executing a mixing process for mixing the voice of the virtual assistant into the call.

According to the present disclosure, it is possible to provide a packet monitoring device, a private branch exchange, a virtual assistant calling method, and a program having the following effects. The effect is that even a phone that does not have the ability to have the virtual assistant join the call can have the virtual assistant join the call at the right time during the call.

It is a block diagram which shows one configuration example of the packet monitoring apparatus which concerns on Embodiment 1. FIG. It is a flow diagram for demonstrating an example of the virtual assistant calling method in the packet monitoring apparatus of FIG. It is a block diagram which shows one configuration example of the private branch exchange which concerns on Embodiment 2. FIG. 3 is a block diagram showing a configuration example of a telephone system including a packet monitoring device according to the third embodiment. FIG. 3 is a schematic diagram for schematically explaining an example of a telephone call via a packet monitoring device in the telephone system of FIG. 4. It is a flow diagram for demonstrating an example of the preprocessing which calls a virtual assistant executed in the packet monitoring apparatus of FIG. It is a block diagram which shows an example of the packet monitoring analysis part in the packet monitoring apparatus of FIG. It is a block diagram which shows an example of the packet monitoring analysis part in the packet monitoring apparatus of FIG. It is a block diagram for demonstrating an example of the virtual assistant call processing executed by the packet monitoring apparatus in the telephone system which concerns on Embodiment 4. FIG. It is a block diagram for demonstrating an example of the virtual assistant call processing and voice mixing processing executed by the packet monitoring apparatus in the telephone system which concerns on Embodiment 4. FIG. It is a figure which shows an example of the hardware composition of a device.

Hereinafter, embodiments will be described with reference to the drawings. In the embodiment, the same or equivalent elements may be designated by the same reference numerals, and duplicate description may be omitted. In addition, there are drawings that draw unidirectional and bidirectional arrows in the drawings described below, but both arrows simply show the direction of the flow of a certain signal (data), and each of them. It does not exclude bidirectionality and one-wayness.

<Embodiment 1>
First, the first embodiment will be described with reference to FIG. FIG. 1 is a block diagram showing a configuration example of the packet monitoring device according to the first embodiment.

As shown in FIG. 1, the packet monitoring device 1 according to the present embodiment includes a control unit 1a and a monitoring unit 1b, and although not shown, a plurality of private telephones and a private branch exchange accommodating those private telephones are connected to each other. Can be connected to a network.

Here, the network may be, for example, the Internet, an intranet, or the like. Further, the premises can refer to, for example, the inside of an office such as a government office, a company, or a school, or the inside of a business establishment. The private branch exchange relays a call by packet communication, and can be referred to as an electronic private branch exchange or the like. Examples of the private branch exchange include IP (Internet Protocol) -PBX (Private Branch eXchanger). Examples of the private branch exchange include a button telephone device equipped with an external line connection unit such as VoIP (Voice over Internet Protocol) and an extension connection unit such as an IP telephone.

Any of the private telephones and the external telephones that can be the destinations of calls using the private telephones may be those capable of making a call by packet communication, for example, fixed IP telephones and mobile phones (including those called smartphones). ) And so on. Both the internal telephone and the external telephone may be a terminal device (terminal) such as a stationary PC (Personal Computer) or a mobile PC as long as they have a call function by packet communication.

The monitoring unit 1b monitors the communication packet data of a call between a plurality of telephones including at least one private telephone in the network. That is, the telephones are connected to each other via the packet monitoring device 1 during a call, and the call is monitored by the packet monitoring device 1. Therefore, a communication unit such as a communication interface for connecting to the network and transmitting / receiving communication packet data may be provided in the monitoring unit 1b or another part of the packet monitoring device 1.

As described above, the monitoring unit 1b is configured to monitor the communication packet data of a call (extension call) between two or more private telephones, and is between one or more private telephones and one or more external telephones. It can be configured to monitor the communication packet data of a telephone call (outside line call). Of course, the monitoring unit 1b can adopt both of these configurations. Calls at three or more points may be referred to as multipoint meetings.

The control unit 1a can control the monitoring unit 1b, but can be configured to control the entire packet monitoring device 1 including other parts (not shown). The control unit 1a can be realized by, for example, a CPU (Central Processing Unit), a working memory, a non-volatile storage device that stores a program for performing control as described above, and the like. Further, the control unit 1a can also be realized by, for example, an IC (Integrated Circuit). The control unit 1a can also be referred to as a controller.

Then, the control unit 1a executes voice recognition processing for the communication packet data monitored by the monitoring unit 1b. Next, the control unit 1a executes a call process for calling the virtual assistant when a predetermined keyword is included in the recognition result (for example, the result of converting the voice into text) in the voice recognition process. The above-mentioned predetermined keywords can be registered in advance.

Here, the packet monitoring device 1 can be connected to a server device (not shown) that provides a virtual assistant via the above network. This server device is a device that provides a virtual assistant, and provides a virtual assistant in response to a participation request from the packet monitoring device 1 via the above network (and other external network). As described above, the call process can be a process of causing the server device to provide the virtual assistant via the network. In this case, the packet monitoring device 1, the private branch exchange, the plurality of private telephones, and the server device can form a telephone system.

Further, the virtual assistant can be stored in a storage unit (referred to as a virtual assistant storage unit) inside the packet monitoring device 1 so that it can be read from the control unit 1a. The telephone system in this case does not require the above server device, and can be configured to include a packet monitoring device 1, a private branch exchange, and a plurality of private telephones.

On the other hand, if the recognition result does not include the above-mentioned predetermined keyword, the control unit 1a prevents such a calling process. Of course, the control unit 1a can be configured to execute this call processing even when a keyword similar to the above-mentioned predetermined keyword is included, which means that a similar search is performed when searching for a recognition result. It can also be realized by applying.

Then, the control unit 1a executes a mixing process for mixing the voice of the virtual assistant into the above-mentioned call. The mixing processing method does not matter. For example, the control unit 1a can change the communication packet data for the call so as to mix the voice of the virtual assistant with the call.

As described above, the packet monitoring device 1 monitors the communication packet data, recognizes the voice input by the user, calls the virtual assistant when the predetermined keyword is detected, and mixes the voice into the call between the telephones.

Further, the control unit 1a can also execute the voice recognition process on the communication packet data monitored by the monitoring unit 1b even after the mixing process. In that case, the control unit 1a can be configured to execute a stop process for stopping the provision of the virtual assistant when the recognition result in the voice recognition process includes another predetermined keyword.

This cancellation process may be performed for the virtual assistant provider (call destination). For example, when the server device is a provider, this cancel process can refer to a process of instructing the server device to leave (erase) the virtual assistant via the network. For example, when the provider is a virtual assistant storage unit, this cancellation process can be a process of canceling the reading of the virtual assistant (the program) from the virtual assistant storage unit. By such a stop process, the mixing of the voice of the virtual assistant is stopped.

An example of such a virtual assistant calling method in the packet monitoring device 1 will be described with reference to FIG. FIG. 2 is a flow chart for explaining an example of a virtual assistant calling method in the packet monitoring device of FIG. This virtual assistant calling method can include a monitoring step, a recognition step, a calling step, and a mixing step as follows.

First, as a monitoring step, the packet monitoring device 1 monitors the communication packet data of a call between telephones including at least one private telephone in the network (step S1). As a recognition step, the packet monitoring device 1 executes voice recognition processing for the communication packet data monitored in the monitoring step (step S2).

As a call step, the packet monitoring device 1 determines whether or not a predetermined keyword is included in the recognition result in the voice recognition process (step S3), and if it is included (YES), the call process is performed. Execute (step S4). This calling process can be a process in which the packet monitoring device 1 calls a virtual assistant from the internal virtual assistant storage unit, or a process in which the server device provides the virtual assistant via the network.

On the other hand, if NO in step S3, the process returns to step S1 and monitoring is continued. The voice recognition result in step S2 can be held, for example, for a certain period of time, or can be held until it can be determined that the keyword does not match a predetermined keyword.

Following step S4, as a mixing step, the packet monitoring device 1 executes a mixing process of mixing the voice of the virtual assistant into a call after the call process (step S5), and ends the process.

The process of FIG. 2 can be repeated even after the mixing process, and for example, another virtual assistant can be called as described later in another embodiment. Further, after the process of FIG. 2, it is also possible to perform a process of erasing the called virtual assistant, and in that case as well, it can be executed when another predetermined keyword is included in the voice recognition result. ..

As described above, in the packet monitoring device 1 according to the present embodiment, a call between users (a user whose at least one uses a private telephone) is monitored on the network, and a virtual assistant is called during the call. .. Therefore, the user can use the virtual assistant during the call. Here, in the present embodiment, not only the virtual assistant can be joined during a call between two parties, but also the virtual assistant can be joined during a call (conference call) between three or more parties.

Further, according to the packet monitoring device 1 according to the present embodiment, the virtual assistant can be made to participate in the call even if the telephone does not have the function for making the virtual assistant participate in the call.

For example, if the private telephone is not a telephone that can additionally install application software (hereinafter, application) such as a smartphone, the client module cannot be additionally installed. The client module here refers to a client application for calling a virtual assistant from a server device. In particular, many multifunctional telephones housed in telephone systems generally used in enterprises do not have such a client module. However, in the present embodiment, even a telephone such as a simple IP telephone (for example, a telephone having a processing capacity capable of making a two-way call), instead of a telephone having such a special function, is in a call. Virtual assistants can participate.

Therefore, when using the virtual assistant, it is not necessary to introduce a high-performance telephone or smart speaker for the user, and the existing environment can be used. Further, even if the telephone can be additionally equipped with an application such as a smartphone, it is not necessary to additionally install a client application (client module) for calling the virtual assistant. That is, in the present embodiment, by introducing the packet monitoring device 1, the packet monitoring device 1 calls the virtual assistant and mixes the voices. Therefore, even if the processing capacity and functions of the private telephone are low, the call is virtual. You can have an assistant join you.

Further, in the present embodiment, when the virtual assistant participates in the user's call during the call, a plurality of sessions are not set up, and there is no need to mix them. For example, in the present embodiment, it is not necessary to establish a session between the source user and the destination user and a session between the source user and the virtual assistant on the calling telephone, and it is not necessary to mix a plurality of sessions. Note that this session can refer to, for example, a SIP (Session Initiation Protocol) session.

As described above, according to the present embodiment, not only is it not necessary to implement the virtual assistant client module in the telephone, it is not necessary to set up a plurality of sessions, and it is not necessary to mix those sessions.

From this point as well, according to the present embodiment, it can be said that the virtual assistant can participate during a call even if the telephone is not a telephone having a special function but a telephone such as a simple IP telephone, and the existing environment can be used. It can be said that it can be used. In particular, multifunction telephones housed in telephone systems commonly used in enterprises often have a limit on the number of sessions that can be used at the same time, and in some cases, multiple sessions may be mixed and used. It is assumed that it cannot be done. However, in the present embodiment, one session is sufficient when the virtual assistant is involved, so even when a telephone whose session is limited to one is used, the voice of the virtual assistant is mixed and used. can do.

Further, although the network bandwidth is consumed only for the session, in the present embodiment, since one session is sufficient, it can be said that the load on the network bandwidth can be reduced to the minimum necessary.

Further, according to the packet monitoring device 1 according to the present embodiment, it is possible to have the virtual assistant participate in the call at an appropriate timing during the call. Specifically, in the present embodiment, it is possible to monitor the contents of a call between users and join a virtual assistant when a predetermined keyword is detected, so that only the scenes that the user needs are virtual. You can have the assistant join the user's call. Therefore, when calling the virtual assistant during a call, the user does not need to intentionally call the virtual assistant, and the user does not need to take an action intentionally.

Further, in the present embodiment, since the virtual assistant can be joined when the user needs it, it can be said that the load on the network bandwidth can be suppressed from this point as well. In addition, by adopting the above-mentioned cancellation process, the virtual assistant can be joined only in the situations where the user needs it, so that the load on the network bandwidth can be minimized.

<Embodiment 2>
The second embodiment will be described with reference to FIG. 3, focusing on the differences from the first embodiment, but various examples described in the first embodiment can be applied. FIG. 3 is a block diagram showing a configuration example of the private branch exchange according to the second embodiment.

As shown in FIG. 3, the private branch exchange 2 according to the present embodiment includes a control unit 2a, a monitoring unit 2b, and a connection unit 2c. The connection unit 2c connects a plurality of private telephones (not shown) to be accommodated via a network such as the Internet or an intranet. The connection unit 2c performs call control and terminal control (control of the premises telephone) via the above network, and enables a call between the premises telephone and the premises telephone (external line call) and a call between the premises telephones (extension call). Can have the function of That is, the private branch exchange 2 can have a function of connecting an outside line to an extension, a function of connecting a private telephone to an external line, and a function of connecting between private telephones, mainly by having a connection portion 2c.

The monitoring unit 2b monitors the communication packet data of the call between the telephones including at least one private telephone in the network. The monitoring unit 2b monitors the communication packet data of the call between the telephones connected by the connection unit 2c.

Examples of the private branch exchange 2 include an IP-PBX, a button telephone device, and the like as described in the first embodiment. Further, a system including a private branch exchange 2 according to the present embodiment, a plurality of private telephones, and a server device described later can be referred to as a telephone system. The telephone system according to the present embodiment is the telephone system according to the first embodiment, in which the packet monitoring device 1 is provided inside the private branch exchange. Therefore, the control unit 2a and the monitoring unit 2b have the functions of the control unit 1a and the monitoring unit 1b of FIG. 1, respectively, and the functions of the private telephone and the server device in the present embodiment are basically the same as those of the first embodiment. Will be.

For example, the control unit 2a executes voice recognition processing on the communication packet data monitored by the monitoring unit 2b. Then, when the recognition result in the voice recognition process includes a predetermined keyword, the control unit 2a executes a call process for calling the virtual assistant. After the call processing is performed, the control unit 2a will execute a mixing process for mixing the voice of the virtual assistant into the call.

The control unit 2a can control the monitoring unit 2b and the connection unit 2c, but can be configured to control the entire private branch exchange 2 including other parts (not shown). The control unit 2a can be realized by, for example, a CPU, a working memory, and a non-volatile storage device that stores a program for performing the above-mentioned control. Further, the control unit 2a can also be realized by, for example, an IC. The control unit 2a can also be referred to as a controller.

Further, the method of calling the virtual assistant in the private branch exchange 2 has been described with reference to FIG. 2, except that the main body thereof changes from the packet monitoring device 1 to the private branch exchange 2 (the part that functions as the packet monitoring device 1 in the private branch exchange 2). Similar to the one.

As is clear from the above description, the private branch exchange 2 according to the present embodiment has the same effect as the packet monitoring device 1 according to the first embodiment.

<Embodiment 3>
Although the third embodiment will be described mainly on the differences from the first embodiment with reference to FIGS. 4 to 10, various examples described in the first and second embodiments can be applied. When the packet monitoring device according to the present embodiment is applied to the second embodiment, the function of the packet monitoring device according to the present embodiment may be mounted on the private branch exchange 2 according to the second embodiment. FIG. 4 is a block diagram showing a configuration example of a telephone system including the packet monitoring device according to the third embodiment.

As shown in FIG. 4, in the telephone system according to the present embodiment, the packet monitoring device 10, the private branch exchange 20, the user terminals 30a and 30b which are examples of the private telephone, and the server device (hereinafter, simply server) 40 are connected to the network. It can be a connected configuration. The packet monitoring device 10, the private branch exchange 20, the user terminals 30a and 30b, and the server 40 are examples of the packet monitoring device 1, the private branch exchange, the private telephone, and the server device in the first embodiment, respectively.

The user terminals 30a and 30b are terminals such as IP telephones, and will be described as terminals used by users A and B, respectively. Of course, premises telephones can be arranged on the network other than the user terminals 30a and 30b, and even if there is only one premises telephone, it is limited to outside line calls, but this embodiment can be applied.

The above-mentioned network can be a LAN (Local Area Network), an intranet using a wireless LAN, or the like, and can be connected to a network outside the premises. In the following, an example in which the telephone system according to the present embodiment is incorporated in an internal network of a certain company will be described. In this case, the private branch exchange 20 can be referred to as an in-house exchange or the like.

In the telephone system according to the present embodiment, user terminals 30a and 30b such as IP telephones may be connected to the in-house network, and the extension telephone network and the LAN may be integrated. The user A and the user B can make a call via the in-house network by using the user terminal 30a and the user terminal 30b belonging to the subordinate of the private branch exchange 20.

The user terminal 30a of the user A and the user terminal 30b of the user B belong to the private branch exchange 20 and are connected to each other via the in-house network. The packet monitoring device 10 is connected to the company network. The server 40 can be recallably stored in a storage unit (not shown) so that the virtual assistant 41 can be provided by a control unit (not shown).

However, the virtual assistant 41 may be built in the packet monitoring device 10, and in that case, the telephone system shall be configured not to include the server 40 or to have the server 40 built in the packet monitoring device 10. Can be done. Further, the virtual assistant 41 may be a single unit or a plurality of virtual assistants 41. In the configuration in which a plurality of virtual assistants can be used, it can be configured to be callable from both the packet monitoring device 10 and the server 40.

The packet monitoring device 10 can include a control unit 11 and a communication unit 12. The communication unit 12 can have a communication interface for connecting to the in-house network. The control unit 11 can have a function of the control unit 1a of FIG. 1 and a part of the functions of the monitoring unit 1b. In this example, the packet monitoring analysis unit 13, the virtual assistant calling unit 14, and the voice mixing unit to be described later. It can have a portion 15.

The control unit 11 can control the communication unit 12, but can be configured to control the entire packet monitoring device 10 including other parts (not shown). The control unit 11 can be realized by, for example, a CPU, a working memory, and a non-volatile storage device that stores a program for performing the above-mentioned control. Further, the control unit 11 can also be realized by, for example, an IC. The control unit 11 can also be referred to as a controller.

Next, the packet monitoring analysis unit 13, the virtual assistant calling unit 14, and the voice mixing unit 15 will be described with reference to FIGS. 5 to 10.

The packet monitoring analysis unit 13 monitors and analyzes the communication packet data of the call between users. The packet monitoring analysis unit 13 can also be referred to as a packet data monitoring analysis unit. First, the target call will be described with reference to FIG. FIG. 5 is a schematic diagram for schematically explaining an example of a call via the packet monitoring device 10 in the telephone system of FIG.

Calls between users are made by packet communication, and at that time, a communication protocol such as RTP (Real-time Transport Protocol) can be applied. The above communication packet data can be RTP packet data. Further, in this packet communication, in order to exchange information with the other party in advance, a call control protocol for call control processing in a call via an IP network such as SIP can be adopted. Hereinafter, the packet transmitted / received by this packet communication is referred to as a SIP / RTP packet.

From the start of the call to the monitoring and analysis of the SIP / RTP packet, it can be executed as follows. As shown by the broken line arrow in FIG. 5, the user A and the user B make a call via the in-house network on the user terminal 30a and the user terminal 30b belonging to the private branch exchange 20. At this time, the call between the user terminal 30a and the user terminal 30b (packet communication during a call) is always carried out through the packet monitoring device 10 as shown by the solid line arrow in FIG. 5 for sending and receiving SIP / RTP packets. To. In the following, it is assumed that this call is always executed via the communication unit 12, and the communication packet received by the communication unit 12 is input to the packet monitoring analysis unit 13 of the control unit 11.

Next, the details of the packet monitoring analysis unit 13 will be described with reference to FIGS. 6 to 8. FIG. 6 is a flow chart for explaining an example of preprocessing for calling a virtual assistant. 7 and 8 are block diagrams showing an example of the packet monitoring analysis unit 13. FIG. 7 is a diagram showing a portion of the packet monitoring / analysis unit 13 focusing on monitoring and analysis processing, and FIG. 8 is a diagram showing a portion of the packet monitoring / analysis unit 13 focusing on pre-call processing for the virtual assistant calling unit 14.

As shown in FIG. 7, the packet monitoring analysis unit 13 can include a packet monitoring unit 51, a SIP header information storage unit 52, and a packet analysis unit 53, and monitors communication packet data between the user terminal 30a and the user terminal 30b. And analyze.

The packet monitoring unit 51 can include a SIP header information reading unit 54 and an RTP data reading unit 55. The SIP header information reading unit 54 reads the SIP header information (source / destination information) of users A and B from the communication packet data, records it in the SIP header information storage unit 52, and makes a call between users. Save it when you start. After starting the call, the RTP data reading unit 55 reads the RTP data (voice data) from the communication packet data, and flows the voice data into the packet analysis unit 53.

The packet analysis unit 53 can have a voice data text conversion unit 56, a data comparison unit 57, a command instruction unit 58, and a specific keyword (KW) registration database (DB) 59. Here, in the specific KW registration DB 59, one or a plurality of predetermined keywords (hereinafter, specific keywords) can be registered. The specific KW registration DB 59 may be stored in a storage device mounted on the packet monitoring device 10.

The voice data text conversion unit 56 can have a conversion engine that inputs voice data flowed from the RTP data reading unit 55, analyzes it in real time, and converts it into text. This conversion engine may have a function of converting voice data into text data in appropriate phrase units. In this way, the voice data text conversion unit 56 converts the voice data into text (step S11 in FIG. 6).

The data comparison unit 57 compares the converted text data with the specific keyword registered in the specific KW registration DB 59 (for example, "explanation is required") at any time for each phrase, and the comparison result is compared with the command instruction unit 58. (Step S12).

As shown in FIG. 8, the command instruction unit 58 can have a matching determination unit 58a and a virtual assistant call pre-processing unit 58b. As a result of the comparison in the data comparison unit 57, the matching determination unit 58a determines whether or not a word matching the specific keyword registered in the specific KW registration DB 59 exists in the text data (step S12). If there is a matching word (YES in step S12), the virtual assistant call preprocessing unit 58b executes the following preprocessing (step S13). On the other hand, if NO in step S12, the process can be returned to step S11 and the process can be repeated for the next and subsequent clauses until YES in step S12.

The pre-processing executed in step S13 is a process of sending a virtual assistant calling instruction to the virtual assistant calling unit 14. Specifically, the virtual assistant call pre-processing unit 58b (hereinafter, pre-processing unit 58b) transmits the call information to the virtual assistant calling unit 14.

This call information can include information indicating that it is necessary to call (matching) or an actual call instruction, but indicates what action (including remarks) is desired from the virtual assistant. Behavioral information can also be included. For example, the pre-processing unit 58b can transmit the match_true as the former information and the match_action as the latter information (behavioral information) to the virtual assistant calling unit 14. In order to obtain such behavior information, the registration data 59a and the behavior data 59b can be stored in association with each other in the specific KW registration DB 59 as shown in FIG. The registration data 59a can be data in which a specific keyword is registered, and the action data 59b can be data indicating the action of the virtual assistant called for the specific keyword. It is assumed that the specific keyword and the action correspond to each other on a one-to-one basis, a one-to-many basis, or a many-to-one basis.

As an example, when user A says "this case requires explanation of user C" during a call, the specific keyword "needs explanation" is associated with the behavior data of "calling someone". The case where it is registered in the specific KW registration DB 59 will be described. In this case, the pre-processing unit 58b provides the virtual assistant calling unit 14 with information (match_true) indicating matching and an action command (match_action) corresponding to the matching by matching the text data with “explanation is required”. Send. In this case, the preprocessing unit 58b can read the action data "call someone" associated with "need to explain" from the specific KW registration DB 59 as an action command.

In this way, the preprocessing unit 58b transmits to the virtual assistant calling unit 14 information indicating that the virtual assistant actually performs an action with respect to the matched specific keyword, together with information indicating that the match has been made. The virtual assistant calling unit 14 receives the calling information as described above and performs the virtual assistant calling process.

Next, an example of this virtual assistant call process will be described with reference to FIG. FIG. 9 is a block diagram for explaining an example of the virtual assistant call process executed by the packet monitoring device 10.

As shown in FIG. 9, the virtual assistant calling unit 14 has a virtual assistant calling processing unit 14a, and performs a process of calling a virtual assistant (based on the above calling information) from the analysis result by the packet analysis unit 53. The virtual assistant call processing unit 14a uses the information match_true indicating matching as a key (based on the presence or absence of the information) to determine whether or not to call the virtual assistant.

When it is decided to call, the virtual assistant call processing unit 14a transmits a request to the private branch exchange 20 to which the parameter of the action command (match_action) is added and to connect to the virtual assistant 41. As shown by the dashed arrow in FIG. 9, the private branch exchange 20 conveys this request to the server 40.

The server 40 receives a request from the private branch exchange 20 and starts a call session with the packet monitoring device 10. Specifically, the virtual assistant 41 of the server 40 reads the received match_action data, recognizes an instruction to be executed, and the server 40 or the virtual assistant 41 establishes a SIP session with the packet monitoring device 10. The virtual assistant 41 can have a function of reading the received match_action data and recognizing an instruction to be executed.

Next, an example of the voice mixing process will be described with reference to FIG. FIG. 10 is a block diagram for explaining an example of a virtual assistant calling process and a voice mixing process executed by the packet monitoring device 10.

As shown in FIG. 10, after establishing the SIP session with the server 40, the virtual assistant call processing unit 14a connects the RTP session with the server 40 to the voice mixing unit 15. As a result, the voice mixing unit 15 starts the voice mixing process of mixing the voice of the virtual assistant 41 with the communication packet data of the call between the user terminal 30a and the user terminal 30b. At this time, the voice mixing unit 15 mixes the voices of the user terminal 30a and the virtual assistant 41 and transmits them to the user terminal 30b, and mixes the voices of the user terminal 30b and the virtual assistant 41 and transmits the voices to the user terminal 30a. In this way, the packet monitoring device 10 including the voice mixing unit 15 establishes a call between the three parties.

After the start of the audio mixing process, the virtual assistant 41 implements a scenario based on the received match_action value. Here, after participating in the conversation between user A and user B, for example, "Do you want someone to participate in this conversation?" Is said. Since the subsequent processing is the processing of the virtual assistant 41 itself (the processing of the engine itself of the virtual assistant 41), the description thereof will be omitted. For example, when the user A or the user B says "Join user C", the virtual assistant 41 can call the user C and have the user A and the user B participate in the call.

In addition, an example was given when he said, "This case requires explanation by user C." As another example, for example, when User A or User B says "I need an explanation of the virtual assistant" during a call, the virtual assistant 41 is called and the virtual assistant 41 is asked "What do you want?" And so on.

It should be noted that any virtual assistant 41 can be applied to the present embodiment regardless of the type / type of the virtual assistant 41, what kind of action the engine is mounted on, and the like.

As described above, according to the present embodiment, the same effect as that of the first embodiment can be obtained.

Further, as illustrated in the specific KW registration DB 59, the packet monitoring device 10 can include a storage unit that stores a plurality of specific keywords. This makes it possible to call the virtual assistant for a plurality of specific keywords.

Further, the control unit 11 can make the calling process different according to the specific keyword included in the voice recognition result. Such various examples will be specifically described below.

For example, as shown in the example of reading the action data 59b and designating the action command, the control unit 11 outputs at least one of the voice content (behavior) and the voice quality according to the specific keyword included in the voice recognition result. Can be specified to execute the call processing. In this case, it is assumed that one virtual assistant 41 can change at least one of behavior and voice quality. The content of the voice output by the virtual assistant may refer to the action of the virtual assistant, or may indicate the content associated with the action.

As an alternative process, the control unit 11 can also specify information indicating a specific keyword included in the voice recognition result and execute the call process. In this case, the virtual assistant 41 may make at least one of the output voice content (behavior) and voice quality different according to the designated information (specific keyword indicated by). In the virtual assistant 41, the information indicating the specific keyword may be associated with the action to be taken and / or the voice quality to be used.

Further, when the control unit 11 specifies information indicating a specific keyword included in the voice recognition result and executes the call processing, the packet monitoring device 10 selects and provides a virtual assistant according to the designated information. It can also be done. However, in this case, it is premised that one or both of the server 40 and the packet monitoring device 10 can provide a plurality of virtual assistants.

For example, the server 40 may select and provide a virtual assistant according to the specified information. In this case, it is assumed that the server 40 is configured to be able to provide a plurality of virtual assistants. Alternatively, the packet monitoring device 10 may select and read the virtual assistant according to the specified information (in other words, according to a specific keyword). In this case, it is assumed that the virtual assistant storage unit of the packet monitoring device 10 stores a plurality of virtual assistants. Alternatively, the packet monitoring device 10 may select and call a virtual assistant to be called from a plurality of virtual assistants stored in both the virtual assistant storage unit and the server 40 according to the designated information.

Further, although the description has been made on the premise that the specific keyword is common regardless of the user during the call, the specific keyword applied to each user can be different. Specifically, the control unit 11 first identifies the user terminal (or the user who uses it) during a call from the communication packet data. Here, it is possible to specify the user terminal that is making a call only by the terminal information of the user terminal, but even when specifying the user, the user terminal and the user may be associated with each other. If the user is specified more accurately, for example, it can be executed from the utterance data of the voice for each user registered in advance by the voice recognition process.

Then, the control unit 11 may use different specific keywords as triggers for calling the virtual assistant 41 (triggers for call processing) according to the specified user terminal. The specific keyword that serves as a trigger can also be referred to as a trigger phrase, a wake word, or the like, and the latter is so referred to because it evokes the virtual assistant 41.

In this way, the wake word can be different for each user. It is also possible to register specific keywords so that one user can use two or more specific keywords.

Further, the description has been made on the premise that the control unit 11 identifies one user terminal (or one user). However, the control unit 11 only needs to specify at least one user terminal, and all or some of the users participating in the call, even if they are not one user terminal (or one user; the same applies hereinafter). You may specify the combination of terminals. In any case, the specified user terminal or the combination of user terminals may be associated with a specific keyword as a trigger for calling the virtual assistant 41 and stored. Thereby, the wake word can be made different according to the specified user terminal or the combination of the specified user terminal.

Further, when the control unit 11 identifies a user terminal during a call from communication packet data and executes a call process, the control unit 11 is configured to specify and execute a virtual assistant according to the specified user terminal. You can also. However, also in this case, it is premised that a plurality of virtual assistants can be provided by either one or both of the server 40 and the packet monitoring device 10.

Further, the control unit 11 can be configured to execute the voice recognition process, the call process, and the mixing process when the monitored communication packet data is the communication packet data of the call between the user terminals. That is, in this example, the voice recognition process, the call process, and the mixing process are executed only for the extension call, not for the outside line call. As a result, it is possible to prevent unnecessary mixing for outside line calls that are expected to be made outside the company.

Further, it is preferable that the control unit 11 has a setting unit for changing a specific keyword based on an operation from the user terminal. As a result, the user of the user terminal can specify a wake word to call the virtual assistant, and the wake word is not fixed to one wake word, so that the user convenience can be further improved.

The various examples described in the present embodiment can be appropriately implemented by one or a combination of two or more.

<Other embodiments>
In each of the above-described embodiments, each part of the device such as the packet monitoring device and the private branch exchange has been described, but it is sufficient that each device can realize these functions.

Further, in each of the above-described embodiments, the description is limited to the situation where the users talk to each other by the telephone. However, each of the above-described embodiments can be similarly applied to a call involving video such as a video conference or a video call (which may be accompanied by a chat). For example, it can be said that the system on the terminal side for video conference includes a private telephone or an external telephone. It can be said that a terminal such as a smartphone having a video call function by application software or the like includes a private telephone or an external telephone. In addition, when making a call with video, it is possible to call not only audio but also video as a virtual assistant.

Further, the devices such as the packet monitoring device, the private branch exchange, the server device, and the telephone according to each embodiment can have the following hardware configurations. FIG. 11 is a diagram showing an example of the hardware configuration of the device according to each embodiment.

The device 100 shown in FIG. 11 has a processor 101, a memory 102, and an interface 103. The interface 103 can be an interface for connecting to a device to be connected (not shown). The function of each device described in each embodiment can be realized by the processor 101 reading the program stored in the memory 102 and executing the program in cooperation with the interface 103 as needed. This program can be the program described in each embodiment.

In the above example, the program can be stored and supplied to a computer using various types of non-transitory computer readable medium. Non-temporary computer-readable media include various types of tangible storage media. Examples of non-temporary computer-readable media include magnetic recording media (eg, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (eg, magneto-optical disks). Further, this example includes a CD-ROM (Read Only Memory), a CD-R, and a CD-R / W. Further, this example includes semiconductor memories (eg, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)). The program may also be supplied to the computer by various types of transient computer readable medium. Examples of temporary computer readable media include electrical, optical, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

Further, in the various embodiments described above, the procedure of the virtual assistant calling method in the packet monitoring device has been described, for example, with reference to FIG. As described above, the present disclosure may also take the form of a virtual assistant calling method including a monitoring step, a recognition step, a calling step, and a mixing step executed by the packet monitoring device (or a private branch exchange having the function thereof). Further, it can be said that the above program is a program for causing a computer (control computer) inside the packet monitoring device or the private branch exchange to execute the above-mentioned virtual assistant calling method.

The present disclosure is not limited to the various embodiments described above, and can be appropriately modified without departing from the spirit. Further, the present disclosure may be carried out by appropriately combining the respective embodiments.

Some or all of the above embodiments may also be described, but not limited to:
<Additional notes>

(Appendix 1)
In a network in which a plurality of private telephones and a private branch exchange accommodating the plurality of private telephones are connected, a monitoring unit for monitoring communication packet data of a call between telephones including at least one of the private telephones, and a monitoring unit.
Control unit and
Equipped with
The control unit executes voice recognition processing on the communication packet data monitored by the monitoring unit, and calls a virtual assistant when a predetermined keyword is included in the recognition result in the voice recognition processing. Execute the process and execute the mixing process to mix the voice of the virtual assistant with the call.
Packet monitoring device.

(Appendix 2)
A storage unit that stores a plurality of the predetermined keywords is provided.
The packet monitoring device according to Appendix 1.

(Appendix 3)
The control unit makes the calling process different according to the predetermined keyword included in the recognition result.
The packet monitoring device according to Appendix 2.

(Appendix 4)
The control unit executes the calling process by designating at least one of the content and voice quality of the voice to be output according to the predetermined keyword included in the recognition result.
The packet monitoring device according to Appendix 3.

(Appendix 5)
The control unit executes the calling process by designating information indicating the predetermined keyword included in the recognition result.
The virtual assistant makes at least one of the output audio content and voice quality different according to the specified information.
The packet monitoring device according to Appendix 3.

(Appendix 6)
Multiple virtual assistants can be provided,
The control unit executes the calling process by designating information indicating the predetermined keyword included in the recognition result.
The virtual assistant is selected and provided according to the specified information.
The packet monitoring device according to Appendix 3.

(Appendix 7)
The control unit identifies the private telephone in a call from the communication packet data, and makes the predetermined keyword that is a trigger for calling the virtual assistant different according to the identified private telephone.
The packet monitoring device according to any one of Supplementary note 2 to 6.

(Appendix 8)
Multiple virtual assistants can be provided,
When the control unit identifies the private telephone during a call from the communication packet data and executes the call processing, the control unit designates and executes the virtual assistant according to the specified private telephone.
The packet monitoring device according to any one of Supplementary note 1 to 7.

(Appendix 9)
The control unit executes the voice recognition process, the call process, and the mixing process when the communication packet data monitored by the monitoring unit is the communication packet data of a call between the private telephones.
The packet monitoring device according to any one of Supplementary note 1 to 8.

(Appendix 10)
The control unit has a setting unit that changes the predetermined keyword based on an operation from the private telephone.
The packet monitoring device according to any one of Supplementary note 1 to 9.

(Appendix 11)
When the recognition result by the voice recognition process after the mixing process contains another predetermined keyword, the control unit executes a stop process for discontinuing the provision of the virtual assistant.
The packet monitoring device according to any one of Supplementary note 1 to 10.

(Appendix 12)
Connected to the server device that provides the virtual assistant via the network,
The call process includes a process of causing the server device to provide the virtual assistant via the network.
The packet monitoring device according to any one of Supplementary note 1 to 11.

(Appendix 13)
A virtual assistant storage unit for storing the virtual assistant so that it can be read from the control unit is provided.
The packet monitoring device according to any one of Supplementary note 1 to 12.

(Appendix 14)
A connection part that connects multiple premises telephones to be accommodated via a network,
In the network, a monitoring unit that monitors communication packet data of a call between telephones including at least one of the private telephones, and a monitoring unit.
Control unit and
Equipped with
The control unit executes voice recognition processing on the communication packet data monitored by the monitoring unit, and calls a virtual assistant when a predetermined keyword is included in the recognition result in the voice recognition processing. Execute the process and execute the mixing process to mix the voice of the virtual assistant with the call.
Private branch exchange.

(Appendix 15)
A monitoring step in which a packet monitoring device monitors communication packet data of a call between telephones including at least one of the private telephones in a network in which a plurality of private telephones and a private branch exchange accommodating the plurality of private telephones are connected. When,
A step in which the packet monitoring device executes voice recognition processing on the communication packet data monitored in the monitoring step, and
When the packet monitoring device includes a predetermined keyword in the recognition result in the voice recognition process, a step of executing a call process for calling the virtual assistant, and a step of executing the call process.
A step in which the packet monitoring device executes a mixing process of mixing the voice of the virtual assistant into the call.
How to call a virtual assistant.

(Appendix 16)
On the computer
In a network in which a plurality of private telephones and a private branch exchange accommodating the plurality of private telephones are connected, a monitoring step for monitoring communication packet data of a call between telephones including at least one of the private telephones, and a monitoring step.
A step of executing voice recognition processing for the communication packet data monitored in the monitoring step, and
When a predetermined keyword is included in the recognition result in the voice recognition process, a step of executing a call process for calling the virtual assistant and a step of executing the call process.
A step of executing a mixing process for mixing the voice of the virtual assistant into the call, and
A program to execute.

1, 10 Packet monitoring device 1a, 11 Control unit 1b Monitoring unit 2, 20 Private branch exchange control unit 2b Private branch exchange monitoring unit 2c Connection unit 12 Communication unit 13 Packet monitoring analysis unit 14 Virtual assistant calling unit 14a Virtual assistant Call processing unit 15 Voice mixing unit 30a, 30b User terminal (telephone)
40 Server 41 Virtual Assistant 51 Packet Monitoring Unit 52 SIP Header Information Storage Unit 53 Packet Analysis Unit 54 SIP Header Information Reading Unit 55 RTP Data Reading Unit 56 Voice Data Text Conversion Unit 57 Data Comparison Unit 58 Command Instruction Unit 58a Matching Judgment Unit 58b Virtual Assistant call pre-processing unit 59 Specific KW registration DB
59a Registration data 59b Behavior data 100 Device 101 Processor 102 Memory 103 Interface

Claims (9)

In a network in which a plurality of private telephones and a private branch exchange accommodating the plurality of private telephones are connected, a monitoring unit for monitoring communication packet data of a call between telephones including at least one of the private telephones, and a monitoring unit.
Control unit and
Equipped with
When the communication packet data monitored by the monitoring unit is the communication packet data of a call between the premises telephones, the control unit performs voice recognition processing on the communication packet data monitored by the monitoring unit. When the recognition result in the voice recognition process includes a predetermined keyword, a call process for calling the virtual assistant is executed, and a mixing process for mixing the voice of the virtual assistant into the call is executed.
Packet monitoring device. A storage unit that stores a plurality of the predetermined keywords is provided.
The packet monitoring device according to claim 1. The control unit makes the calling process different according to the predetermined keyword included in the recognition result.
The packet monitoring device according to claim 2. The control unit identifies the private telephone in a call from the communication packet data, and makes the predetermined keyword that is a trigger for calling the virtual assistant different according to the identified private telephone.
The packet monitoring device according to claim 2 or 3. Multiple virtual assistants can be provided,
When the control unit identifies the private telephone during a call from the communication packet data and executes the call processing, the control unit designates and executes the virtual assistant according to the specified private telephone.
The packet monitoring device according to any one of claims 1 to 4. The control unit has a setting unit that changes the predetermined keyword based on an operation from the private telephone.
The packet monitoring device according to any one of claims 1 to 5 . A connection part that connects multiple premises telephones to be accommodated via a network,
In the network, a monitoring unit that monitors communication packet data of a call between telephones including at least one of the private telephones, and a monitoring unit.
Control unit and
Equipped with
When the communication packet data monitored by the monitoring unit is the communication packet data of a call between the premises telephones, the control unit performs voice recognition processing on the communication packet data monitored by the monitoring unit. When the recognition result in the voice recognition process includes a predetermined keyword, a call process for calling the virtual assistant is executed, and a mixing process for mixing the voice of the virtual assistant into the call is executed.
Private branch exchange. A monitoring step in which a packet monitoring device monitors communication packet data of a call between telephones including at least one of the private telephones in a network in which a plurality of private telephones and a private branch exchange accommodating the plurality of private telephones are connected. When,
When the communication packet data monitored in the monitoring step is the communication packet data of a call between the premises telephones , the packet monitoring device performs voice recognition processing on the communication packet data monitored in the monitoring step. And the steps to perform
When the packet monitoring device includes a predetermined keyword in the recognition result in the voice recognition process, a step of executing a call process for calling the virtual assistant, and a step of executing the call process.
A step in which the packet monitoring device executes a mixing process of mixing the voice of the virtual assistant into the call.
How to call a virtual assistant. On the computer
In a network in which a plurality of private telephones and a private branch exchange accommodating the plurality of private telephones are connected, a monitoring step for monitoring communication packet data of a call between telephones including at least one of the private telephones, and a monitoring step.
When the communication packet data monitored in the monitoring step is the communication packet data of a call between the private telephones, a step of executing voice recognition processing on the communication packet data monitored in the monitoring step, and a step of executing voice recognition processing.
When a predetermined keyword is included in the recognition result in the voice recognition process, a step of executing a call process for calling the virtual assistant and a step of executing the call process.
A step of executing a mixing process for mixing the voice of the virtual assistant into the call, and
A program to execute.

Images