JP2023105607A - Program, information processing device, and information processing method - Google Patents

Abstract

To provide an information processing device allowing a customer to achieve a talk with a user in more appropriate voice in talks between a plurality of users, an information processing method, and a program.SOLUTION: A program causes a computer to allow a talk between a first user and a second user. The program causes a processor of an information processing device to perform a voice acquisition step, a conversion step, an output step, and an attribute acquisition step. The voice acquisition step acquires a talk voice from the first user. The conversion step converts the talk voice acquired by the voice acquisition step. The output step outputs the talk voice converted by the conversion step to the second user. The attribute acquisition step acquires talk attributes on a talk. The conversion step converts the talk voice acquired by the voice acquisition step on the basis of the talk attributes acquired by the attribute acquisition step.SELECTED DRAWING: Figure 12

Description

The present disclosure relates to a program, an information processing device, and an information processing method.

Conventionally, among acoustic devices such as earphones and headphones worn mainly on the head by users, there have been known acoustic devices capable of suppressing environmental sounds (so-called noise) from the external environment and enhancing the sound insulation effect. ing.
Japanese Patent Laid-Open No. 2002-200002 discloses a technique that enables listening to sound in a more suitable manner without complicated operations even under circumstances where the user's state or situation changes from time to time.
Patent Literature 2 discloses a technique for automatically determining an appropriate noise canceling filter.
Patent Literature 3 discloses a speech recognition apparatus that can cope with an environment such as a sudden change in environment or a plurality of environments appearing alternately.

Retable No. 2018/61491 JP 2020-86099 A JP-A-2000-330587

However, in a call between a plurality of users such as a user and a customer, it has not been possible to realize a call more suitable for the user.

Therefore, the present disclosure has been made to solve the above problems, and its purpose is to provide a technology that enables a customer to talk with a user with a more suitable voice in a call between a plurality of users. It is to be.

A program, comprising a processor and a storage unit, for performing a telephone call between a first user and a second user in a computer, the program comprising: a voice acquisition step for acquiring a call voice from the first user in the processor; a conversion step of converting the call voice acquired in the voice acquisition step; an output step of outputting the call voice converted in the conversion step to the second user; and an attribute acquisition step of acquiring call attributes related to the call. and the converting step includes converting the call voice obtained in the voice obtaining step based on the call attribute obtained in the attribute obtaining step.

Advantageous Effects of Invention According to the present disclosure, in a call between multiple users, the customer can make a call with the user with a more suitable voice.

1 is a diagram showing an overall configuration of an information processing system 1; FIG. 3 is a block diagram showing the functional configuration of the server 10; FIG. 2 is a block diagram showing a functional configuration of a user terminal 20; FIG. 3 is a block diagram showing the functional configuration of a CRM system 30; FIG. 3 is a block diagram showing the functional configuration of a customer terminal 50; FIG. FIG. 10 is a diagram showing the data structure of a user table 1012; FIG. FIG. 10 is a diagram showing the data structure of an organization table 1013; FIG. It is a figure which shows the data structure of the call table 1014. FIG. FIG. 10 is a diagram showing the data structure of a voice processing table 1015; FIG. FIG. 4 is a diagram showing the data structure of a learning data set 1031; FIG. 11 shows the data structure of a customer table 3012; FIG. It is a flowchart which shows the operation|movement of a sound-conversion process (1st Example). It is a flowchart which shows the operation|movement of a sound-conversion process (2nd Example). It is a flowchart which shows the operation|movement of a sound-conversion process (3rd Example). FIG. 11 is a diagram showing an example of a display screen of the user terminal 20 in voice conversion processing (third embodiment); 2 is a block diagram showing the basic hardware configuration of computer 90. FIG.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In all the drawings for explaining the embodiments, common constituent elements are denoted by the same reference numerals, and repeated explanations are omitted. It should be noted that the following embodiments do not unduly limit the content of the present disclosure described in the claims. Also, not all the components shown in the embodiments are essential components of the present disclosure. Each figure is a schematic diagram and is not necessarily strictly illustrated.

<Overview of information processing system 1>
An information processing system 1 according to the present disclosure is an information processing system that provides a call service according to the present disclosure. The information processing system 1 is an information processing system for providing services related to calls between users and customers, and for storing and managing data related to calls.

<Basic Configuration of Information Processing System 1>
The information processing system 1 is connected via a network N to a server 10, a plurality of user terminals 20A, 20B, 20C, a CRM system 30, a voice server (PBX) 40, and a voice server (PBX) 40. It comprises customer terminals 50A, 50B, and 50C connected via a telephone network T. FIG.

FIG. 1 is a diagram showing the overall configuration of an information processing system 1. As shown in FIG.
FIG. 2 is a block diagram showing the functional configuration of the server 10. As shown in FIG.
FIG. 3 is a block diagram showing the functional configuration of the user terminal 20. As shown in FIG.
FIG. 4 is a block diagram showing the functional configuration of the CRM system 30. As shown in FIG.
FIG. 5 is a block diagram showing the functional configuration of the customer terminal 50. As shown in FIG.

The server 10 is an information processing device that provides a service of storing and managing data (call data) related to calls made between users and customers.

The user terminal 20 is an information processing device operated by a user who uses the service. The user terminal 20 may be, for example, a stationary PC (Personal Computer), a laptop PC, or a mobile terminal such as a smart phone or a tablet. Moreover, it may be a wearable terminal such as an HMD (Head Mount Display) or a wristwatch type terminal.

The CRM system 30 is an information processing device managed and operated by a company (CRM company) that provides CRM (Customer Relationship Management) services. CRM services include SalesForce, HubSpot, Zoho CRM, Kintone, and the like.

The voice server (PBX) 40 is an information processing device that functions as a switching system that enables communication between the user terminal 20 and the customer terminal 50 by connecting the network N and the telephone network T to each other.

The customer terminal 50 is an information processing device operated by the customer when talking to the user. The customer terminal 50 may be, for example, a mobile terminal such as a smart phone or a tablet, a stationary PC (Personal Computer), or a laptop PC. Moreover, it may be a wearable terminal such as an HMD (Head Mount Display) or a wristwatch type terminal.

Each information processing device is composed of a computer having an arithmetic device and a storage device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by the hardware configuration will be described later. Descriptions of the server 10, the user terminal 20, the CRM system 30, the voice server (PBX) 40, and the customer terminal 50 that overlap with the basic hardware configuration of the computer and the basic functional configuration of the computer, which will be described later, will be omitted.

The configuration and operation of each device will be described below.

<Functional Configuration of Server 10>
FIG. 2 shows a functional configuration realized by the hardware configuration of the server 10. As shown in FIG. The server 10 has a storage unit 101 and a control unit 104 .

<Configuration of Storage Unit of Server 10>
Storage unit 101 of server 10 includes application program 1011 , user table 1012 , organization table 1013 , call table 1014 , speech processing table 1015 , evaluation model 1021 , generation model 1022 , speech processing model 1023 , and learning data set 1031 .
FIG. 6 is a diagram showing the data structure of the user table 1012. As shown in FIG.
FIG. 7 is a diagram showing the data structure of the organization table 1013. As shown in FIG.
FIG. 8 is a diagram showing the data structure of the call table 1014. As shown in FIG.
FIG. 9 is a diagram showing the data structure of the audio processing table 1015. As shown in FIG.
FIG. 10 is a diagram showing the data structure of the learning data set 1031. As shown in FIG.

The user table 1012 is a table that stores and manages information on member users (hereinafter referred to as users) who use the service. By registering to use the service, the user's information is stored in a new record in the user table 1012 . This enables the user to use the service according to the present disclosure. The user table 1012 is a table having user ID as a primary key and columns of user ID, CRM ID, organization ID, user name, and user attribute.

User ID is an item that stores user identification information for identifying a user.
CRMID is an item that stores identification information for identifying a user in the CRM system 30 . The user can receive CRM services by logging into the CRM system 30 with the CRM ID. That is, the user ID in the server 10 and the CRMID in the CRM system 30 are linked.
The organization ID is an item that stores the organization ID of the organization to which the user belongs.
The user name is an item that stores the name of the user.
The user attribute is an item that stores information related to user attributes such as age, gender, hometown, dialect, occupation (sales, customer support, etc.) of the user.

The organization table 1013 is a table that defines information about the organization to which the user belongs. Organizations include arbitrary organizations and groups such as companies, corporations, corporate groups, circles, and various organizations. Organizations may also be defined by more detailed sub-groups such as company departments (Sales Department, General Affairs Department, Customer Support Department). The organization table 1013 is a table having columns of organization ID, organization name, and organization attribute with organization ID as a primary key.

The organization ID is an item that stores organization identification information for identifying an organization.
The organization name is an item that stores the name of the organization. The name of the organization includes arbitrary organization names and group names such as company names, corporate names, corporate group names, circle names, and various organization names.
The organization attribute is an item that stores information on the organization attributes such as the organization type (company, corporate group, other organization, etc.) and industry (real estate, finance, etc.).

Call table 1014 is a table that stores and manages call data related to calls made between users and customers. The call table 1014 is a table having call ID as a primary key and columns of call ID, user ID, customer ID, call category, incoming/outgoing type, and voice data.

The call ID is an item that stores call data identification information for identifying call data.
The user ID is an item that stores the user's user ID (user identification information) in a call between the user and the customer.
The customer ID is an item that stores the customer's customer ID (customer identification information) in a call between the user and the customer.
The call category is an item that stores the type (category) of calls made between the user and the customer. Call data is classified by call category. In the call category, values such as telephone operator, telemarketing, customer support, and technical support are stored according to the purpose of the call between the user and the customer.
The incoming/outgoing type is an item that stores information for distinguishing whether a call made between a user and a customer is originated by the user (outbound) or received by the user (inbound). .
Voice data is an item that stores voice data of a call made between a user and a customer. As the audio data format, various audio data formats such as mp4 and wav can be used. Also, it is possible to store reference information (path) to an audio data file located at another location.
The voice data may be data in a format in which identifiers are set so that the voice of the user and the voice of the customer are independently identifiable. In this case, the control unit 104 of the server 10 can perform independent analysis processing on the user's voice and the customer's voice.
In the present disclosure, video data including audio information may be used instead of audio data. Also, audio data in the present disclosure is a concept including audio data included in moving image data.

The audio processing table 1015 is a table that stores information (audio processing information) on audio processing such as effects and filters applied to audio data.
The voice processing table 1015 is a table having voice processing ID as a main key and columns of voice processing ID and voice processing content.

The audio processing ID is an item for storing audio processing identification information for identifying audio processing content.
The audio processing content is an item that stores the audio processing content to be applied to the audio data. References to functions, methods, programs, etc. that perform audio processing located elsewhere may be stored.
The audio processing content includes audio conversion processing such as changing the voice of the audio data. Voice conversion includes converting voice data into male voice, female voice, voice of a specific person, or voice of a specific character. Voice conversion includes converting voice data into voice of specific emotions (joy, sadness, anger, surprise, fear, disgust). Audio conversion includes conversion processing that changes the shape of the intensity (speech spectrum structure, frequency distribution) for each frequency included in audio data. Speech conversion includes conversion processing that changes the fundamental frequency, the intensity of intonation, the speed of speech, and the change of intonation (increase or decrease). Speech conversion includes processing to remove fillers (eg, hesitation such as er, er, etc.) contained in the speech.
The voice conversion processing may include processing for converting a person's voice component included in the voice data, and may not include processing for converting voice components other than the person's voice component, such as background noise, noise, and noise.

The evaluation model 1021 is a learning model that outputs (infers) evaluation index values using user attributes, customer attributes, call categories, call attributes such as incoming and outgoing call types, and voice data as input data.
The evaluation model 1021 does not need to be a single learning model, and may be implemented by switching between a plurality of independent learning models for each type of evaluation index to be output (evaluation type). For example, the evaluation model 1021 includes a plurality of different independent learning models according to the type of output evaluation index (first index, second index, etc.).
An example of the evaluation model is a SIIB (Speech intelligence in bits) model (hereinafter referred to as the first model). An SIIB score (hereinafter referred to as the first index) is obtained by applying the speech data to the first model. The SIIB score is described in arXiv:2104.08499, etc., and is a quantitative evaluation index regarding the ease of hearing (easiness of perception) on the listener's side using voice data as input data.
Evaluation models include, for example, HASPI (Hearing-aid speech index), ESTOI (Extended short-time objective intelligence), PESQ (Perceptual evaluation of speech quality), ViSQOL (Vi Prepared according to the evaluation index such as speech quality objective listener) I don't mind.
In addition, arbitrary machine learning, deep learning, artificial intelligence models, etc. may be constructed using call attributes and voice data as input data, and evaluation results such as questionnaires on the listener side as teacher data. For example, evaluation results such as questionnaires may include items such as reliability, credibility, comfort, comfort, preference, stress value, degree of intimidation, and interest. In other words, the learning model may be a learning model that uses call attributes and voice data as input data, and outputs evaluation indices such as reliability, credibility, comfortability, comfortability, preference, stress value, intimidation level, and interest on the listener side.

The generative model 1022 is a learning model that outputs (infers) converted voice data using call attributes and voice data as input data.
The learning process of the generative model 1022 will be described later.
The generative model 1022 does not need to be a single learning model, and may be implemented by switching between a plurality of independent learning models for each call attribute and evaluation type. Specifically, the generative model 1022 may include a plurality of independent learning models for each input call attribute and output evaluation type.
For example, the generative model 1022 may be implemented by selectively switching between a plurality of generative models that output suitable converted voice data according to each combination of call attributes.
For example, the generative model 1022 selectively switches between a first generative model and a second generative model that output suitable converted speech data for each of a plurality of evaluation indices such as a first index and a second index. It can be realized.

The voice processing model 1023 is a learning model that outputs (infers) a voice processing ID using call attributes and voice data as input data.
The learning process of the speech processing model 1023 will be described later.
The voice processing model 1023 does not need to be a single learning model, and may be implemented by switching between a plurality of independent learning models for each call attribute and evaluation type. Specifically, the voice processing model 1023 may include a plurality of independent learning models for each input call attribute and output evaluation type.
For example, the voice processing model 1023 may be implemented by selectively switching between a plurality of voice processing models that output suitable converted voice data according to each combination of call attributes.
For example, the speech processing model 1023 selectively selects a first speech processing model, a second speech processing model, etc. that output suitable converted speech data for each of a plurality of evaluation indices such as the first index and the second index. It may be realized by switching to

The evaluation model 1021, the generative model 1022, and the speech processing model 1023 are, for example, types of machine learning, artificial intelligence, deep learning models, and the like.
As an example of the evaluation model 1021, the generative model 1022, and the speech processing model 1023, a deep learning model using a deep neural network in deep learning will be described. The deep learning model is any deep learning model that takes arbitrary time series data as input data, such as RNN (Recurrent Neural Network), LSTM (Long Short Term Memory), GRU (Gated Recurrent Unit), etc. It doesn't matter if there is. A learning model includes, for example, any deep learning model including Attention, Transformer, and the like.
The evaluation model 1021, generative model 1022, and speech processing model 1023 do not necessarily need to be deep learning models, and may be arbitrary machine learning or artificial intelligence models.

The learning data set 1031 is a table that stores data sets used for learning processing of the generative model 1022 . The learning data set 1031 is a data set used for learning processing such as machine learning and deep learning in which attribute information related to users, attribute information related to customers, and attribute information related to calls in call data are associated and stored.
The learning data set 1031 may be created by combining the call table 1014, the user table 1012, the organization table 1013, the customer table 3012, etc., which store past call data between users and customers.
The learning data set 1031 is a table having columns of user attribute information, customer attribute information, call attribute information, voice data, first index, second index, and third index.

The attribute information about the user is an item that stores the user attribute of the user, the organization name of the organization to which the user belongs, or the information of the organization attribute in the call data. The attribute information about the user may include information (emotional information) about the user's emotions (joy, sadness, anger, surprise, fear, disgust) in call data.
The attribute information about the customer is an item that stores the customer attribute of the customer, the organization name of the organization to which the customer belongs, or the information of the organization attribute in the call data. The customer-related attribute information may include the customer's emotion information in the call data.
Attribute information about a call is an item that stores information on the call category and the type of caller/receiver in the call data. The call attribute information may include user and customer sentiment information in the call data.
Voice data is an item that stores voice data of a call made between a user and a customer. Since it is the same as the voice data of the call table 1014, the description is omitted.

<Configuration of Control Unit of Server 10>
The control unit 104 of the server 10 includes a user registration control unit 1041 , a voice conversion unit 1042 and a learning unit 1051 . Control unit 104 implements each functional unit by executing application program 1011 stored in storage unit 101 .

The user registration control unit 1041 performs processing for storing information of users who wish to use the service according to the present disclosure in the user table 1012 .
The information stored in the user table 1012 is transmitted to the server 10 by the user opening a web page or the like operated by the service provider from any information processing terminal, inputting the information into a predetermined input form. The user registration control unit 1041 of the server 10 stores the received information in a new record of the user table 1012, and user registration is completed. As a result, the users stored in the user table 1012 can use the service.
Prior to registration of user information in the user table 1012 by the user registration control unit 1041, the service provider may perform a predetermined examination to limit whether the user can use the service.
The user ID may be any character string or number that can identify the user, any character string or number desired by the user, or any character string or number automatically set by the user registration control unit 1041 of the server 10. can be

The voice conversion unit 1042 executes voice conversion processing (first example), voice conversion processing (second example), voice conversion processing (third example), and voice conversion processing (fourth example). Details will be described later.
The learning unit 1051 executes learning processing. Details will be described later.

<Functional Configuration of User Terminal 20>
A functional configuration realized by the hardware configuration of the user terminal 20 is shown in FIG. The user terminal 20 includes a storage unit 201 , a control unit 204 , an input device 206 connected to the user terminal 20 and an output device 208 . Input device 206 includes camera 2061 , microphone 2062 , position information sensor 2063 , motion sensor 2064 , keyboard 2065 and mouse 2066 . The output device 208 includes a display 2081 and speakers 2082 .

<Configuration of Storage Unit of User Terminal 20>
The storage unit 201 of the user terminal 20 stores a user ID 2011 for identifying a user who uses the user terminal 20, an application program 2012, and a CRM ID 2013.
The user ID is the user's account ID for the server 10 . The user transmits the user ID 2011 from the user terminal 20 to the server 10 . The server 10 identifies the user based on the user ID 2011 and provides the user with the service according to the present disclosure. The user ID includes information such as a session ID temporarily assigned by the server 10 to identify the user using the user terminal 20 .
CRMID is the user's account ID for the CRM system 30 . The user transmits the CRMID 2013 from the user terminal 20 to the CRM system 30 . The CRM system 30 identifies the user based on the CRMID 2013 and provides the CRM service to the user. The CRMID 2013 includes information such as a session ID temporarily assigned by the CRM system 30 to identify the user using the user terminal 20 .
The application program 2012 may be stored in the storage unit 201 in advance, or may be downloaded from a web server or the like operated by the service provider via the communication IF. Application program 2012 includes an interpreted programming language such as JavaScript (registered trademark) that runs on a web browser application stored in user terminal 20 .

<Configuration of Control Unit of User Terminal 20>
The control unit 204 of the user terminal 20 has an input control unit 2041 and an output control unit 2042 . By executing the application program 2012 stored in the storage unit 201 , the control unit 204 implements functional units of an input control unit 2041 and an output control unit 2042 .
The input control unit 2041 of the user terminal 20 acquires information output from input devices such as a camera 2061, a microphone 2062, a position information sensor 2063, a motion sensor 2064, a keyboard 2065, and a mouse 2066 connected to the user terminal 20, and various Execute the process. The input control unit 2041 of the user terminal 20 executes a process of transmitting information acquired from the input device 206 to the server 10 together with the user ID 2011 . Similarly, the input control unit 2041 of the user terminal 20 performs a process of transmitting information acquired from the input device 206 to the CRM system 30 together with the CRMID 2013 .
The output control unit 2042 of the user terminal 20 receives information from the user's operation on the input device 206 and the server 10 and the CRM system 30, and determines the display content of the display 2081 connected to the user terminal 20 and the audio output content of the speaker 2082. Execute control processing.

<Functional Configuration of CRM System 30>
A functional configuration realized by the hardware configuration of the CRM system 30 is shown in FIG. The CRM system 30 has a storage unit 301 and a control unit 304 .
The user has also concluded a separate contract with a CRM company, and accesses (logs in to) the website operated by the CRM company via a web browser using the CRM ID 2013 set for each user to access the CRM service. can be provided.

<Configuration of Storage Unit of CRM System 30>
The storage unit 301 of the CRM system 30 has a customer table 3012 .
FIG. 11 is a diagram showing the data structure of the customer table 3012. As shown in FIG.

The customer table 3012 is a table for storing and managing customer information. The customer table 3012 is a table having customer ID as a primary key and columns of customer ID, user ID, name, telephone number, customer attribute, customer organization name, and customer organization attribute.

The customer ID is an item that stores customer identification information for identifying the customer.
The user ID is an item that stores the user ID (user identification information) of the user associated with the customer. The user can display a list of customers associated with his/her own user ID, and can make calls (calls) to the customers.
In the present disclosure, the customer is associated with the user, but may be associated with the organization (organization ID of the organization table 1013). In that case, a user belonging to an organization can display a list of customers associated with his/her own organization ID, or can send a message to the customer.
The name is an item for storing the customer's name.
The phone number is an item that stores the customer's phone number.
A user accesses a website provided by the CRM system, selects a customer to call, and performs a predetermined operation such as "Call" to make a call to the customer's telephone number from the user terminal 20. be able to.
The customer attribute is an item that stores information related to customer attributes such as customer age, gender, hometown, dialect, occupation (sales, customer support, etc.).
The customer organization name is an item that stores the name of the organization to which the customer belongs. The name of the organization includes arbitrary organization names and group names such as company names, corporate names, corporate group names, circle names, and various organization names.
The customer organization attribute is an item that stores information related to organization attributes such as the customer's organization type (company, corporate group, other organization, etc.) and type of industry (real estate, finance, etc.).
The customer attribute, the customer organization name, and the customer organization attribute may be stored by inputting them by the user, or may be input by the customer when the customer accesses a predetermined website.

<Configuration of Control Unit of CRM System 30>
The control section 304 of the CRM system 30 has a user registration control section 3041 . Control unit 304 implements each functional unit by executing application program 3011 stored in storage unit 301 .

The CRM system 30 provides functions called API (Application Programming Interface), SDK (Software Development Kit), and code snippets (hereinafter referred to as "beacons"), and the user can use the server 10 and By setting account information and the like for the CRM system 30, the control unit 104 of the server 10 and the control unit 304 of the CRM system 30 can communicate with each other to realize arbitrary information processing.

<Outline of voice server (PBX) 40>
The voice server (PBX) 40 makes a call to the customer terminal 50 when the user makes a call to the customer.
The voice server (PBX) 40 sends a message (hereinafter referred to as an "incoming call notification message") to the user terminal 20 to indicate that the customer has made a call to the user. Also, the voice server (PBX) 40 can send an incoming call notification message to a beacon, SDK, API, etc. provided by the server 10 .

<Functional Configuration of Customer Terminal 50>
FIG. 5 shows a functional configuration realized by the hardware configuration of the customer terminal 50. As shown in FIG. The customer terminal 50 includes a storage unit 501 , a control unit 504 , a touch panel 506 , a touch sensitive device 5061 , a display 5062 , a microphone 5081 , a speaker 5082 , a position information sensor 5083 , a camera 5084 and a motion sensor 5085 .

<Configuration of Storage Unit of Customer Terminal 50>
The storage unit 501 of the customer terminal 50 stores telephone numbers 5011 and application programs 5012 of customers who use the customer terminal 50 .
The application program 5012 may be stored in advance in the storage unit 501, or may be downloaded from a web server or the like operated by the service provider via the communication IF. The application program 5012 includes an interpreted programming language such as JavaScript (registered trademark) executed on a web browser application stored in the customer terminal 50 .

<Configuration of Control Unit of Customer Terminal 50>
The control section 504 of the customer terminal 50 has an input control section 5041 and an output control section 5042 . By executing the application program 5012 stored in the storage unit 501 , the control unit 504 realizes the functional units of the input control unit 5041 and the output control unit 5042 .
The input control unit 5041 of the customer terminal 50 outputs from input devices such as the user's operation content to the touch sensitive device 5061 of the touch panel 506, voice input to the microphone 5081, position information sensor 5083, camera 5084, motion sensor 5085, etc. Acquire information and execute various processes.
The output control unit 5042 of the customer terminal 50 receives information from the user's operation on the input device and the server 10, and executes control processing of display contents of the display 5062, audio output contents of the speaker 5082, and the like.

<Operation of information processing system 1>
Each process of the information processing system 1 will be described below.
FIG. 12 is a flow chart showing the operation of voice conversion processing (first embodiment).
FIG. 13 is a flow chart showing the operation of voice conversion processing (second embodiment).
FIG. 14 is a flow chart showing the operation of voice conversion processing (third embodiment).
FIG. 15 is a diagram showing an example of the display screen of the user terminal 20 in the voice conversion process (third embodiment).

<Definition of terms>
In describing each process of the information processing system 1, terms are defined as follows.
The call data is data relating to calls made between the user and the customer, and is data including data stored in each item of the call table 1014 .
The call attribute is data related to the attributes of the call made between the user and the customer, and includes the user attribute, the organization name or organization attribute of the organization to which the user belongs, information on the user's emotion in the call (attribute information on the user), Customer attributes, organization name or organization attributes of the organization to which the customer belongs, information on the customer's emotions in the call (attribute information on the customer), call category, type of recipient and caller, information on the emotion in the call (attribute information on the call), etc. include. In other words, call data is characterized by call attributes such as user attribute information, customer attribute information, and call attribute information.
Call attributes in the present disclosure include attribute information about individual users and customers, and do not include information about the surrounding environment of users and customers and the call environment. For example, it does not include information about noise and noise conditions around the user and customer.

<Outgoing process>
The calling process is a process of making a call (calling) from the user to the customer.

<Outline of call processing>
The calling process is a series of processes in which the user selects a customer who wishes to make a call from among a plurality of customers displayed on the screen of the user terminal 20 and performs a calling operation to make a call to the customer.

<Details of outgoing processing>
A call processing of the information processing system 1 when a user makes a call to a customer will be described.

When a user calls a customer, the information processing system 1 performs the following processes.

By operating the user terminal 20 , the user activates the web browser and accesses the CRM service website provided by the CRM system 30 . The user can display a list of his/her own customers on the display 2081 of the user terminal 20 by opening the customer management screen provided by the CRM service.
Specifically, the user terminal 20 transmits to the CRM system 30 a CRM ID 2013 and a request to display a list of customers. When the CRM system 30 receives the request, it searches the customer table 3012 and transmits to the user terminal 20 the user's customer information such as the customer ID, name, telephone number, customer attributes, customer organization name, and customer organization attributes. The user terminal 20 displays the received customer information on the display 2081 of the user terminal 20 .

The user presses and selects a customer who wishes to make a call from the customers listed on the display 2081 of the user terminal 20 . With the customer selected, pressing the "call" button or phone number button displayed on the display 2081 of the user terminal 20 sends a request including the phone number to the CRM system 30 . The CRM system 30 that has received the request transmits the request including the telephone number to the server 10 . The server 10 that has received the request transmits a call origination request to the voice server (PBX) 40 . Upon receiving the call request, the voice server (PBX) 40 makes a call (call) to the customer terminal 50 based on the received telephone number.

Along with this, the user terminal 20 controls the speaker 2082 and the like to ring to indicate that the voice server (PBX) 40 is making a call (calling). Also, the display 2081 of the user terminal 20 displays information indicating that the voice server (PBX) 40 is making a call to the customer. For example, the display 2081 of the user terminal 20 may display the characters "calling".

When the customer picks up the receiver (not shown) of the customer terminal 50 or presses the "Receive" button displayed on the touch panel 506 of the customer terminal 50 when receiving a call, the customer terminal 50 becomes ready for communication. Along with this, the voice server (PBX) 40 transmits information indicating that the customer terminal 50 has responded (hereinafter referred to as a "response event") to the user terminal 20 via the server 10, the CRM system 30, and the like. Send.
As a result, the user and the customer are ready to talk using the user terminal 20 and the customer terminal 50, respectively, so that the user and the customer can talk to each other. Specifically, the user's voice collected by the microphone 2062 of the user terminal 20 is output from the speaker 5082 of the customer terminal 50 . Similarly, the customer's voice collected from the microphone 5081 of the customer terminal 50 is output from the speaker 2082 of the user terminal 20 .

The display 2081 of the user terminal 20 receives the response event and displays information indicating that a call is being made when the call is ready. For example, the display 2081 of the user terminal 20 may display the characters "answering".

<Incoming processing>
Incoming call processing is processing in which a user receives a call (receives a call) from a customer.

<Overview of Incoming Call Processing>
The incoming call process is a series of processes in which the user receives an incoming call when the user has started an application on the user terminal 20 and the customer has made a call to the user.

<Details of incoming call processing>
Incoming call processing of the information processing system 1 when the user receives an incoming call (receives a call) from a customer will be described.

When a user receives an incoming call from a customer, the information processing system 1 performs the following processes.

By operating the user terminal 20 , the user activates the web browser and accesses the CRM service website provided by the CRM system 30 . At this time, it is assumed that the user logs in to the CRM system 30 with his own account on the web browser and waits. It is sufficient for the user to be logged in to the CRM system 30, and the user may be performing other work related to the CRM service.

The customer operates the customer terminal 50 , inputs a predetermined telephone number assigned to the voice server (PBX) 40 , and makes a call to the voice server (PBX) 40 . The voice server (PBX) 40 receives the outgoing call from the customer terminal 50 as an incoming call event.

A voice server (PBX) 40 sends an incoming call event to the server 10 . Specifically, the voice server (PBX) 40 transmits an incoming call request including the customer's telephone number 5011 to the server 10 . The server 10 transmits an incoming call request to the user terminal 20 via the CRM system 30 .
Along with this, the user terminal 20 controls the speaker 2082 and the like to ring to indicate that the voice server (PBX) 40 is receiving an incoming call. The display 2081 of the user terminal 20 displays information indicating that the voice server (PBX) 40 has received an incoming call from the customer. For example, the display 2081 of the user terminal 20 may display the characters "incoming call".

The user terminal 20 receives a response operation by the user. The response operation is, for example, by lifting the handset (not shown) of the user terminal 20, or by operating the mouse 2066 to press the button labeled "answer the call" on the display 2081 of the user terminal 20. Realized.
Upon accepting the response operation, the user terminal 20 transmits a response request to the voice server (PBX) 40 via the CRM system 30 and the server 10 . The voice server (PBX) 40 receives the transmitted response request and establishes voice communication. As a result, the user terminal 20 becomes ready for communication with the customer terminal 50 .
The display 2081 of the user terminal 20 displays information indicating that a call is being made. For example, the display 2081 of the user terminal 20 may display the characters "busy".

When it becomes possible to make a call, voice conversion processing (first embodiment), voice conversion processing (second embodiment), voice conversion processing (third embodiment), and voice conversion processing (fourth embodiment), which will be described later, are executed. be.
One of speech conversion processing (first embodiment), speech conversion processing (second embodiment), and speech conversion processing (third embodiment) is performed for each specific pair of speaker and listener. It is good also as a structure to be used. The voice conversion process (fourth embodiment) may be executed simultaneously with the voice conversion process (first embodiment), the voice conversion process (second embodiment), and the voice conversion process (third embodiment).
When three or more people are talking, voice conversion processing (first embodiment), voice conversion processing (second Second embodiment) or voice conversion processing (third embodiment) may be executed. That is, different speech conversion processes may be performed for each combination of two different speaker-listener pairs out of three.

<Modification>
It should be noted that the method by which the user can make a call with the customer is not limited to the outgoing call process and the incoming call process, and any method for realizing a call between the user and the customer may be used. For example, a virtual communication space called a room is created on the server 10 for communication between the user and the customer, and the user and the customer are stored in the user terminal 20 and the customer terminal 50 in the room. A method of enabling communication by accessing via a web browser or an application program may also be used. In this case, the voice server (PBX) 40 becomes unnecessary.
Specifically, the user who is the organizer of the call operates the input device 206 of the user terminal 20 to transmit a request for holding the call to the server 10 . Upon receiving the request, the control unit 104 of the server 10 issues room identification information such as a unique room ID and transmits a response to the user terminal 20 . The user transmits the received room identification information to the customer who is the other party of the call by any means of communication such as e-mail. The user can enter the room by operating the input device 206 of the user terminal 20, accessing the URL of the server 10 providing room-related services using a web browser, etc., and entering the room identification information. Similarly, the customer can enter the room by operating the touch panel 506 of the customer terminal 50, accessing the URL of the server 10 providing room-related services using a web browser, etc., and entering the room identification information. As a result, the user and the customer can talk via the user terminal 20 and the customer terminal 50, respectively, within a virtual communication space called a room associated with the room identification information.
By inputting room identification information, multiple users and multiple customers can enter one room. As a result, a plurality of users and a plurality of customers can talk via the user terminal 20 and the customer terminal 50 in a virtual communication space called a room associated with the room identification information. .

<Call memory processing>
A call storage process is the process of storing data relating to calls made between a user and a customer.

<Overview of call memory processing>
The call storage process is a series of processes for storing data related to calls in the call table 1014 when a call is started between a user and a customer.

<Details of call memory processing>
When a call is started between the user and the customer, the voice server (PBX) 40 records voice data regarding the call between the user and the customer and transmits the data to the server 10 . Upon receiving the voice data, the control unit 104 of the server 10 creates a new record in the call table 1014, and stores data relating to the call made between the user and the customer. Specifically, the control unit 104 of the server 10 stores the user ID, customer ID, call category, incoming/outgoing type, and contents of voice data in the call table 1014 .

The control unit 104 of the server 10 acquires the user ID 2011 of the user from the user terminal 20 in the outgoing call process or the incoming call process, and stores it in the user ID item of the new record.
The control unit 104 of the server 10 makes an inquiry to the CRM system 30 based on the telephone number in outgoing call processing or incoming call processing. The CRM system 30 acquires the customer ID by searching the customer table 3012 by telephone number and transmits it to the server 10 . The control unit 104 of the server 10 stores the acquired customer ID in the customer ID item of the new record.
The control unit 104 of the server 10 stores the value of the call category set in advance for each user or customer in the item of the call category of the new record. Note that the call category may be stored by the user selecting or inputting a value for each call.
The control unit 104 of the server 10 identifies whether the call being made is originated by the user or originated by the customer, and puts outbound (originating from the user) or inbound in the incoming/outgoing type item of the new record. (originating from the customer).
The control unit 104 of the server 10 stores the voice data received from the voice server (PBX) 40 in the voice data item of the new record. The voice data may be stored as a voice data file in another location, and the reference information (path) to the voice data file may be stored after the call is finished. Also, the control unit 104 of the server 10 may be configured to store data after the end of the call.

<Voice conversion processing (first embodiment)>
Speech conversion processing (first embodiment) is processing for outputting converted speech data obtained by applying a generative model 1022 selected based on call attributes to speech data uttered by a user to a customer. .
This allows the customer to communicate with the user with a more suitable voice.

<Outline of voice conversion processing (first embodiment)>
The voice conversion process (first embodiment) is started when the user and the customer are ready to talk. The voice conversion process (first embodiment) acquires call attributes, inputs the call attributes and voice data uttered by the user into the generation model 1022 as input data, and outputs converted voice data to the customer. It is a series of processes.

<Details of voice conversion processing (first embodiment)>
In step S101, when the user and the customer are ready to talk, the voice conversion process (first embodiment) is started.

In step S102, the voice conversion unit 1042 of the server 10 acquires call attributes related to the call.
Specifically, the speech conversion unit 1042 of the server 10 searches the user ID item of the user table 1012 based on the user ID of the user, and acquires the organization ID and user attribute items. The speech conversion unit 1042 of the server 10 searches the organization ID item in the organization table 1013 based on the acquired organization ID, and acquires the organization name and organization attribute items. That is, the voice conversion unit 1042 of the server 10 acquires attribute information regarding the user. The speech conversion unit 1042 of the server 10 may estimate the user's emotional state from the user's uttered voice and acquire the user's emotional information as attribute information about the user.
The voice conversion unit 1042 of the server 10 transmits an inquiry request including the customer's customer ID to the CRM system 30 . The CRM system 30 retrieves the customer ID item in the customer table 3012 based on the customer ID included in the received request, acquires the customer attribute, customer organization name, and customer organization attribute items, and transmits them to the server 10 . The voice conversion unit 1042 of the server 10 acquires the items of customer attribute, customer organization name, and customer organization attribute from the CRM system 30 . That is, the voice conversion unit 1042 of the server 10 acquires the attribute information regarding the customer. The voice conversion unit 1042 of the server 10 may estimate the customer's emotional state from the customer's uttered voice and acquire the customer's emotional information as the customer's attribute information.
The voice conversion unit 1042 of the server 10 refers to the call table 1014 and acquires the information on the call category and the type of incoming and outgoing calls included in the call data stored by the call storage process. The voice conversion unit 1042 of the server 10 searches the item of call ID in the call table 1014 based on the call ID related to the call, and acquires the items of the call category and the type of reception/transmission. That is, the voice conversion unit 1042 of the server 10 acquires the attribute information regarding the call. The speech conversion unit 1042 of the server 10 may estimate the emotional state of the user and the customer from the uttered voice of the user and the customer, and acquire the emotional information of the user and the customer as attribute information regarding the call.

The voice conversion unit 1042 of the server 10 may acquire at least one of attribute information about the user, attribute information about the customer, and attribute information about the call as the call attribute. For example, only attribute information about the user may be acquired as a call attribute.
In addition, the voice conversion unit 1042 of the server 10 is configured to convert the user attribute, the organization name or organization attribute of the organization to which the user belongs, the user's emotion information, the customer attribute, the organization name or organization attribute of the organization to which the customer belongs, and the customer's emotion information. , the category of the call, the type of the sender and receiver, and the emotional information of the call may be acquired as the call attribute. For example, only the call category may be acquired as a call attribute.

In step S104, the voice conversion unit 1042 of the server 10 acquires the call voice from the user and converts the acquired call voice. At this time, the voice conversion unit 1042 of the server 10 converts the acquired call voice based on the call attribute acquired in step S102. The voice conversion unit 1042 of the server 10 converts the acquired call voice by applying the generation model 1022 to the call attributes and call voice acquired in step S102.
Specifically, the voice conversion unit 1042 of the server 10 sequentially acquires voice data uttered by the user from the voice server (PBX) 40 . After the user speaks, the voice conversion unit 1042 of the server 10 preferably acquires voice data spoken by the user with as little delay as possible.
The speech conversion unit 1042 of the server 10 inputs the acquired call attributes and speech data to the generation model 1022 as input data, and acquires converted speech data to be output.

The speech conversion unit 1042 of the server 10 may selectively switch and apply a plurality of generation models 1022 according to the type of evaluation index of the converted speech data output to the customer, and convert the speech data.
For example, the speech data may be converted using a generative model 1022 that provides more credibility to the customer. For example, speech data may be converted using a generative model 1022 that is easier for the customer to hear (easier to hear). For example, evaluation indexes such as SIIB, HASPI, ESTOI, PESQ, and ViSQOL, and evaluation indexes such as reliability, credibility, comfort, comfort, preference, stress value, intimidation, and interest are suitable for customers. A generative model 1022 may be used to convert speech data.

The voice conversion unit 1042 of the server 10 may convert the call voice using at least one of the user attribute information, the customer attribute information, and the call attribute information as the call attribute. For example, the call voice may be converted using only the attribute information about the user as the call attribute.
In addition, the voice conversion unit 1042 of the server 10 can also convert the user attribute, the organization name or organization attribute of the organization to which the user belongs, the user's emotion information, the customer attribute, the organization name or organization attribute of the organization to which the customer belongs, and the customer's emotion information. , the category of the call, the type of the sender and receiver, or the emotional information of the call may be used as the call attribute to convert the call voice. For example, only the call category may be used as the call attribute to convert the call voice.

In step S105, the voice conversion unit 1042 of the server 10 outputs the call voice converted in step S104 to the customer. The voice converter 1042 of the server 10 transmits the converted voice data to the voice server (PBX) 40 . The voice server (PBX) 40 outputs the received converted voice data to the customer terminal 50 . The speaker 5082 of the customer terminal 50 outputs the received converted voice data as the user's call voice.
That is, voice data relating to the user's voice collected by the microphone 2062 of the user terminal 20 is converted into converted voice data by the voice converter 1042 of the server 10 and output from the speaker 5082 of the customer terminal 50 .

<Voice Conversion Processing (Second Embodiment)>
The voice conversion process (second embodiment) is a process of outputting to the customer converted voice data to which voice processing content specified by applying the voice processing model 1023 is applied to the voice data uttered by the user. be.
This allows the user to select speech processing that can translate into speech that is more suitable for the customer.

<Outline of voice conversion processing (second embodiment)>
The voice conversion process (second embodiment) is started when the user and the customer are ready to talk. The voice conversion process (second embodiment) acquires the call attribute, inputs the call attribute as input data to the voice processing model 1023, selects the voice processing content based on the output voice processing ID, and converts the voice uttered by the user. This is a series of processes for outputting the converted voice data to the customer by applying the selected voice processing content to the data.

<Details of voice conversion processing (second embodiment)>
In step S301, when the user and the customer are ready to talk, voice conversion processing (second embodiment) is started.

In step S302, the voice conversion unit 1042 of the server 10 acquires call attributes related to the call. Since step S302 is the same as step S102 in the voice conversion process (first embodiment), description thereof is omitted.

In step S303, the voice conversion unit 1042 of the server 10 selects a predetermined voice process from among a plurality of voice processes based on the acquired call attribute. The voice conversion unit 1042 of the server 10 selects predetermined voice processing by applying the voice processing model 1023 to the call attributes acquired in step S302.
Specifically, the voice conversion unit 1042 of the server 10 inputs the acquired call attribute as input data to the voice processing model 1023, and acquires the output voice processing ID. The voice conversion unit 1042 of the server 10 searches the item of the voice processing ID in the voice processing table 1015 based on the acquired voice processing ID, and acquires the content of the voice processing. That is, the voice conversion unit 1042 of the server 10 specifies and selects voice processing content based on the call attribute.

In step S304, the voice conversion unit 1042 of the server 10 acquires the call voice from the user and converts the acquired call voice. At this time, the voice conversion unit 1042 of the server 10 converts the acquired call voice based on the call attribute acquired in step S302. The voice conversion unit 1042 of the server 10 converts the call voice by applying the predetermined voice processing selected in step S302 to the call voice.
Specifically, the voice conversion unit 1042 of the server 10 sequentially acquires voice data uttered by the user from the voice server (PBX) 40 . After the user speaks, the voice conversion unit 1042 of the server 10 preferably acquires voice data spoken by the user with as little delay as possible.
The voice conversion unit 1042 of the server 10 applies the voice processing content selected in step S303 to the acquired call attribute and voice data, and acquires converted voice data to be output.

The speech conversion unit 1042 of the server 10 may selectively switch and apply a plurality of speech processing models 1023 according to the type of the evaluation index of the converted speech data output to the customer, and convert the speech data. .
For example, voice data may be converted using a voice processing model 1023 that provides more reliability from customers. For example, voice data may be converted using a voice processing model 1023 that is easier for the customer to hear (easier to hear). For example, evaluation indexes such as SIIB, HASPI, ESTOI, PESQ, and ViSQOL, and evaluation indexes such as reliability, credibility, comfort, comfort, preference, stress value, intimidation, and interest are suitable for customers. The audio data may be converted using an audio processing model 1023 such as

The voice conversion unit 1042 of the server 10 may convert the call voice using at least one of the user attribute information, the customer attribute information, and the call attribute information as the call attribute. For example, the call voice may be converted using only the attribute information about the user as the call attribute.
In addition, the voice conversion unit 1042 of the server 10 is configured to convert the user attribute, the organization name or organization attribute of the organization to which the user belongs, the user's emotion information, the customer attribute, the organization name or organization attribute of the organization to which the customer belongs, and the customer's emotion information. , the category of the call, the type of the sender and receiver, and the emotional information of the call may be used as the call attribute to convert the call voice. For example, only the call category may be used as the call attribute to convert the call voice.

In step S305, the voice conversion unit 1042 of the server 10 outputs the call voice converted in step S304 to the customer. Since step S305 is the same as step S105 in the voice conversion process (first embodiment), description thereof is omitted.

<Voice Conversion Processing (Third Embodiment)>
The speech conversion process (third embodiment) is a process of outputting converted speech data obtained by applying the speech processing model 1023 selected by the user to the speech data uttered by the user to the customer.
As a result, the customer can talk with the user in a more comfortable voice according to the user's selection instruction.

<Outline of voice conversion processing (third embodiment)>
The voice conversion process (third embodiment) is started when the user and the customer are ready to talk. In the speech conversion process (third embodiment), the user selects the content of speech processing, applies the selected speech processing content to the speech data uttered by the user, and outputs the converted speech data to the customer. It is a series of processing to output.

<Details of voice conversion processing (third embodiment)>
In step S501, when the user and the customer are ready to talk, voice conversion processing (third embodiment) is started.

In step S503, predetermined audio processing is selected from the plurality of audio processing based on the selection instruction received from the user. For example, by selecting voice processing that reduces intonation from a plurality of voice processing, it is possible to reduce the intimidating impression on the customer. If the customer is a woman, the intimidating impression on the customer can be reduced by selecting voice processing that reduces loud voices.
The user may select voice processing at any time during the call between the user and the customer. Also, the user may be configured to select voice processing in advance before starting a call.
Specifically, the user operates the input device 206 of the user terminal 20 to transmit to the server 10 a request including an audio processing ID related to the desired audio processing content. The voice conversion unit 1042 of the server 10 searches the voice processing ID item of the voice processing table 1015 based on the voice processing ID included in the received request, and acquires the voice processing content. That is, the voice conversion unit 1042 of the server 10 specifies and selects voice processing content based on the selection instruction received from the user.

FIG. 15 shows an example of the display screen of the user terminal 20 in the voice conversion process (third embodiment). A call screen 80 is displayed on the display 2081 of the user terminal 20 . The call screen 80 displays customer information 801 currently on the phone and a user interface 802 for selecting voice processing content. The customer information 801 may include customer attribute information about the customer, information on the organization name or organization attribute of the organization to which the customer belongs, and customer emotion information, which are attribute information about the customer. By operating the input device 206 of the user terminal 20 to press the switch 803 associated with the content of the audio processing, the user selects a predetermined audio processing from among the plurality of audio processing. FIG. 15 shows that the audio processing content with the audio processing ID M002 is selected.

<Modification>
Note that the user may operate the input device 206 of the user terminal 20 to select the type of evaluation index to be optimized. Specifically, the user operates the input device 206 of the user terminal 20 to obtain evaluation indices such as SIIB, HASPI, ESTOI, PESQ, and ViSQOL, reliability, credibility, comfort, comfort, preference, stress value, etc. , the degree of intimidation, and the degree of interest may be selected. For example, the user may operate the input device 206 of the user terminal 20 to select an option that makes it easier for the customer to hear (easier to hear) or an option that makes the customer more reliable.
Note that the display 2081 of the user terminal 20 may be configured to present a list of user-selectable evaluation indices to the user. The user may operate the input device 206 of the user terminal 20 to select an item to be optimized from the presented evaluation indices, thereby selecting the type of evaluation index to be optimized.

The user terminal 20 transmits a request including the selected evaluation index type to the server 10 .

The speech conversion unit 1042 of the server 10 selects the speech processing model 1023 based on the type of evaluation index included in the received request. Specifically, the speech conversion unit 1042 of the server 10 selects an optimized (learned) speech processing model 1023 to obtain a larger evaluation index according to the type of the received evaluation index.
At this time, the voice conversion unit 1042 of the server 10 acquires call attributes related to the call, similarly to steps S302 and S303 of the voice conversion process (second embodiment), and performs a plurality of voice processes based on the acquired call attributes. A predetermined audio process is selected from among the above.

Also, the speech conversion unit 1042 of the server 10 may select the generative model 1022 based on the type of evaluation index included in the received request. Specifically, the speech conversion unit 1042 of the server 10 may select the optimized (learned) generation model 1022 to obtain a larger evaluation index according to the received evaluation index type.

That is, in step S503, the voice conversion unit 1042 of the server 10 may directly receive the voice processing ID and specify and select the voice processing content based on the selection instruction received from the user, or may indirectly select the voice processing content. The processing model 1023 may be used to specify an audio processing ID to specify and select audio processing content. Further, the speech conversion unit 1042 of the server 10 may specify and select the generative model 1022 based on a selection instruction received from the user.

In step S504, the voice conversion unit 1042 of the server 10 acquires the call voice from the user and converts the acquired call voice. At this time, the voice conversion unit 1042 of the server 10 converts the acquired call voice based on the voice processing content selected in step S503. Note that the voice conversion unit 1042 of the server 10 may convert the acquired call voice based on the generation model 1022 selected in step S503.
Specifically, the voice conversion unit 1042 of the server 10 sequentially acquires voice data uttered by the user from the voice server (PBX) 40 . After the user speaks, the voice conversion unit 1042 of the server 10 preferably acquires voice data spoken by the user with as little delay as possible.
The voice conversion unit 1042 of the server 10 applies the voice processing content selected in step S503 to the acquired call attribute and voice data, and acquires converted voice data to be output. Also, the speech conversion unit 1042 of the server 10 may apply the generation model 1022 selected in step S503 to the acquired call attributes and speech data, and acquire converted speech data to be output.

In step S505, the voice conversion unit 1042 of the server 10 outputs the call voice converted in step S504 to the customer. Since step S505 is the same as step S105 in the voice conversion process (first embodiment), description thereof is omitted.

<Modification>
In the voice conversion process (first embodiment), the voice conversion process (second embodiment), and the voice conversion process (third embodiment), only the user's uttered voice can be voice-converted. A configuration may be adopted in which voice conversion cannot be performed on the uttered voice.
Specifically, the customer cannot issue a selection instruction for voice processing or the like, and the voice conversion unit 1042 of the server 10 may be configured not to receive selection instructions regarding voice processing from the customer. At this time, the voice conversion unit 1042 of the server 10 may be configured to output the customer's uttered voice to the user without converting it. Specifically, voice data relating to the customer's voice collected by the microphone 5081 of the customer terminal 50 is output from the speaker 2082 of the user terminal 20 without being converted by the voice converter 1042 of the server 10 .
Thereby, the user can talk with the customer while confirming the customer's voice without converting the customer's voice.

<Voice Conversion Processing (Fourth Embodiment)>
The voice conversion process (fourth embodiment) is a process of outputting to the user converted voice data obtained by applying the voice processing model 1023 selected by the user to the voice data uttered by the customer.

<Outline of voice conversion processing (fourth embodiment)>
The voice conversion process (fourth embodiment) is started when the user and the customer are ready to talk. In the voice conversion process (fourth embodiment), the user selects the contents of voice processing, applies the selected voice processing contents to the voice data uttered by the customer, and outputs converted voice data to the user. It is a series of processing to output.

<Details of voice conversion processing (fourth embodiment)>
When the user and the customer are ready to talk, the voice conversion process (fourth embodiment) is started.
Predetermined audio processing is selected from a plurality of audio processing based on a selection instruction received from a user. For example, among a plurality of audio processes, a predetermined audio process that makes the audio easier to hear for the user may be selected. For example, among a plurality of audio processing, a predetermined audio processing that provides more comfort to the user may be selected. For example, among a plurality of audio processes, a user's favorite audio, a predetermined audio process that reduces the stress value, and increases interest may be selected. For example, when a customer is angry, the user can reduce the psychological stress associated with dealing with the customer by selecting voice processing that reduces the inflection or voice from a plurality of voice processing. can be reduced.
The user may select voice processing at any time during the call between the user and the customer. Also, the user may be configured to select voice processing in advance before starting a call.
Specifically, the user operates the input device 206 of the user terminal 20 to transmit to the server 10 a request including an audio processing ID related to the desired audio processing content. The voice conversion unit 1042 of the server 10 searches the voice processing ID item of the voice processing table 1015 based on the voice processing ID included in the received request, and acquires the voice processing content. That is, the voice conversion unit 1042 of the server 10 specifies and selects voice processing content based on the selection instruction received from the user.

<Modification>
Note that the user may operate the input device 206 of the user terminal 20 to select the type of evaluation index to be optimized. Specifically, the user operates the input device 206 of the user terminal 20 to obtain evaluation indices such as SIIB, HASPI, ESTOI, PESQ, and ViSQOL, reliability, credibility, comfort, comfort, preference, stress value, etc. , the degree of intimidation, and the degree of interest may be selected. For example, the user may operate the input device 206 of the user terminal 20 to select an option that makes it easier for the user to hear (easier to hear) or an option that makes the user more comfortable.
Note that the display 2081 of the user terminal 20 may be configured to present a list of user-selectable evaluation indices to the user. The user may operate the input device 206 of the user terminal 20 to select an item to be optimized from the presented evaluation indices, thereby selecting the type of evaluation index to be optimized.
The user terminal 20 transmits a request including the selected evaluation index type to the server 10 .
The speech conversion unit 1042 of the server 10 selects the speech processing model 1023 based on the type of evaluation index included in the received request. Specifically, the speech conversion unit 1042 of the server 10 selects an optimized (learned) speech processing model 1023 to obtain a larger evaluation index according to the type of the received evaluation index.
At this time, the voice conversion unit 1042 of the server 10 acquires call attributes related to the call, similarly to steps S302 and S303 of the voice conversion process (second embodiment), and performs a plurality of voice processes based on the acquired call attributes. A predetermined audio process is selected from among the above. At this time, unlike steps S302 and S303 of the voice conversion process (second embodiment), the attribute information regarding the user and the attribute information regarding the customer are exchanged and applied as the call attribute. This is because in the voice conversion process (fourth embodiment), the customer is the speaker of the voice data and the user is the listener of the converted voice data.

That is, the voice conversion unit 1042 of the server 10 may directly receive the voice processing ID and specify and select the voice processing content based on the selection instruction received from the user, or may indirectly select the voice processing model 1023. may be used to identify the audio processing ID to identify and select the content of the audio processing.

The voice conversion unit 1042 of the server 10 acquires call voice from the customer and converts the acquired call voice. At this time, the voice conversion unit 1042 of the server 10 converts the acquired call voice based on the selected voice processing content.
Specifically, the voice conversion unit 1042 of the server 10 sequentially acquires voice data uttered by the customer from the voice server (PBX) 40 . After the customer speaks, the voice conversion unit 1042 of the server 10 preferably acquires voice data spoken by the customer with as little delay as possible.
The voice conversion unit 1042 of the server 10 applies the selected voice processing content to the acquired call attribute and voice data, and acquires converted voice data to be output.

The voice conversion unit 1042 of the server 10 outputs the call voice converted in step S504 to the user. Step S505 is the same as step S105 in the voice conversion process (first embodiment) except that the user and the customer are replaced, so the description is omitted.

<Modification>
In steps S104, S304, and S504 of the voice conversion process (first embodiment), voice conversion process (second embodiment), and voice conversion process (third embodiment), respectively, the above-described user and customer are placed in a virtual room called a room. When a call is made in a typical call space, the voice conversion unit 1042 of the server 10 sequentially acquires the voice data uttered by the user received by the server 10 without going through the voice server (PBX) 40. It may be configured. Similarly, the voice conversion unit 1042 of the server 10 may be configured to sequentially acquire the voice data spoken by the customer received by the server 10 without going through the voice server (PBX) 40 .

Similarly, in steps S105, S305, and S505 of voice conversion processing (first embodiment), voice conversion processing (second embodiment), and voice conversion processing (third embodiment), respectively, the above-described user and customer are in the room. When a call is made in a so-called virtual call space, the voice conversion unit 1042 of the server 10 may output the converted voice data to the customer terminal 50 without going through the voice server (PBX) 40. good. That is, the voice server (PBX) 40 is not an essential component in the voice conversion process (first embodiment), voice conversion process (second embodiment), and voice conversion process (third embodiment).

In the voice conversion process (fourth embodiment), when the above-described user and customer make a call in a virtual call space called a room, the voice conversion unit 1042 of the server 10 uses the voice server (PBX) 40 as The voice data uttered by the customer accepted by the server 10 may be sequentially acquired without intervention. Similarly, the voice conversion unit 1042 of the server 10 may be configured to sequentially acquire the voice data uttered by the user received by the server 10 without going through the voice server (PBX) 40 .

Similarly, in the voice conversion process (fourth embodiment), when the above-described user and customer make a call in a virtual call space called a room, the voice conversion unit 1042 of the server 10 uses a voice server (PBX ) 40, the converted voice data may be output to the user terminal 20. FIG. In other words, the voice server (PBX) 40 is not an essential component in the voice conversion process (fourth embodiment).

<Learning processing>
Learning processing of the generative model 1022 and the speech processing model 1023 will be described below. The learning process below relates to the learning process for a specific evaluation index (for example, the first index). Learning processing is performed for each of the plurality of generative models 1022 and speech processing models 1023 .

<Learning processing of generative model 1022>
The learning process of the generative model 1022 is a process of learning the learning parameters of the deep neural network included in the generative model 1022 by deep learning.

<Overview of Learning Processing of Generative Model 1022>
The learning process of the generative model 1022 uses attribute information about users, attribute information about customers, attribute information about calls, and voice data as input data (input vectors) so that converted voice data that provides a larger evaluation index is output. This is processing for learning the learning parameters of the deep neural network included in the generative model 1022 by deep learning.

<Details of the learning process of the generative model 1022>
The learning unit 1051 of the server 10 acquires each item of call attributes and voice data from the learning data set 1031 . The learning unit 1051 of the server 10 uses the call attribute and voice data as input data and applies them while changing the learning parameters of the deep neural network included in the generation model 1022 to generate a plurality of converted voice data.
At this time, the learning unit 1051 of the server 10 includes at least one attribute information regarding the user, attribute information regarding the customer, and attribute information regarding the call as call attributes in the input data, and executes the learning process while excluding the other data. I don't mind. The learning unit 1051 of the server 10 acquires, as call attributes, user attribute, organization name or organization attribute of the organization to which the user belongs, user's emotion information, customer attribute, organization name or organization attribute of the organization to which the customer belongs, customer's emotion Any one of the information, the call category, the type of the caller and receiver, and the emotional information of the call may be included in the input data, and the other data may be excluded from the learning process.
The learning unit 1051 of the server 10 applies the call attribute and voice data as input data to the evaluation model 1021, thereby obtaining an evaluation index on the listener side for each of the plurality of converted voice data. The learning unit 1051 of the server 10 optimizes the learning parameters of the deep neural network included in the generative model 1022 so as to obtain a larger evaluation index.
As a result, it is possible to obtain a generative model 1022 that outputs converted speech data that provides a larger evaluation index by using call attributes and speech data as input data.

The learning unit 1051 of the server 10 may configure the generative model 1022 as an arbitrary learning model such as a GAN (Generative Adversarial Network).

<Learning processing of the speech processing model 1023>
The learning process of the speech processing model 1023 is a process of learning the learning parameters of the deep neural network included in the speech processing model 1023 by deep learning.

<Overview of Learning Processing of Audio Processing Model 1023>
The learning process of the voice processing model 1023 uses call attributes such as user attributes, customer attributes, call categories, and incoming and outgoing call types as input data (input vectors), and outputs voice processing content that can obtain a larger evaluation index. This is processing for learning the learning parameters of the deep neural network included in the speech processing model 1023 by deep learning.
Specifically, the voice processing model 1023 in the learning process of the voice processing model 1023 is a learning model that outputs a voice processing ID in the voice processing table 1015 that provides a larger evaluation index using call attributes as input data (input vector). is.

<Details of the learning process of the speech processing model 1023>
The learning unit 1051 of the server 10 generates a plurality of voice processing data by applying the voice processing contents stored in the voice processing table 1015 to the voice data associated with the call attribute included in the learning data set 1031. Generate.
The learning unit 1051 of the server 10 uses the call attribute as input data and applies the plurality of voice processing data associated with the call attribute to the evaluation model 1021, thereby obtaining an evaluation index for each of the plurality of voice processing data. . The learning unit 1051 of the server 10 optimizes the learning parameters of the deep neural network included in the speech processing model 1023 so as to obtain a speech processing ID that provides a larger evaluation index.
As a result, it is possible to obtain a voice processing model 1023 that outputs a voice processing ID that provides a larger evaluation index using call attributes as input data.

In the learning process of the speech processing model 1023, the input data (input vector) may include speech data in addition to call attributes.

<Computer hardware configuration>
FIG. 16 is a block diagram showing the basic hardware configuration of computer 90. As shown in FIG. The computer 90 includes at least a processor 901, a main storage device 902, an auxiliary storage device 903, and a communication IF 991 (Interface). These are electrically connected to each other by a communication bus 921 .

The processor 901 is hardware for executing an instruction set described in a program. The processor 901 is composed of an arithmetic unit, registers, peripheral circuits, and the like.

The main storage device 902 is for temporarily storing programs and data processed by the programs. For example, it is a volatile memory such as a DRAM (Dynamic Random Access Memory).

Auxiliary storage device 903 is a storage device for storing data and programs. For example, flash memory, HDD (Hard Disc Drive), magneto-optical disk, CD-ROM, DVD-ROM, semiconductor memory, and the like.

The communication IF 991 is an interface for inputting and outputting signals for communicating with other computers via a network using a wired or wireless communication standard.
The network is composed of various mobile communication systems constructed by the Internet, LAN, wireless base stations, and the like. For example, networks include 3G, 4G, and 5G mobile communication systems, LTE (Long Term Evolution), wireless networks (for example, Wi-Fi (registered trademark)) that can be connected to the Internet through predetermined access points, and the like. For wireless connection, communication protocols include, for example, Z-Wave (registered trademark), ZigBee (registered trademark), Bluetooth (registered trademark), and the like. In the case of wired connection, the network includes direct connection using a USB (Universal Serial Bus) cable or the like.

It should be noted that the computer 90 can be virtually realized by distributing all or part of each hardware configuration to a plurality of computers 90 and connecting them to each other via a network. Thus, the computer 90 is a concept that includes not only the computer 90 housed in a single housing or case, but also a virtualized computer system.

<Basic Functional Configuration of Computer 90>
A functional configuration of the computer realized by the basic hardware configuration (FIG. 16) of the computer 90 will be described. The computer includes at least functional units of a control section, a storage section, and a communication section.

Note that the functional units included in the computer 90 can also be implemented by distributing all or part of each functional unit to a plurality of computers 90 interconnected via a network. The computer 90 is a concept that includes not only a single computer 90 but also a virtualized computer system.

The control unit is implemented by the processor 901 reading out various programs stored in the auxiliary storage device 903, developing them in the main storage device 902, and executing processing according to the programs. The control unit can implement functional units that perform various information processing according to the type of program. Thereby, the computer is implemented as an information processing device that performs information processing.

A storage unit is realized by the main storage device 902 and the auxiliary storage device 903 . The storage unit stores data, various programs, and various databases. Also, the processor 901 can secure a storage area corresponding to the storage unit in the main storage device 902 or the auxiliary storage device 903 according to a program. In addition, the control unit can cause the processor 901 to execute addition, update, and deletion processing of data stored in the storage unit according to various programs.

A database refers to a relational database, and is used to manage tabular tables structurally defined by rows and columns, and data sets called masters in association with each other. In a database, a table is called a table, a master is called a column, and a row is called a record. In a relational database, relationships between tables and masters can be set and associated.
Normally, each table and each master has a primary key column for uniquely identifying a record, but setting a primary key to a column is not essential. The control unit can cause the processor 901 to add, delete, and update records in specific tables and masters stored in the storage unit according to various programs.

Note that the database and master in the present disclosure may include any data structure (list, dictionary, associative array, object, etc.) in which information is structurally defined. The data structure also includes data that can be regarded as a data structure by combining data with functions, classes, methods, etc. written in any programming language.

A communication unit is implemented by the communication IF 991 . The communication unit implements a function of communicating with another computer 90 via a network. The communication section can receive information transmitted from another computer 90 and input it to the control section. The control unit can cause the processor 901 to execute information processing on the received information according to various programs. Also, the communication section can transmit information output from the control section to another computer 90 .

<Appendix>
The items described in the above embodiments will be added below.

(Appendix 1)
A program, comprising a processor and a storage unit, for performing a telephone call between a first user and a second user in a computer, the program comprising: a voice acquisition step for acquiring a call voice from the first user in the processor; (S104, S304); a conversion step (S104, S304) of converting the call voice acquired in the voice acquisition step; and an output step (S105, S305) of outputting the call voice converted in the conversion step to the second user; , an attribute acquisition step (S102, S302) for acquiring a call attribute related to the call, and a conversion step of converting the call voice acquired in the voice acquisition step based on the call attribute acquired in the attribute acquisition step. including, program.
As a result, in a call between a plurality of users, the customer can make a call with the user with a more suitable voice according to the call attribute.

(Appendix 2)
The program according to appendix 1, wherein the conversion step is a step of converting the call voice by applying a generative model to the call attribute acquired in the attribute acquisition step and the call voice acquired in the voice acquisition step.
As a result, in a call between a plurality of users, the customer can make a call with the user with a more suitable voice according to the call attribute.

(Appendix 3)
The program causes the processor to execute a selection step (S303) of selecting a predetermined voice process from among a plurality of voice processes based on the call attribute acquired in the attribute acquisition step, and the conversion step is acquired in the voice acquisition step. The program according to appendix 1, wherein the step of converting the call voice by applying the predetermined voice processing selected in the selecting step to the call voice.
As a result, in a call between a plurality of users, the customer can make a call with the user with a more suitable voice according to the call attribute.

(Appendix 4)
3. The program according to appendix 3, wherein the selection step is a step of selecting a predetermined audio process that makes the audio easier to hear for the second user, from among the plurality of audio processes.
As a result, in a call between a plurality of users, the customer can make a call with the user in a more audible voice according to the call attribute.

(Appendix 5)
5. The program according to supplementary note 3 or 4, wherein the selecting step is a step of selecting a predetermined voice processing that is more reliable for the second user from among the plurality of voice processing.
As a result, in a call between a plurality of users, the users can give a trustworthy impression to the customer through the call according to the call attribute.

(Appendix 6)
6. The program according to any one of appendices 3 to 5, wherein the selecting step is a step of selecting a predetermined voice processing (S304, S504) by applying a voice processing model to the call attributes acquired in the attribute acquiring step. .
As a result, in a call between a plurality of users, the customer can make a call with the user with a more suitable voice according to the call attribute.

(Appendix 7)
A program, comprising a processor and a storage unit, for performing a telephone call between a first user and a second user in a computer, the program comprising: a voice acquisition step for acquiring a call voice from the first user in the processor; (S504); a conversion step (S504) of converting the call voice acquired in the voice acquisition step; an output step (S505) of outputting the call voice converted in the conversion step to the second user; a selection step (S503) of selecting a predetermined audio process from among a plurality of audio processes based on the selection instruction, and the conversion step converts the call voice acquired in the voice acquisition step into the call voice selected in the selection step. A program that converts phone call audio by applying predetermined audio processing.
As a result, in a call between a plurality of users, the customer can make a call with the user with a more suitable voice based on the selection instruction from the user.

(Appendix 8)
The program causes the processor to execute an attribute acquisition step of acquiring a call attribute related to the call, and the selection step is a step of receiving a selection instruction of the type of evaluation index from the first user, and the call attribute acquired in the attribute acquisition step and selecting a predetermined audio process from among the plurality of audio processes based on the received evaluation index.
As a result, in a call between a plurality of users, the customer can make a call with the user with a more suitable voice according to the call attribute based on the type of evaluation index selected by the user.

(Appendix 9)
The selection step includes a step of receiving a selection instruction from the first user for selecting a type of evaluation index to be optimized, and a step of optimizing the evaluation index among the plurality of voice processes based on the call attributes obtained in the attribute obtaining step and the received evaluation index. and selecting a predetermined audio processing such that the voice processing is optimized.
As a result, in a call between a plurality of users, the customer can make a call with the user with a more suitable voice according to the call attribute based on the type of evaluation index selected by the user.

(Appendix 10)
The selection step includes a step of receiving an instruction to select a type of evaluation index from the first user, and applying a speech processing model to the call attribute acquired in the attribute acquisition step and the received evaluation index, thereby selecting a plurality of speech processing. and selecting a predetermined audio processing among them.
As a result, in a call between a plurality of users, the customer can make a call with the user with a more suitable voice according to the call attribute based on the type of evaluation index selected by the user.

(Appendix 11)
The voice acquisition step includes a step of acquiring a second call voice from the second user, the selection step cannot accept a selection instruction from the second user, and the conversion step includes the second call voice acquired in the voice acquisition step. 11. The program according to any one of appendices 8 to 10, wherein the step does not convert call voice.
Accordingly, the first user can make a call with the second user while confirming the second user's voice without converting the second user's voice.

(Appendix 12)
The attribute acquisition step includes a step of acquiring attribute information about the second user, and the conversion step includes a step of converting the call voice acquired in the voice acquisition step based on the attribute information related to the second user acquired in the attribute acquisition step. 12. The program of any one of Appendixes 1-6, 8-11, comprising:
As a result, in a call between a plurality of users, the customer can make a call with the user with a more suitable voice according to the customer-related attribute information.

(Appendix 13)
The attribute acquisition step includes a step of acquiring attribute information about the first user, and the conversion step includes a step of converting the call voice acquired in the voice acquisition step based on the attribute information related to the first user acquired in the attribute acquisition step. 13. The program of any one of Appendixes 1-6, 8-12, comprising:
As a result, in a call between a plurality of users, the customer can make a call with the user with a more suitable voice according to the user attribute information.

(Appendix 14)
Supplementary Note 1, wherein the attribute acquisition step includes a step of acquiring attribute information about the call, and the conversion step includes a step of converting the call voice acquired in the voice acquisition step based on the attribute information related to the call acquired in the attribute acquisition step. 6, 8-13.
As a result, in a call between a plurality of users, the customer can make a call with the user with a more suitable voice according to the attribute information regarding the call.

(Appendix 15)
The attribute acquisition step includes a step of acquiring attribute information about the call, and the conversion step is a step of converting the call voice acquired in the voice acquisition step based on the information about the user's or customer's emotion in the call acquired in the attribute acquisition step. 15. The program according to any one of Appendices 1 to 6, 8 to 14, comprising
As a result, in a call between a plurality of users, the customer can make a call with the user with a more suitable voice according to the emotional information of the user or the customer. For example, depending on the user's or customer's emotional state, it is possible to communicate with the user with a more suitable voice.

(Appendix 16)
16. The program according to any one of appendices 1 to 15, wherein the call attributes acquired in the attribute acquisition step do not include information about the surrounding environment of the user and the customer and the call environment.
As a result, in a call between a plurality of users, the customer can make a call with the user with a more suitable voice according to the call attribute.

(Appendix 17)
The conversion step includes a step of converting the speech component of the person out of the call voice acquired in the voice acquisition step, and background noise, noise, noise, etc. other than the voice component of the person out of the call voice acquired in the voice acquisition step. 17. The program according to any of the clauses 1-16, which does not include the step of converting the audio component of the .
As a result, in a call between a plurality of users, the customer can make a call with the user with a more suitable voice according to the call attribute.

(Appendix 18)
The program causes the processor to execute a second selection step of selecting the second audio processing from among the plurality of audio processing based on a second selection instruction received from the first user, and the audio acquisition step is performed by the second user. wherein the converting step applies the second speech processing selected in the second selecting step to the second speech speech obtained in the obtaining step to obtain the second speech speech from 18. The program according to any one of appendices 1 to 17, wherein the step of converting includes the step of outputting the second call voice converted in the converting step to the first user.
As a result, for example, it is possible to make a call with a customer using a voice that is easier for the user to hear, among a plurality of voice processes. For example, among a plurality of voice processes, it is possible to make a call with a customer using a predetermined voice that is more comfortable for the user. For example, among a plurality of voice processes, it is possible to make a call with a customer using a voice that is preferred by the user, a predetermined voice that reduces the stress value, and increases interest. For example, when a customer is angry, the user can reduce the psychological stress associated with dealing with the customer by selecting voice processing that reduces the inflection or voice from multiple voice processing. can be reduced.

(Appendix 19)
An information processing apparatus comprising a processor and a storage unit, wherein the processor includes a voice acquisition step (S104, S304) for acquiring call voice from a first user, and a conversion step for converting the call voice acquired in the voice acquisition step. (S104, S304), an output step (S105, S305) of outputting the call voice converted in the conversion step to the second user, and an attribute acquisition step (S102, S302) of acquiring call attributes related to the call. and the conversion step includes a step of converting the call voice acquired in the voice acquisition step based on the call attribute acquired in the attribute acquisition step.
As a result, in a call between a plurality of users, the customer can make a call with the user with a more suitable voice according to the call attribute.

(Appendix 20)
An information processing method executed by a computer comprising a processor and a storage unit, wherein the processor is provided with a voice acquisition step (S104, S304) for acquiring a call voice from a first user, and the call voice acquired in the voice acquisition step. an output step (S105, S305) of outputting the call voice converted in the conversion step to the second user, and an attribute acquisition step of acquiring call attributes related to the call (S102, S302 ), and the conversion step includes a step of converting the call voice acquired in the voice acquisition step based on the call attribute acquired in the attribute acquisition step.
As a result, in a call between a plurality of users, the customer can make a call with the user with a more suitable voice according to the call attribute.

1 information processing system, 10 server, 101 storage unit, 103 control unit, 20A, 20B, 20C user terminal, 201 storage unit, 204 control unit, 30 CRM system, 301 storage unit, 304 control unit, 50A, 50B, 50C customer terminal, 501 storage unit, 504 control unit

Claims (20)

A program comprising a processor and a storage unit, and carrying out a call between a first user and a second user in a computer,
The program causes the processor to:
a voice acquisition step of acquiring call voice from the first user;
a conversion step of converting the call voice acquired in the voice acquisition step;
an output step of outputting the call voice converted in the conversion step to a second user;
an attribute acquisition step of acquiring call attributes related to the call;
and
The conversion step includes a step of converting the call voice acquired in the voice acquisition step based on the call attribute acquired in the attribute acquisition step,
program. The conversion step is a step of converting the call voice by applying a generative model to the call attribute acquired in the attribute acquisition step and the call voice acquired in the voice acquisition step,
A program according to claim 1. The program causes the processor to:
a selection step of selecting a predetermined voice process from among a plurality of voice processes based on the call attribute acquired in the attribute acquisition step;
and
The conversion step is a step of converting the call voice by applying the predetermined voice processing selected in the selection step to the call voice acquired in the voice acquisition step.
A program according to claim 1. The selecting step is a step of selecting a predetermined audio process that makes the audio easier to hear for the second user, from among the plurality of audio processes.
4. A program according to claim 3. The selecting step is a step of selecting a predetermined voice processing that is more reliable for the second user from among the plurality of voice processing.
5. A program according to claim 3 or 4. The selecting step is a step of selecting the predetermined voice processing by applying a voice processing model to the call attribute acquired in the attribute acquiring step.
A program according to any one of claims 3 to 5. A program comprising a processor and a storage unit, and carrying out a call between a first user and a second user in a computer,
The program causes the processor to:
a voice acquisition step of acquiring call voice from the first user;
a conversion step of converting the call voice acquired in the voice acquisition step;
an output step of outputting the call voice converted in the conversion step to a second user;
a selection step of selecting a predetermined audio process from among a plurality of audio processes based on a selection instruction received from a first user;
and
The conversion step is a step of converting the call voice by applying the predetermined voice processing selected in the selection step to the call voice acquired in the voice acquisition step.
program. The program causes the processor to:
an attribute acquisition step of acquiring call attributes related to the call;
and
The selection step includes:
a step of receiving a selection instruction of a type of evaluation index from a first user;
a step of selecting the predetermined voice processing from among the plurality of voice processing based on the call attribute obtained in the attribute obtaining step and the received evaluation index;
including,
8. A program according to claim 7. The selection step includes:
a step of receiving a selection instruction from a first user for a type of evaluation index to be optimized;
a step of selecting a predetermined voice processing that optimizes the evaluation index from among a plurality of voice processing based on the call attribute obtained in the attribute obtaining step and the received evaluation index;
including,
9. A program according to claim 8. The selection step includes:
a step of receiving a selection instruction of a type of evaluation index from a first user;
selecting the predetermined voice processing from among the plurality of voice processing by applying a voice processing model to the call attribute acquired in the attribute acquisition step and the received evaluation index;
including,
10. A program according to claim 8 or 9. said acquiring voice includes acquiring a second call voice from a second user;
The selection step cannot receive a selection instruction from a second user,
The converting step is a step of not converting the second call voice acquired in the voice acquiring step,
A program according to any one of claims 8 to 10. The attribute obtaining step includes obtaining attribute information about the second user,
The conversion step includes a step of converting the call voice acquired in the voice acquisition step based on the attribute information about the second user acquired in the attribute acquisition step,
12. The program according to any one of claims 1 to 6 and 8 to 11. The attribute obtaining step includes obtaining attribute information about the first user,
The conversion step includes a step of converting the call voice acquired in the voice acquisition step based on the attribute information about the first user acquired in the attribute acquisition step,
13. The program according to any one of claims 1 to 6 and 8 to 12. the attribute obtaining step includes obtaining attribute information about the call;
The conversion step includes a step of converting the call voice acquired in the voice acquisition step based on the attribute information related to the call acquired in the attribute acquisition step,
14. The program according to any one of claims 1 to 6 and 8 to 13. the attribute obtaining step includes obtaining attribute information about the call;
The conversion step includes a step of converting the call voice acquired in the voice acquisition step based on the information about the user's or customer's emotion in the call acquired in the attribute acquisition step,
15. The program according to any one of claims 1 to 6 and 8 to 14. The call attributes acquired in the attribute acquisition step do not include information about the surrounding environment of the user and the customer, and the call environment.
16. The program according to any one of claims 1 to 6 and 8 to 15. The conversion step includes:
including a step of converting a voice component of a person out of the call voice acquired in the voice acquisition step;
Not including a step of converting audio components such as background noise, noise, and noise other than human voice components from the call voice acquired in the voice acquisition step;
A program according to any one of claims 1 to 16. The program causes the processor to:
a second selection step of selecting a second audio process from among a plurality of audio processes based on a second selection instruction received from a first user;
and
said acquiring voice includes acquiring a second call voice from a second user;
the converting step includes converting the second speech speech by applying the second speech processing selected in the second selecting step to the second speech speech obtained in the obtaining step;
The output step includes outputting the second call voice converted in the conversion step to the first user,
A program according to any one of claims 1 to 17. An information processing device comprising a processor and a storage unit,
to the processor;
a voice acquisition step of acquiring call voice from the first user;
a conversion step of converting the call voice acquired in the voice acquisition step;
an output step of outputting the call voice converted in the conversion step to a second user;
an attribute acquisition step of acquiring call attributes related to the call;
and
The conversion step includes a step of converting the call voice acquired in the voice acquisition step based on the call attribute acquired in the attribute acquisition step,
Information processing equipment. An information processing method executed by a computer comprising a processor and a storage unit,
to the processor;
a voice acquisition step of acquiring call voice from the first user;
a conversion step of converting the call voice acquired in the voice acquisition step;
an output step of outputting the call voice converted in the conversion step to a second user;
an attribute acquisition step of acquiring call attributes related to the call;
and
The conversion step includes a step of converting the call voice acquired in the voice acquisition step based on the call attribute acquired in the attribute acquisition step,
Information processing methods.