KR20220136846A - Method of feedback salesman by analyzing the sounds or face image of both themselves and client and the apparatus thereof - Google Patents

Abstract

The present invention relates to a content production and training system for sales staff training. More specifically, the present invention relates to a content production and training system for sales staff training. The system comprises: a step (S301) of receiving data from a terminal (110) while sales work is being performed; a step (S302) of extracting sound and image data from the received data; a step (S303) of analyzing the extracted sound and images; a step (S304) of generating data for content for sales staff training from the analyzed data; and a step (S305) of processing and outputting content necessary for providing content for sales staff training, wherein each step is performed by a service server (120). The step (S301) of receiving data from the terminal (110) while sales work is being performed is to receive basic information, individual information, voice or face screens of a user and a customer. According to the present invention, it is possible to produce and provide content for sales staff training by collecting verbal and/or non-verbal elements occurring in the field where actual sales work is being performed.

Description

Method of feedback salesman by analyzing the sounds or face image of both themselves and client and the apparatus thereof

The present invention relates to a sales staff business management and training system.

Airline staff such as flight attendants, hospital coordinators, service instructors, etc., who perform sales duties such as service jobs requiring customer service (hereinafter referred to as "sales staff" in the present invention), serve customers before serving customers. It is necessary to manage when, where, when, and with what content, according to the flow of time, before, during, and after serving customers, such as after serving customers.

In particular, sales are often conducted verbally, so even if a memo is left, it is not recorded 100% accurately unless it is recorded through recording or recording.

Therefore, it is good for the sales staff to record or record the sales contents in the process of dealing with customers. The need for business management is required to determine what kind of content came and went during the performance of sales duties, even after recording or recording, and how to proceed after that.

In addition, even if the recorded sound or recorded video is viewed after the sales job is performed, it is cumbersome, such as having to spend a lot of time to organize it, and the work efficiency is lowered due to factors other than the sales job such as a waste of time. is occurring

On the other hand, for sales staff, not only linguistic elements such as words, intonation, and stress, but also so-called non-verbal elements such as smiles, postures, or speaking appearances are very important in their job performance to make customers feel comfortable. are considered

Therefore, there are private academies that educate people to understand and learn sales jobs. Looking at the recent trends in the service industry and tourism industry, it is expected that the size of the workforce in these specialized service industries will gradually expand.

Therefore, we train sales staff with a lot of money and time, and provide various textbooks and programs to identify and improve the verbal and non-verbal factors of sales staff.

However, since sales staff individually deal with customers and perform their duties based on their own verbal and non-verbal factors, the results are different for each sales employee, and even customers act as individual variables. There was a problem that even other sales staff could not be trained with the uniform manual and program implemented by the company.

In particular, these standardized manuals and programs were difficult to provide education like in the actual sales field, as it was difficult to understand the sound of the actual sales job site, read and view textual textbooks, or just play a role. In the actual field, we had to rely on the unique elements of each employee.

Accordingly, it is required to provide training and education contents based on the so-called successful sales job performance method by storing and analyzing verbal and non-verbal elements in the actual field as they are.

On the other hand, it will be important to give feedback on their sales to the sales staff themselves, and to self-analyze in which cases the sales will succeed and in which cases the sales will fail. Alternatively, a system for providing feedback based on a recorded image or the like is required.

As described above, the present invention aims to solve the following problems.

Its purpose is to provide a business management system that eliminates the need for separate arrangement after sales through recording or recording of sales content so that sales employees can manage their business so that they can be faithful to their sales duties.

In addition, an object of the present invention is to provide a sales staff training system that generates and provides content for training sales staff by collecting verbal and/or non-verbal elements generated in the field performing actual sales duties.

In addition, an object of the present invention is to provide a sales staff training system capable of self-evaluation, such as being able to give feedback to sales staff themselves and predicting sales results.

Each of the following steps is performed by the service server 120, and receiving data from the terminal 110 while performing a sales job (S301); extracting sound and image data from the received data (S302); analyzing the extracted audio and video (S303); Generating data for sales staff work management and training from the analyzed data (S304); Including; processing and outputting content necessary for providing content for sales staff training (S305), the step of processing and outputting content necessary for providing content for sales staff training (S305) is at a certain point in time A sales staff business management and training system, characterized in that by converting and generating recorded audio data or recorded video data from to a certain end point to text data, can be one means.

Preferably, the step (S305) of processing and outputting content necessary for providing content for sales staff training is such that the audio or video data can be generated so that it can be reproduced in parallel in chronological order together with the text data, Sales staff work management and training system can be a means of work.

Also preferably, the step (S305) of processing and outputting content necessary for providing content for sales staff training includes analyzing the voice or face image of a customer or sales employee and giving feedback to the corresponding employee, Feedback provides voice data as text data in the form of a script, and links the analysis result of voice data or face image with the text data. The result of analysis of voice data or facial image of the customer is provided together, and the customer's negative, neutral or positive state is derived from the customer's voice or face image. It is characterized in that, based on the derived state of the customer, characterized in that for predicting the sales results, the sales staff work management and training system can be one means.

However, the means for solving the problems of the present invention are not limited thereto.

According to the above solution means, the present invention can achieve the following effects.

It is possible to provide a business management system that does not require separate arrangement after sales through recording or recording of sales content so that sales employees can manage their business so that they can be faithful to their sales duties.

In addition, it is possible to provide a sales staff training system that generates and provides content for training sales staff by collecting verbal and/or non-verbal elements generated in the field performing actual sales duties.

In addition, the present invention can provide a sales staff job training system capable of self-evaluation, such as giving feedback to the sales staff themselves and predicting sales results.

1 is a block schematic diagram of a training and content creation system for employee education of the present invention.
2 is a block diagram of a service server 120 according to an embodiment of the present invention.
3 is a flowchart illustrating a method for managing sales staff work and providing training content according to an embodiment of the present invention.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings. DETAILED DESCRIPTION The detailed description set forth below in conjunction with the appended drawings is intended to describe exemplary embodiments of the present invention and is not intended to represent the only embodiments in which the present invention may be practiced.

Only the present embodiments are provided so that the disclosure of the present invention is complete, and the scope of the invention to those of ordinary skill in the art to which the present invention belongs, is provided, and the present invention is to be defined by the scope of the claims. only

In some cases, in order to avoid obscuring the concept of the present invention, well-known structures and devices may be omitted or shown in block diagram form focusing on core functions of each structure and device. In addition, the same reference numerals are used to describe the same components throughout this specification.

Throughout the specification, when a part is said to "comprising or including" a certain component, it does not exclude other components unless otherwise stated, meaning that other components may be further included. do.

In addition, the term “…unit” described in the specification means a unit that processes at least one function or operation, which may be implemented as hardware or software or a combination of hardware and software. Furthermore, "a or an", "one", and like related terms in the context of describing the present invention are used in both the singular and the plural unless otherwise indicated herein or otherwise clearly contradicted by the context. may be used in a sense including

In addition, specific terms used in the embodiments of the present invention are provided to help the understanding of the present invention, and unless otherwise defined, all terms used herein, including technical or scientific terms, refer to the present invention. It has the same meaning as commonly understood by those of ordinary skill in the art to which it belongs. The use of these specific terms may be changed to other forms without departing from the technical spirit of the present invention.

Meanwhile, the system according to the present invention can be written as a program that can be executed on a computer, and can be implemented in a general-purpose digital computer that operates the program using a computer-readable medium. In addition, the structure of the data used in the above method may be recorded in a computer-readable medium through various means. A computer readable medium storing executable computer code for performing various methods of the present invention includes a magnetic storage medium (eg, ROM, floppy disk, hard disk, etc.), an optically readable medium (eg, CD-ROM, DVD). storage media, such as).

Those of ordinary skill in the art related to the embodiments of the present invention will understand that it may be implemented in a modified form within a range that does not depart from the essential characteristics of the description. Therefore, the disclosed methods are to be considered in an illustrative and not a restrictive sense. The scope of the present invention is indicated in the claims rather than in the detailed description of the invention, and all differences within the scope equivalent thereto should be construed as being included in the scope of the present invention.

1 is a schematic diagram of a sales staff work management and training system according to an embodiment of the present invention;

Referring to FIG. 1 , the sales staff work management and training system 100 is configured to include a terminal 110 and a service server 120 , but the present invention is not limited thereto. It may be implemented by further including the above components.

On the other hand, communication or data communication (hereinafter 'data communication') between the components belonging to the training and content production system 100 for employee education is made through a wired/wireless communication network, and a communication protocol between each component They don't all have to be the same.

The terminal 110 is a device for an employee performing a sales job to use a training and content production system for employee education, and the employee transmits various information and data for the system from the service server 120 to the service server 120 . ) can be transmitted.

The terminal 110 may receive various information and data for training and content production system for employee education, including video and audio input/output devices, and may output various information and data received from the service server 120 . have.

The terminal 110 may support or provide a training and content production system for employee education to one user through a plurality of terminals. For example, the first terminal may receive various information and data for the system, and the second terminal may output various information and data received from the service server 120 .

As described above, the first terminal receives audio and/or video signals on sales activities performed by sales staff, and includes fixed terminals such as TVs, PCs, and digital signage, smartphones, tablet PCs, and notebook computers. At least one of mobile terminals such as

As described above, the second terminal outputs the data processed by the service server 120 in the training and content production system for employee education, for example, a fixed terminal such as TV, PC, digital signage, At least one of a mobile terminal such as a smartphone, a tablet PC, and a notebook computer is sufficient.

In addition, although only one terminal 110 is illustrated in FIG. 1 , according to an embodiment, n (here, n is a positive integer) number of terminals are simultaneously (simultaneous) or at the same time (異time) service server 120 and data communication may be performed, and each terminal may provide a service for each user.

On the other hand, one first terminal and a plurality of second terminals may constitute the terminal 110, and in this case, the first terminal performs employee education for a plurality of users at the same time with respect to at least one content for employee education. It is also possible to provide guide data for providing a training service for

According to an embodiment, the terminal 110 may provide training and content creation services for employee education through the execution of an application. In this case, the software such as the application or program is downloaded from the service server 120 in advance. can be installed

Meanwhile, the terminal 110 may include a related application program interface (API) or embedded software for the above-described service.

In other words, hereinafter, the sales staff work management and training system 100 according to the present invention is made through an application including an API or embedded software pre-installed in the terminal 110 as an embodiment. However, the present invention is not limited thereto, and may be provided in the form of a web service.

Meanwhile, the terminal 110 may transmit or receive data or a signal (including it) to the service server 120 by itself, and may also perform necessary processing such as data conversion for this. In addition,

The terminal 110 may be assisted by or through one or more devices when necessary to transmit data to or receive data from the service server 120 .

Next, the service server 120 is an entity that performs data communication with at least one or more other devices, that is, the terminal 110 through a wired/wireless communication network, and in order to provide a service according to the present invention, the terminal 110 ) and data communication, processing data through it, and generating, storing, and transmitting necessary data, a device that performs at least one function, and may include a program or software for it.

Although one service server 120 is illustrated in FIG. 1 , the present invention is not limited thereto, and a plurality of service servers 120 may be included in the service system 100 , for data processing for employee education and training. It may further include one or more servers, such as a client server.

Meanwhile, the service server 120 according to the present invention includes at least one function of collecting, processing, storing data/big data necessary for providing a service for employee education and training, including data received through the terminal 110 . A database (DB: database) that performs There may be a plurality of such databases.

Service server 120, cloud (Cloud) server, IMS (IP Multimedia Subsystem) server, telephony application (Telephony Application) server, IM (Instant Messaging) server, MGCF (Media Gateway Control Function) server, MSG (Messaging Gateway) At least one of a server and a Call Session Control Function (CSCF) server may be used as an embodiment.

The wired/wireless communication network means a data communication network that supports data communication of data in various formats, such as text, image, and voice, between the terminal 110 and the service server 120, and the type is not particularly limited. .

For example, the wired/wireless communication network may be an IP (Internet Protocol) network supporting large-capacity data communication through the Internet Protocol (IP) or an All IP network integrating different IP networks.

In addition, the wired / wireless communication network includes a wired network, a Wibro (Wireless Broadband) network, a mobile communication network including WCDMA, a High Speed Downlink Packet Access (HSDPA) network, and a Long Term Evolution (LTE) network including a mobile communication network, LTE advanced (LTE-A) including a mobile communication network, satellite communication network, and Wi-Fi (Wi-Fi) network, or may be formed by a combination of at least one or more of them.

Meanwhile, in FIG. 1 , the first terminal and the second terminal may also perform data communication based on a wired/wireless communication protocol. For example, the first terminal and the second terminal may be paired with each other through Bluetooth or may exchange data through Wi-Fi. The data may also include control data for performing a specific function or operation.

2 is a block diagram of a service server 120 according to an embodiment of the present invention.

FIG. 2 shows an embodiment of a configuration block diagram of the service server 120, and shows only essential components, but the present invention is not necessarily limited thereto.

According to an embodiment, some components may be added or removed in FIG. 2 , and a plurality of components may be modularized and implemented as one configuration or vice versa.

In addition, according to an embodiment, some components may be removed from FIG. 2 and a corresponding function may be performed by other components. For example, according to an embodiment, FIG. 2 may be viewed as a component of the terminal 110 .

The service server 120 includes a communication unit 201 , a data extraction unit 202 , a data analysis unit 203 , a data generation unit 204 , a content processing unit 205 , a control unit 206 , and a database (DB) 207 . ) may be included.

The communication unit 201 supports an interface for data communication with the terminal 110 , and receives and transmits data from the terminal 110 .

The data extraction unit 202 extracts sound and image data from the received data.

The data analysis unit 203 analyzes necessary data and unnecessary data from the extracted audio and video data.

The data generation unit 204 generates data for content for sales staff education and training from the analyzed necessary data and unnecessary data.

The content processing unit 205 processes content or a content list required to provide content for training sales staff of the terminal 110 .

The control unit 206 performs overall function control of the service server 120 including the above-described components.

The database 207 is a configuration for temporarily storing data received from the terminal 110 , and may be implemented inside or outside the service server 120 , depending on the embodiment, or may be plural.

However, here, each description of the components is a description of a schematic function, and a detailed description of each component may be added throughout the description of the present invention.

3 is a flowchart illustrating a sales staff work management and training system according to an embodiment of the present invention.

Referring to FIG. 3 , a sales staff work management and training system is described from the perspective of the service server 120 .

The service server 120 receives data while performing a sales job from the terminal 110 (S301).

The data extraction unit 202 extracts sound and image data from the received data (S302).

The data analysis unit 203 analyzes necessary data and unnecessary data from the extracted audio and video data (S303).

The data generation unit 204 generates data for content for sales staff education and training from the analyzed necessary data and unnecessary data (S304).

The content processing unit 205 processes and outputs content or a content list required to provide content for training the sales staff of the terminal 110 (S305).

In parallel with each of the steps listed above, the control unit 206 controls each component according to the present invention, and the database 207 stores all data received, generated, processed, output by the service server 120, and the like. And even if not mentioned otherwise, it can be said to be obvious to those with ordinary knowledge.

In addition, the functions or roles performed by each component may be distinguished, but in the following description, the functions or roles performed by each component may be described overlapping each other, and it is obvious to those of ordinary skill in the art to understand the present invention. something to do.

The step ( S301 ) of the service server 120 receiving data while performing a sales job from the terminal 110 will be described in detail.

The data received from the terminal 110 while performing a sales job may include basic information. In this case, the basic information may include user personal information and identification information of the terminal 110 .

The user personal information may include physical information such as gender, age, height, and weight, appearance, voice, attire, personality, hobbies, specialties, dispositions, habits, etc. of the sales staff as a whole.

The basic information is transmitted to the service server 120 according to a user's input or selection according to an item provided during the service application execution process being executed in the terminal 110, but is not necessarily limited thereto, and includes voice, gesture, image It may be transmitted in the form of or a combination thereof.

In addition, the data received from the terminal 110 while performing a sales job may be a sound and an image.

Here, the sound or image refers to a recording or recording while the user interacts with the customer while performing the sales job, but is not limited thereto, and includes all sounds or images that may be necessary or helpful in the performance of the sales job.

The sound is, for example, recorded from a certain point in time selected by a user performing a sales job while performing his/her duties, and may include a recording prior to customer service, a recording during customer service, and a recording after customer service.

Recordings before customer service refer to recordings that the user checks and uses internally before responding to customers. It refers to the recording for the user to check and use internally after the customer service is finished. However, this is only a division for the purpose of explaining the present invention, and if the entire recording is viewed separately, the reference point of the division may be different.

The image is, for example, recorded from a certain point in time selected by a user performing a sales job while performing his/her duties, and may include recordings before customer service, recording during customer service, and recording after customer service, which This is merely a division for the purpose of explaining the present invention, and if the entire recording is viewed separately, the reference point of the division may be different.

Each recording item has the same description as the above-described recording item and is omitted here, but a more detailed description of the sound or image will be described later.

In addition, the sales job performance data received from the terminal 110 may include the date and time and place where the sales job was performed, the number of attendees, customer information, sales purpose, language, and the like.

Since the above-listed date and time of execution, place of execution, number of attendees, customer information and business purpose, language, etc., are different individual information for each customer that is different from the basic information, it is called 'individual information' to be distinguished from the basic information. .

The basic information and the individual information are transmitted to the service server 120 according to the user's input or selection according to the items provided during the service application execution process being executed in the terminal 110, or automatically at the time of recording or recording start. The terminal 110 may determine the date and time and place and transmit it to the service server 120, but is not limited thereto.

In addition, the basic information and individual information are tagged at the stage when the recorded sound or recorded image is input, received, analyzed, processed, output, stored, etc. according to the present invention, so that the subject or date and time of the sound or image is , location, etc.

Meanwhile, the above-described recording or recording will be described in more detail.

Users may obtain prior consent prior to recording or recording when interacting with customers, and prior consent may be written in writing, but recording or recording is also possible. It includes a process of recording or recording the process, or explaining the process with video and audio input to the terminal 110 in advance, and the customer answers or electronically signs the process.

In addition, prior consent may be omitted in the case of a customer who has already obtained consent. For example, if you record or record each time for customers you meet more than twice, there is a cumbersome procedure to obtain consent each time. It may be decided not to proceed with the consent acquisition procedure, or a simplified procedure may be provided to obtain prior consent through sound or video.

For example, in the prior consent process, there may be a process of analyzing the voice print of the user and the customer or obtaining prior information such as a face image.

For example, it is not necessary to specifically inquire whether or not the informed consent procedure has already been obtained, and it can also be performed through glottal analysis or image analysis, which will be described later. More specifically, for customers who will come into contact with you more than once, if you have already accepted the prior consent for recording or recording in all future sales activities when you have met in the past, you may be asked to write a letter in the process of initiating recording or recording. Through analysis or image analysis, it is possible to record or record as it is without the need to go through a separate procedure for customers who have accepted prior consent in the past.

However, if the customer's voiceprint analysis or image analysis has not been consented to in advance, it can be recorded or not recorded any more without consent on the spot. Therefore, it is necessary to obtain a glottis or an image for the glottal analysis or image analysis or the like.

For example, by repeating the voices of the user and the customer one or more times to be input to the terminal 110, the terminal 110 allows the voices of the user and the customer to be recorded before a specific sales process is performed, so as to analyze the voiceprint. Basic information can be obtained.

Also, for example, by repeating the faces of the user and the customer one or more times and inputting them into the terminal 110, the terminal 110 records the faces of the user and the customer before a specific sales process is performed to perform image analysis. information can be obtained for On the other hand, information obtained in advance in the process of acquiring a voice letter or image for prior consent is not limited to voice or voice, but is not limited to voices or sounds, It is also possible to obtain a background image of a user or customer's overall appearance or business location).

In order to obtain such prior information, the terminal 110 may provide an interface for the user to obtain prior information.

In addition, the terminal 110 may provide a basic interface such as start, end, pause, etc. in the course of recording or recording.

In addition, the interface may be provided with a bookmark function to emphasize important content when exchanging with the customer or to perform a function such as a bookmark.

For example, when providing an interface such as recording start, recording end, and pause, the terminal 110 may provide a physical button, or it may be a soft key type button output through the display unit of the terminal 110 . Also, it is possible to provide a start, end, highlight, or bookmark function by means of a pre-recorded or recorded specific sound, expression, or gesture without physical contact.

In order to obtain the sound or image, the terminal 110 may include one or more input devices, preferably, two or more microphones and two or more cameras.

In the case of a single microphone or a single camera, it is not possible to simultaneously acquire a salesperson's voice print, voice and face image, customer's voice print, or voice and face image. because it can

The salesperson may allow the recording or recording to stop after the customer service has been completed, or it may continue to record or maintain the recording.

Even when recording or recording is stopped, a separate interface may be provided to provide an interface that allows sales staff to input information after responding to customers, and if recording or recording is continued, voice or gestures, etc. may be provided. Through this, information can be entered after responding to the customer.

Above, a scenario in which a sales employee performs a process before, during, and after serving a customer using the terminal 110 performing the system according to the present invention can be seen as follows.

[Scenario #1]

The salesperson enters the customer's basic and personal information before serving the customer.

The sales staff proceeds with the procedure for obtaining prior consent for recording or recording for customers who have not obtained prior consent, even if they are customers they meet for the first time or customers they meet twice or more. If there is one, record or record as it is.

At the same time, the terminal 110 acquires the customer's voiceprint or image, and if prior consent has already been obtained using the acquired voiceprint or image, the recording or recording proceeds as it is, and if the customer is not already acquired, the recording or recording is performed. Recording or recording can be performed only after stopping and obtaining the customer's prior consent.

While the salesperson is interacting with the customer, the conversation between the salesperson and the customer and the scene may be recorded or recorded during the response process. can be

Sales staff can enter information directly after interacting with customers by stopping recording or recording, or continuing recording or recording and entering information after interacting with customers by voice or gesture.

The data extraction unit 202 extracts sound and image data from the received data (S302), basic information and individual information among the received data, and the recorded sound and recorded sound while performing sales duties in response to customers The image is extracted, and the data analysis unit 203 analyzes the extracted data (S303), and the extraction step (S302) and the analysis step (S303) by the data extraction unit 202 are described in detail together.

First, user basic information and individual information may be utilized by forming a database.

For example, basic information and individual information obtained from a plurality of sales staff may be compared with each other and classified and analyzed into a group based on the same or similar points.

For example, even if sales staff and customers are different, they can be classified and analyzed into a group based on the same and similar points despite such differences.

Here, the plurality of sales staff may be information obtained from sales staff having completely different sales fields, or information obtained from employees belonging to a department having the same or similar sales field, but is not limited thereto.

As an example, basic information or individual information obtained in the course of performing a sales job by the same person may be accumulated and classified and analyzed into a group based on the same or similarity among the accumulated information.

Classification and/or analysis of a group compares the obtained basic information or individual information with each other, and if the basic information of the user is the same or similar, if the customer information is the same or similar, the location or date of provision is the same or similar Analyze to form one category for each information such as case.

Based on information obtained from multiple sales staff through this, characteristics of the same or similar sales staff, characteristics of the same or similar customers, and multiple sales staff performing sales duties at the same or similar place or time, etc. It can be used as data for training sales staff by categorizing and analyzing the same or similar points in spite of different situations such as characteristics.

In addition, by comparing the information accumulated in the process of contacting different customers in the process of performing duties of the same sales staff, if there are any similarities or similarities, what characteristics are there, and the results of the user's sales job performance are analyzed or future sales of the same person are compared. You can enable employees to receive feedback on the progress of their sales job performance.

Meanwhile, the data analyzer 203 may analyze the sound or image extracted by the data extractor 202 as the sound or recorded image recorded by the terminal 110 .

According to an exemplary embodiment, when analyzing the recorded sound, the sound obtained in advance is analyzed so that the speaker and the listener can be distinguished from the recorded sound in order to analyze the voiceprints of the sales staff and the customer.

Each of the speaker and the listener may be at least one or more, and in this case, each glottal analysis may be performed.

The glottal analysis may be performed based on the loudness, tone, frequency (pitch), speed of speech, and characteristics of pronunciation, but is not limited thereto.

According to an embodiment, when analyzing the recorded image, the face obtained in advance is analyzed so that the speaker and the listener can be distinguished from the recorded image in order to analyze the faces of the sales staff and the customer.

Each of the speaker and the listener may be at least one or more, and in this case, the position, size, proportion, hairstyle, etc. of each feature may be used as a criterion, but the present invention is not limited thereto.

It is possible to analyze and remove non-speech noise from the recorded sound or recorded video.

Here, the noise may be a sound other than the speaker's and listener's voices, for example, it may be a background sound input from the terminal 110 excluding the speaker and listener's voice.

In this way, for noise, it is distinguished from the sound or image of the speaker and listener by comparing it with the basic or individual information entered before the actual sales job performance, the previously accumulated data, and the typical noise data identified from the place or time when the business was performed. can be analyzed.

For example, in the case of a sales job performed in a cafe, background noise, which is a typical noise generated in a cafe, noise generated during the coffee making process, and a buzzing sound generated according to the number of people in the cafe are received from the accumulated data or from outside. to distinguish the noise by comparing it with the currently recorded sound or recorded video.

Scenarios for classifying noise can be viewed as follows.

[Scenario #2]

Sales staff enter the location and date and time of sales in the process of entering basic and individual information before interacting with customers. In this case, the terminal 110 may provide a representative view for inputting the place and time to the sales staff on the interface or may allow the actual input.

For example, if it is assumed that the performance is carried out in the early morning at a cafe located on the side of the road, the sales staff may say that a lot of surrounding vehicles pass, that there are not many vehicles, or that there are few or many people resident in the cafe in the early morning. Select a representative example provided so that it can be evaluated qualitatively or quantitatively, or actually enter it specifically.

The sales person may input his/her own voice print and face image in advance, or may input the customer's voice print or face image together when they are input, and then the sales employee will develop sales activities for the customer.

Thereafter, the data extraction unit 202 and the data analysis unit 203 receive information such as location, time, people around, and vehicle noise input by the sales staff so as to leave only the speaker and listener's voice from the recorded sound or recorded image. , the noise is removed based on the typical noise data that such noise has, and the typical data of the noise received from the outside.

According to an embodiment, when the words of the speaker and the listener overlap, the data analysis unit 203 may distinguish and analyze the speaker's and the listener's voices, which may be performed based on the speech text analysis of the speaker and the listener obtained and analyzed in advance. can

Through this, it is possible to prevent a case in which the contents of the utterance cannot be clearly heard when the speaker and the listener's voices overlap in time and are heard in the overlapped state.

In this case, based on the unique voice gates of the speaker and the unique voice gates of the listener, if the voice gates overlap, only one voice gate is erased or reduced so that the other gate gate is clearly visible, and as a result, one voice gate overlaps the other gate gate. It is possible to remove or minimize the noise generated by this.

According to an embodiment, the data analysis unit 203 analyzes a voice that performs a start, end, highlight, or bookmark function, that is, a voice that performs a function such as cue sign.

Through this, it is possible to find a voice that causes the user to start, end, highlight, or bookmark the recording data from the recording data so that the start, end highlight or bookmark function is implemented.

According to an embodiment, the data analysis unit 203 may analyze differently according to the language of the speaker and the listener.

For example, in the case of recording in Korean, a clearer and cleaner analysis can be performed by analyzing the previously accumulated data or the Korean data received from the outside, and the Korean characteristics of the speaker and the listener.

In addition, even if the speaker and the listener speak the same Korean, in the case of using a dialect, analysis can be performed based on the typical characteristics of such a dialect.

Also, for example, even when recorded in English, it is possible to perform clearer and cleaner analysis by analyzing previously accumulated data or English data received from the outside, and English characteristics of the speaker and the listener.

In addition, even if the speaker and listener are the same English, it is possible to compare and analyze the English data that reflects the characteristics of the speaker and the listener, such as English spoken by Koreans and English in New York living in New York, received from the accumulated data or from outside. .

According to an exemplary embodiment, the data analysis unit 203 may classify and analyze a noun, a verb, a proposition, a suffix, a specific sentence, and the like from the recorded voice.

For example, the stem is an important meaning, and the ending can be changed in various ways. Because the stem is important for analysis, or the noun and verb will be important in conveying the meaning by comparing and analyzing it with previously accumulated data or data received from the outside. , adverbs and conjunctions are analyzed with less weight.

Here, in order to distinguish and analyze nouns, verbs, propositions, endings, etc., the nouns, verbs, propositions, endings, etc. may have been entered in advance by the sales staff, or existing historical data or data acquired from other sales staff, etc. can be analyzed collectively.

For example, if the content uttered by the speaker and listener is about a matter discussed in the past, it is possible to analyze the noun or verb that begins and ends with the matter discussed in the past, a particular sentence, or a particular cue sign. have.

And if it is about the matter discussed in the past, since it would have been mentioned in the existing data, it can be determined by comparing the degree of agreement with a noun or a verb existing in the existing data.

For example, if the content uttered by the speaker and the listener relates to the next meeting schedule, the utterance content of the date or time may be checked and analyzed as the next meeting schedule.

Also, for example, if the content uttered by the speaker and the listener relates to a financial problem, it can be analyzed through a noun, a verb, a preposition, a ending, a sentence, a cue sign, etc. related to the financial problem.

In this way, if the contents of the speech regarding the next meeting schedule, etc. are identified, it is reflected in the next business schedule of the sales staff so that it can be notified in advance, or if there are important contents (for example, financial contents or important transaction contents) By putting a separate tag on the customer, you can always see important content when viewing information related to the customer.

In analyzing the recorded image, the sound recorded during recording is omitted because it is not different from the above-described analysis method for the recording, and the analysis of the recorded image will be described.

In analyzing the recorded video, the basic expression state of the speaker and listener, the surprised expression state, the happy expression state, the sad expression state, the serious expression state, etc. are analyzed based on the previously inputted facial features of the speaker and listener.

In this case, it may be analyzed by comparing it with previously accumulated data and/or data on various facial expressions received from the outside.

For example, based on the nationalities and age groups of the speaker and the listener, it is possible to analyze the features of the faces and expressions of people of the corresponding nationality and the features of the faces and expressions of a specific age group.

The data generation unit 204 generates data for content for sales staff work management and training from the analyzed data (S304).

Among the analyzed data, basic information and individual information are provided for content creation and training for sales staff education, but may be generated after being classified by category.

For example, if the characteristics of the trainee are generally short, it is generated by classifying it as the data of a short sales employee. It is generated by classification, and is not limited to this example, and may be generated by classification for each category described above.

All words among the analyzed data may be converted into text data and generated.

For example, from the beginning of the recording to the end of the recording without any addition or subtraction, all speech (including both for the user to check internally before and after the customer interaction) can be converted to text and generated, in some cases, the customer interaction process It can be converted and created into text data based on only the generated data.

Also, for example, voice data recorded from a certain point in time to a certain end point may be converted into text data and generated.

In this case, it is possible to convert and create text data by setting various sections such as before the customer's response start, right after the response start, after the response starts, after the response ends, before the response ends, after the response ends, etc.

At this time, the analyzed image data may be converted together with the text data by setting various sections like the audio data.

On the other hand, audio or image data may be generated so that they can be reproduced in parallel in chronological order together with text data, but as will be described later, audio, video, and text are different from each other through output control such as reproduction of the generated data. It can also be created to have a time difference so that the beat occurs.

When text data is generated using audio or video data, noise and speech are distinguished by referring to the results of glottal analysis, and when speech is converted into text data, the responses of the speaker and the listener can be generated together.

For example, when the speaker speaks loudly, when converting to text data, highlighting such as increasing the font size, changing the color, or changing the font in comparison with other text for that part can be placed, and in some cases You can highlight text by placing parentheses at the top, bottom, or in front of it.

Here, the target to be highlighted may be a part input by classifying it as an important part based on voice or image data, or the volume of the voice or the content of the utterance itself.

For example, the input part is classified as important based on audio or video data, and the user separately inputs a button, utters a preset voice through voice, displays parentheses by hand, or checks V. It is possible to input a part called an important part of the image through a preset gesture, such as a gesture.

For example, it may be the volume of speech or the content of the speech itself that distinguishes the subject for highlighting. In this case, the volume of the speech is input larger than the average speech volume of the entire speech data, or the speech content itself is important. In the case of content, the corresponding part may be input as an important part.

Meanwhile, when the audio and video data are converted into text data, they may be converted based on a phrase of a paragraph divided into a speaker and a listener, but is not limited thereto.

For example, when simply converting into text data, the speaker speaks and the listener listens, and then the listener speaks and the text is generated according to the turn system in which the speaker speaks. In this case, after the speaker speaks, the speaker's If there is no voice input, it is assumed that the speaker has finished speaking, and the sentence of one paragraph is concluded. When a different listener speaks after the speaker's speech is over, it is recognized that the listener is speaking and converted into text data, but in this case, the paragraph may be different.

However, the present invention is not limited thereto, and for example, the speaker and the listener may speak overlappingly. For example, when overlapping of words occurs, text data may be overlapped to indicate that words overlap at a certain point in time and do not overlap at a certain point in time.

Here, the overlapping process may be performed such that the uttered texts of the speaker and the listener are different, but some sections are displayed so that they overlap each other, but the present invention is not limited thereto.

Meanwhile, when audio or image data is analyzed and text data is generated, the corresponding text data is connected to the audio or image data so that it can be processed together with the text data.

For example, when the corresponding text data is selected by the content processing unit 205 to be described later, voice or image data related to the corresponding text data can be output together.

On the other hand, when analyzing the image data, it is possible to analyze the expressions of the speaker and the listener, and as a result of the analysis, the expression data is generated by analyzing it in a positive, negative, or neutral state. The facial expression data generated here provides facial expression data when the script and image data are provided by the content processing unit 205, which will be described later, to provide what kind of facial expression the viewer will make.

Here, the criteria for judging the status of positive, negative or neutral based on the customer's expression may be based on the customer's eyebrows, lips, nasolabial folds, blinking eyes, and changes in the eyes, but it is not limited thereto, and in some cases, In the case of the same customer who meets twice or more, it can be judged by referring to the facial expression data of the customer entered up to the previous round.

In addition, in some cases, the determination may be made based on customer information separately input by the sales staff. For example, since each customer may have individual feelings, facial expressions, and dispositions that a sales employee directly feels, a sales employee who actually faces a customer enters the relevant information and based on the input, the sales staff refer to the facial expression data and deny it. , can be evaluated as positive or neutral.

As a specific example, if the employee's feelings after serving the customer are inputted, such as when the customer has no change in facial expression from the beginning or the basic facial expression is expressionless, or when the mood or expression is not good because something bad happened before responding to the customer The value is shifted from a neutral value to a negative one. Conversely, if the customer has a large change in facial expression from the beginning, if the basic expression is very bright, or if a good thing happened before responding and the employee has a good mood or expression, If what you feel after responding is input, the default value of the expression is moved from a neutral value to a positive value, and the zero point is adjusted.

In addition, such facial expression data may provide feedback to sales staff who performed actual sales duties or help predict sales results, and in this case, may be provided together with voice data or text data.

For example, by using the facial expression data, it is possible to analyze the overall customer response during sales job performance, customer response at a specific point in time, and provide feedback to employees who have performed the sales job to help improve work.

Also, for example, by analyzing the facial expressions of customers, it can help to predict the results of sales through the performance of the corresponding sales job.

In addition, for example, by providing facial expression data together with voice data or text data, it can be used as educational materials, such as what kind of facial expression is made at what point in time and what kind of reaction is shown.

On the other hand, the feedback data may be generated by analyzing the video data, but by analyzing the voice or video data of a sales person who is not a customer.

For example, what kind of expression do you make when you start serving customers, what kind of tone and height you want to speak, what is the speed of your speech, the gestures you make in the course of speaking, and listen to the customer without speaking. When no one is speaking, feedback data can be generated to give feedback by analyzing what kind of actions people take when no one is speaking.

The resulting scenario may be as follows.

[Scenario #3]

Before dealing with customers, it is assumed that sales staff deal with customers who have a negative atmosphere and impression due to business or non-sales reasons.

In this case, the terminal 110 may acquire the voice print and face image data of a customer having an unpleasant or negative atmosphere, impression, and expression, and may acquire the voice print or face image data of a sales employee who may feel embarrassed or uncomfortable.

Based on the acquired voiceprint and facial image data, the zero point for analyzing sales staff and customers starts from a neutral to somewhat negative scale, and if the customer responds positively during sales job performance, it starts from a neutral state and is positive. It is possible to give positive feedback on sales contents and results by giving more weight than the case of a positive response.

[Scenario #4]

A sales employee may use the terminal 110 to acquire a customer's own voice or face image instead of the customer's voice or face image.

By using the self-voice or face image, it is possible to provide feedback to the sales staff themselves in connection with the customer's reaction about how their voice is and what their face image is.

In the feedback process, it is possible to know how the customer reacted to what the sales staff said or did, or how the customer reacted to what the customer did.

The content processing unit 205, for the terminal 110, the step (S305) of processing and outputting content or a content list required to provide content for business management and training of the sales staff will be described in detail.

Here, the terminal 110 not only inputs the voice or video of an actual sales employee, which is basic data for business management and training of the sales employee according to the present invention, but also provides content for education and training of trainees as an interface in some cases. It may perform a providing function and may have an interface suitable for performing such a function, but is not limited thereto, and may be provided through a separate terminal.

The content for sales staff training referred to herein may include previously inputted voice data, image data, and text data, and derivatives thereof.

The voice data referred to herein includes data recorded during sales by an employee performing sales duties as the source of the voice data or audio data processed from the data, and the video data is also the same as the original, and the sales employee performs his/her duties It includes data recorded during recording or video data processed from the data, and text data refers to a thing extracted from audio or video data, generated, converted, or processed.

Here, the interface for outputting voice data not only has functions such as playing, pausing, stopping, rewinding, fast forwarding, bookmarking, skipping next, and skipping backwards.

Here, the interface for outputting image data may be the same as or include an interface for outputting audio data.

Here, the interface for outputting text data is provided together with the previous audio data or image data or provided separately, but when provided together with audio data or image data, the interface for outputting audio or image data may be maintained as it is.

In addition, the text data is provided in the form of a script, and the speaker and the listener may be displayed separately.

Here, the speaker and the listener may be displayed separately depending on the font, the character color, the character background, the position of the script, the display order of the script, the difference in shading, and the like, but is not limited thereto.

For example, in a situation where there is one speaker and one listener, the script may show the entire script from the beginning to the end of the content, or may show the entire script sequentially according to the playback time of audio or video data while displaying the entire script, Alternatively, when sequentially displaying the image data according to the playback time, the script may be displayed in a newly created form after not having a script according to the playback time, or it may be displayed sequentially while scrolling down through the entire screen.

For example, when there are at least two speakers and listeners, respectively, scripts of the speakers and listeners may be displayed as different, and in some cases, the real names and photos of the speaker and listener may be displayed together.

Meanwhile, in some cases, the trainee may perform job practice in the role of a sales employee with only the customer's voice, image, or text based on the voice, image, or text data.

In this case, the output customer's voice, video or text may be audio, video, or text acquired through a one-time sales activity, or may be data analyzed and generated through artificial intelligence, etc. according to the results collected from a plurality of data, In some cases, the data analyzed and generated through artificial intelligence or the like is output, and according to the response of the trainee, the analyzed and generated data may be output again through artificial intelligence, etc. in consideration of the trainee's response.

Therefore, trainees can learn greetings, explanations, etc. in response to a given customer's voice, video, text, or a combination thereof, and learn practical experiences by responding in various situations analyzed and created according to artificial intelligence. do.

Through such learning, better learning effects can be achieved than when learning is done through simple role play or text-based manuals or programs without being placed in a specific situation.

Scenarios referenced in these contents are as follows.

[Scenario #5]

Audio data and video data are extracted from the recorded sound or recorded video of a sales employee with the best sales performance among sales staff, and text data is generated therefrom.

The generated text data can be provided to other sales staff in the form of a script.

In addition to converting voice data into text data, the provided script also displays non-verbal elements like fingerprints.

In addition, when the trainee selects the provided script, corresponding audio or image data may be provided together.

On the other hand, sales staff can receive the results of their sales activities, and in this case, they can edit, process, delete, store, etc. from the original data initially obtained to the processed data.

For example, the sales staff may more clearly process and edit the information acquired by the system according to the present invention, and in some cases may add memos, photos, videos, and the like.

This can be used to receive feedback or record keeping after performing the sales job performed, or to add material to help trainees learn.

As such, with reference to the output audio, video, or text data, the output of the reproduced data may be adjusted in order to induce active participation or use of the user during education. For example, in the case of audio data, there may be adjustments such as adjusting an audio output or expanding data being provided in the case of image data.

The data output here can output the data generated by the previous data generation unit 204, and is not limited to those listed in the content processing unit 205 paragraph.

Claims (3)

Each of the following steps is performed in the service server 120,
Receiving data while performing a sales job from the terminal 110 (S301);
extracting sound and image data from the received data (S302);
analyzing the extracted audio and video (S303);
Generating data for sales staff work management and training from the analyzed data (S304);
To include a; processing and outputting the necessary content to provide content for sales staff training (S305);
Step (S305) of processing and outputting the necessary content to provide content for sales staff training,
Characterized in converting and generating recorded audio data or recorded image data from a certain point in time to a certain end point into text data,
Sales staff work management and training system. The method of claim 1,
Step (S305) of processing and outputting the necessary content to provide content for sales staff training,
Audio or video data can be generated so that it can be reproduced in parallel in chronological order with text data,
Sales staff work management and training system. The method of claim 1,
Step (S305) of processing and outputting the necessary content to provide content for sales staff training,
Analyzing the voice or facial image of a customer or sales associate and providing feedback to that employee;
The feedback is
The voice data is provided as text data in the form of a script, and the analysis result of the voice data or face image is linked together with the text data. Alternatively, the analysis result of the face image is provided together, and the customer's negative, neutral or positive state is derived from the customer's voice or face image, but when deriving the state, the customer's pre-sales state is reflected. and
Characterized in predicting sales results based on the derived customer status,
Sales staff work management and training system.

Images