CN108174046A - A kind of personnel monitoring system and method for call center - Google Patents
A kind of personnel monitoring system and method for call center Download PDFInfo
- Publication number
- CN108174046A CN108174046A CN201711107062.2A CN201711107062A CN108174046A CN 108174046 A CN108174046 A CN 108174046A CN 201711107062 A CN201711107062 A CN 201711107062A CN 108174046 A CN108174046 A CN 108174046A
- Authority
- CN
- China
- Prior art keywords
- module
- model
- face
- personnel
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 73
- 238000000034 method Methods 0.000 title claims abstract description 64
- 235000021167 banquet Nutrition 0.000 claims abstract description 74
- 230000008451 emotion Effects 0.000 claims abstract description 18
- 230000002159 abnormal effect Effects 0.000 claims abstract description 14
- 238000012545 processing Methods 0.000 claims abstract description 14
- 230000036651 mood Effects 0.000 claims description 34
- 238000000605 extraction Methods 0.000 claims description 21
- 238000004422 calculation algorithm Methods 0.000 claims description 15
- 230000006870 function Effects 0.000 claims description 15
- 238000009499 grossing Methods 0.000 claims description 12
- 230000001815 facial effect Effects 0.000 claims description 9
- 239000011159 matrix material Substances 0.000 claims description 9
- 238000013528 artificial neural network Methods 0.000 claims description 8
- 239000000284 extract Substances 0.000 claims description 6
- 238000004458 analytical method Methods 0.000 claims description 4
- 238000005094 computer simulation Methods 0.000 claims description 4
- 238000012417 linear regression Methods 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 230000002996 emotional effect Effects 0.000 claims description 3
- 230000007613 environmental effect Effects 0.000 claims description 3
- 230000008921 facial expression Effects 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- NGVDGCNFYWLIFO-UHFFFAOYSA-N pyridoxal 5'-phosphate Chemical compound CC1=NC=C(COP(O)(O)=O)C(C=O)=C1O NGVDGCNFYWLIFO-UHFFFAOYSA-N 0.000 claims description 3
- 238000012549 training Methods 0.000 claims description 3
- 230000010365 information processing Effects 0.000 claims description 2
- 235000013399 edible fruits Nutrition 0.000 claims 1
- 238000005516 engineering process Methods 0.000 abstract description 15
- 230000000694 effects Effects 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000007689 inspection Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 230000005236 sound signal Effects 0.000 description 2
- 206010027951 Mood swings Diseases 0.000 description 1
- 206010068052 Mosaicism Diseases 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000005764 inhibitory process Effects 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 210000003765 sex chromosome Anatomy 0.000 description 1
- 230000001755 vocal effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M3/00—Automatic or semi-automatic exchanges
- H04M3/42—Systems providing special services or facilities to subscribers
- H04M3/50—Centralised arrangements for answering calls; Centralised arrangements for recording messages for absent or busy subscribers ; Centralised arrangements for recording messages
- H04M3/51—Centralised call answering arrangements requiring operator intervention, e.g. call or contact centers for telemarketing
- H04M3/5175—Call or contact centers supervision arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L17/00—Speaker identification or verification techniques
- G10L17/04—Training, enrolment or model building
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L17/00—Speaker identification or verification techniques
- G10L17/06—Decision making techniques; Pattern matching strategies
- G10L17/14—Use of phonemic categorisation or speech recognition prior to speaker recognition or verification
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L17/00—Speaker identification or verification techniques
- G10L17/18—Artificial neural networks; Connectionist approaches
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
- G10L25/63—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination for estimating an emotional state
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Acoustics & Sound (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Signal Processing (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Child & Adolescent Psychology (AREA)
- Hospice & Palliative Care (AREA)
- Psychiatry (AREA)
- Computational Linguistics (AREA)
- Game Theory and Decision Science (AREA)
- Marketing (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Emergency Alarm Devices (AREA)
Abstract
The present invention relates to a kind of personnel monitoring systems for call center, including face sample pattern memory module of attending a banquet, it attends a banquet speech samples model memory module, monitoring module, face recognition module, sound identification module, model fitting module, result judgement module, as a result output module, model fitting module is by face recognition module, sound identification module passes through face, the working condition of voice messaging identification seat personnel is compared with sample pattern, result judgement module generates judgement conclusion information according to the comparing result that model fitting module exports, as a result output module is according to judgement conclusion information output monitoring information data.The method of above system includes obtaining data, processing data, correction data, judges state, output monitoring information.The present invention realizes the mobility status of seat personnel, abnormal emotion fluctuation in monitoring call center workplace in real time using based on face and speech recognition technology scheme, prevents seat personnel from practising fraud, has the characteristics that monitoring effect is good.
Description
Technical field
The present invention relates to a kind of personnel monitoring systems and method for call center, more particularly to a kind of to be known based on face
The personnel monitoring system and method for call center of other technology and speech recognition technology, belong to field of intelligent monitoring.
Background technology
The method of the seat personnel monitoring of present call center's workplace mainstream generally by setting gate inhibition register system, sit
Seat login system software account number cipher checks in condition monitoring to realize to attend a banquet, it is impossible to which effective prevention mutual allograph of employee is arrived and employee
For the situation of class.The job morale of seat personnel normally whether be related to service quality, present call center workplace can't
It realizes real-time mood monitoring, the later stage is generally taken manually to record sampling observation to judge, quality inspection personnel opens recording and monitors call note
It records and examination scoring is carried out to its attitude, therefore there are serious lag sex chromosome mosaicisms.Personnel's prison of conventional call centers
The problem of method generally existing detection of survey is inaccurate, and timeliness lags, the information of registering of employee, such as fingerprint, access card, account
Password etc., it is easy to be acquired, while traditional monitored by personnel's method cannot effectively distinguish employee between gang up allograph mutually
Arrive, privately for class the problem of.The mood monitoring mode of conventional call centers is usually to take later stage artificial judgment, and quality inspection personnel is beaten
It opens recording to monitor message registration and carry out examination scoring to its attitude, in the entire flow that scores, what is used is all with people
The defects of work mode is operated, thus there is processing lag, and artificial subjectivity is high, and identification is inaccurate.Therefore it is badly in need of providing a kind of
New system and method enable call center workplace to monitor the check-in mobility status for the employee that attends a banquet and check-in workplace automatically
Seat personnel whether be me, and the real-time emotion attended a banquet can be detected automatically.
Invention content
The present invention discloses new scheme for the personnel monitoring system and method for call center, using based on recognition of face
Technology and the scheme of speech recognition technology realize the mobility status of the seat personnel in monitoring call center workplace in real time, abnormal feelings
Thread fluctuates, and prevents seat personnel from practising fraud, solves the problems, such as that existing scheme monitoring effect is poor.
The present invention includes attend a banquet face sample pattern memory module, voice of attending a banquet for the personnel monitoring system of call center
Sample pattern memory module, monitoring module, face recognition module, sound identification module, model fitting module, result judgement mould
Block, result output module.Monitoring module sends the real-time face information of the current seat personnel for carrying out customer service work of acquisition
To face recognition module, face recognition module is sent after the real-time face information processing received is obtained real-time face information model
To model fitting module, model fitting module transfers the face sample pattern information for face sample pattern memory module of attending a banquet with receiving
To real-time face information model compared to judge real-time face information model whether in corresponding face sample pattern
In the range of the similarity threshold of information.Monitoring module believes the real-time voice of the current seat personnel for carrying out customer service work of acquisition
Breath is sent to sound identification module, and sound identification module handles the Instant audio messages received to obtain Instant audio messages model
After be sent to model fitting module, model fitting module transfers the speech samples model letter for speech samples model memory module of attending a banquet
It ceases and is compared to judge Instant audio messages model whether in corresponding voice sample with the Instant audio messages model received
In the range of the similarity threshold of this model information.The comparing result generation that result judgement module is exported according to model fitting module is sentenced
Disconnected conclusion information, as a result output module is according to judgement conclusion information output monitoring information data.
Further, the face sample pattern memory module of attending a banquet of this programme includes seat personnel face nozzle type characteristic
Library, seat personnel facial expression feature database, it is special that speech samples model memory module of attending a banquet includes seat personnel speech volume
Levy database, seat personnel speech frequency property data base, seat personnel voice word speed property data base, seat personnel voice sound
Adjust property data base.
Further, the monitoring module of this programme includes video photography module, calling record module, and face recognition module includes
Face characteristic extraction module, faceform's modeling module, sound identification module includes pronunciation extracting module, speech model is built
Mould module.The real-time face information of the current seat personnel for carrying out customer service work of acquisition is sent to face by video photography module
Characteristic extracting module, face characteristic extraction module are sent to faceform's modeling mould after extracting the characteristic parameter of real-time face information
Block, faceform's modeling module establish real-time face information model according to the real-time face information characteristics parameter received.Call record
The Instant audio messages of the current seat personnel for carrying out customer service work of acquisition are sent to pronunciation extracting module by sound module,
Speech model modeling module is sent to after the characteristic parameter of pronunciation extracting module extraction Instant audio messages, speech model is built
Mould module establishes Instant audio messages model according to the Instant audio messages characteristic parameter received.
Further, the result output module of this programme includes cheating signal projector, standby signal transmitter, alarm signal
Transmitter, as a result output module is cheating driving cheating manager of the signal projector to call center according to judgement conclusion information
Alarm signal is sent out, as a result output module is that improper mood drives standby signal transmitter to attending a banquet according to judgement conclusion information
Personnel send out standby signal, and as a result output module is improper mood driving alarm signal transmitting according to judgement conclusion information
Device sends out alarm signal to the manager of call center.
The invention also discloses a kind of personnel monitoring's method for call center, for the personnel monitoring side of call center
Method includes face sample mould of attending a banquet based on the personnel monitoring system for call center for the personnel monitoring system of call center
Type memory module, attend a banquet speech samples model memory module, monitoring module, face recognition module, sound identification module, model
With module, result judgement module, result output module, including step：In advance acquisition seat personnel carry out customer service work when
Normal and the face sample information under abnormal emotion state, voice sample information；(2) the face to the seat personnel acquired in advance
Sample information, voice sample information are pre-processed；(3) face sample information, voice sample information of the extraction by pretreatment
Characteristic parameter；(4) using the Acoustic Modeling method of DNN- deep neural networks and the modeling of the face identification system based on PCA methods
Method create be stored in after the face sample pattern of seat personnel, speech samples model face sample pattern memory module of attending a banquet,
It attends a banquet in speech samples model memory module；(5) the real-time face information of the seat personnel of monitoring module acquisition disengaging workplace, prison
Control module acquires the real-time face information of seat personnel to work on station of attending a banquet, Instant audio messages；Face recognition module,
Sound identification module pre-processes the real-time face information of the seat personnel of acquisition, Instant audio messages；(7) recognition of face
Module, sound identification module extraction are by the real-time face information pre-processed, the characteristic parameter of Instant audio messages；(8) face is known
The Acoustic Modeling method of other module, sound identification module using DNN- deep neural networks, the recognition of face system based on PCA methods
Modeling method of uniting creates real-time face information model, the Instant audio messages model of seat personnel；(9) model fitting module will be real
When face information model, Instant audio messages model matched after being matched with face sample pattern, speech samples model
As a result；(10) result judgement module judges the state of seat personnel disengaging call center workplace according to matching result；(11) result judgement
Module judges whether the personnel of login system are to attend a banquet me according to matching result；(12) result judgement module is sentenced according to matching result
The current emotional state of disconnected seat personnel；(13) judgement conclusion information output monitoring of the result output module according to step (10), (11), (12)
Information data.
Further, the method for this programme step (2), (6) in, the process of pretreatment includes：
The interference noise in picture is smoothed using median filter method, it is assumed that original image is, it is original
The size of image is a X b,
The vertical gray-level projection function of image is：,
Vertically gray scale smoothing processing function is：,
The horizontal environmental well function of image is：,
Horizontal gray scale smoothing processing function is：,
The noise in recording is eliminated using the denoising method of two benches Meier-Wiener filtering,
Filter amplitudes calculation formula is：, in filter amplitudes
In calculation formula,,
Smoothing processing coefficient is：,
Smoothing processing formula is：。
Further, the method for this programme step (3), (7) in, the process for extracting characteristic parameter includes：
Preemphasis is carried out to voice signal using HTK speech feature extractions algorithm, is filtered using Meier wave filter group instead of Bark
Device group carries out critical band analysis, extracts mel-frequency PLP speech characteristic parameters, is realized using cascading linear regression model algorithm
The acquisition of faceform's parameter of 128 characteristic points.
Further, the method for this programme step (4), (8) in, DNN- deep neural network Acoustic Modeling models are：
, in above-mentioned Acoustic Modeling model, N is the number of training sample,It is 1 in mark state point value for the destination probability of mark, the value of other output state points is 0,For
The reality output probability of DNN is based on PCA method face identification system modeler models：
, in above-mentioned face modeler model, sample number a is special
Sign number is b, and subtracting the head portrait sample matrix after mean value is, covariance matrix is c*c, b face of selection
Feature vector composition matrix be。
Further, the result output module of the method for this programme include cheating signal projector, step (11) in, as a result sentence
Cover half root tuber according to matching result whether system setting matching threshold in the range of operate, if in threshold range, result judgement
What module judgement was worked using system is to attend a banquet me, there is no cheating, if not in threshold range, result judgement module
What judgement work using system is not to attend a banquet I, and there are cheating, system forces to use the people of system work offline at present,
Cheating signal projector sends out alarm signal to the manager of call center.As a result output module include standby signal transmitter,
Alarm signal transmitter, step (12) in, result judgement module according to matching result whether system set matching threshold range
Interior operation, if the matching result of faceform and speech model, in the threshold range of setting, the judgement of result judgement module is sat
The job morale of seat personnel is normal, does not need to intervene seat personnel, if the matching result of faceform or speech model
Not in the threshold range of setting, the job morale of result judgement module judgement seat personnel is abnormal, needs to seat personnel
Intervened, standby signal transmitter sends out standby signal to seat personnel, and alarm signal transmitter is to call center
Manager sends out alarm signal.
Further, the alarm signal transmitter of the method for this programme sends out level-one according to the parameter area of abnormal mood and accuses
Alert signal, two level alarm signal, three-level alarm signal, the corresponding parameter area of the Level 1Alarming signal is that facial emotions are non-just
Often and voice mood is improper occurs simultaneously, the corresponding parameter area of the two level alarm signal be facial emotions normally and voice
Mood is improper, and the corresponding parameter area of the three-level alarm signal is that facial emotions are improper and voice mood is normal, calling
The manager at center adjusts the degree for intervening related seat personnel according to the alarm signal of different stage.
The present invention is used for the personnel monitoring system and method for call center based on face recognition technology and speech recognition
The scheme of technology realizes the mobility status of the seat personnel in monitoring call center workplace in real time, abnormal emotion fluctuation, prevents from sitting
Seat personnel practise fraud, and have the characteristics that monitoring effect is good.
Description of the drawings
Fig. 1 is the deployment schematic diagram of call center personnel monitoring system.
Fig. 2 is schematic diagram of the present invention for the personnel monitoring system of call center.
Fig. 3 is attend a banquet sound-groove model and voice mood model foundation flow chart.
Fig. 4 is attend a banquet faceform and face mood model Establishing process figure.
Fig. 5 is the flow diagram of call center personnel monitoring system and method.
Specific embodiment
This programme discloses a kind of monitoring system and method for attending a banquet for call center workplace, and this programme is known using face
Other technology and speech recognition technology realize the mobility status attended a banquet monitored in real time in call center workplace, abnormal emotion fluctuation,
Cheating of attending a banquet is prevented, improves the accuracy of system personnel monitoring and the efficiency of mood monitoring so that the manager of call center
It can automatically, accurately and rapidly find to attend a banquet and whether leave the court, practise fraud for class and identify the abnormal emotion attended a banquet.Such as figure
1st, shown in 2, this programme monitoring system includes：(1) attend a banquet sample acoustic model memory module, for storing each seat obtained in advance
Audio sample information and the vocal print feature that from audio obtains of the seat personnel when being normally carried out customer service work；(2) attend a banquet sample people
Face model memory module, for store face sample information of each seat personnel obtained in advance in normal mood and it is non-just
Face sample information during reason thread；(3) monitoring module is connect with call center system, for obtaining current progress customer service in real time
The face information and voice audio signals of the seat personnel of work；(4) face characteristic extraction module transfers monitoring module acquisition
The above-mentioned face information attended a banquet using face feature extraction algorithm, extracts face characteristic parameter；(5) faceform's modeling module,
According to the face characteristic parameter of extraction, faceform is established using face modeling algorithm；(6) pronunciation extracting module transfers prison
The above-mentioned speech audio information attended a banquet that module obtains is controlled, using speech feature extraction algorithm, extracts every speech characteristic parameter；
(7) according to the speech characteristic parameter of attending a banquet of extraction, acoustic mode is established using acoustic model modeling algorithm for acoustic model modeling module
Type；(8) model fitting module, using Model Matching algorithm, the sample that will be preserved in newly-built acoustic model and faceform and system
This model is compared, and judges whether newly-built acoustic model and faceform are in the similarity threshold of corresponding sample pattern
In the range of；(9) result judgement module, according to Model Matching as a result, judgement it is final as a result, sentencing including whether in person result of attending a banquet
Whether fixed and mood of attending a banquet normally judges；(10) result output module sends monitoring data in real time to the manager of call center, packet
Include normal information and warning information.
In order to remind related personnel in time, unsuitable working condition, the result output mould of this programme monitoring system are corrected
Block further includes：(1) practise fraud signal projector, for not being that system registry attends a banquet me in result judgement module judgement seat personnel
When, to call center management, person sends out alarm signal；(2) standby signal transmitter, for judging the people that attends a banquet in result judgement module
Member is when producing improper mood, to the standby signal that sends out in person of attending a banquet；(3) alarm signal transmitter, in result judgement
When module judgement seat personnel produces improper mood, to call center management, person sends out alarm signal.
More than monitoring system is based on, this programme also discloses a kind of monitoring method, i.e., using face recognition technology and voice
Identification technology realizes the mobility status attended a banquet monitored in real time in call center workplace, abnormal emotion fluctuation, prevents cheating of attending a banquet
Method, include the following steps：
As shown in Figure 3,4, the head portrait sample letter of different moods of each seat personnel when being normally carried out customer service work is obtained in advance
The audio sample information of breath and different moods.
The sample head portrait and sample voice of the user of acquisition are pre-processed, including:Using median filter method, to figure
Interference noise in piece is smoothed.Assuming that original image is, size is a X b.
（The vertical gray-level projection function of formula one, image）,
（Formula two, vertical gray scale smoothing processing function）,
（The horizontal environmental well function of formula three, image）,
（Formula four, horizontal gray scale smoothing processing function）.
The noise in recording is eliminated using the denoising method of two benches Meier-Wiener filtering：
（Formula five, filter amplitudes calculate
Formula）,（Formula six, smoothing processing coefficient）,（Formula seven, smoothing processing formula）.
Extract all kinds of characteristic parameters of sample head portrait and sample audio；Using HTK speech feature extraction algorithms, voice is believed
Number preemphasis is carried out, and carry out critical band analysis instead of Bark wave filters group with Meier wave filter group, extracted Meier frequently
Rate PLP（Mel-frequency, MF-PLP）Speech characteristic parameter.This feature is shown preferably under various acoustic enviroments
Performance；And the acquisition of faceform's parameter of 128 characteristic points is realized using cascading linear regression model algorithm.
Use DNN（Deep neural network）Acoustic Modeling technology and based on PCA methods face identification system modeling skill
The sample acoustic model and sample faceform that art establishment is attended a banquet.
,
In above formula：N is the number of training sample,It is 1 in mark state point value for the destination probability of mark, other
The value of output state point is 0, the reality output probability for being DNN.
In above formula, sample number a, characteristic b, subtracting the head portrait sample matrix after mean value is, association
Variance matrix is c*c, and the matrix of the b head portrait feature vector composition of selection is。
As shown in Figure 1,5, the camera on the doorway of workplace monitors the face information attended a banquet of acquisition disengaging workplace in real time, sits
Camera and sound pick-up outfit on seat station acquire the head portrait and calling record of the seat personnel of work at present in real time.
System to the user of acquisition attend a banquet head portrait and voice pre-processes.
Head portrait picture and call audio signal to acquisition carry out the analyses such as recognition of face, speech recognition, use HTK voices
Feature extraction algorithm and cascading linear regression model algorithm extract speech model parameter and faceform's characteristic parameter.
By faceform's parameter of extraction and speech model parameter, DNN is used（Deep neural network）Acoustic Modeling skill
Art and the face identification system modeling technique acoustic model attended a banquet of establishment based on PCA methods and faceform.
Sample voice model that the speech model newly created and faceform are preserved with system and sample faceform into
Row Model Matching, obtains matching result.
Speech model matching algorithm：, Hausdorff image template matching algorithms：。
Flow of personnel monitoring function, in system record call center workplace, attend a banquet into artificial situation.
Anti- cheating function, system judge whether attending a banquet for login system is to attend a banquet me.According to Model Matching result whether
In the range of system setting matching threshold, if in threshold range, judge to work using system is to attend a banquet me, and there is no works
Disadvantage behavior if it is not, then judge to work using system is not to attend a banquet me, there are cheating, and forces this to attend a banquet under user
Line.
Mood monitoring function, system judge current emotional state of attending a banquet.Whether set according to Model Matching result in system
It puts in the range of matching threshold, if acoustic model and the matching result of faceform, in the threshold range of setting, judgement is attended a banquet
Job morale it is normal, do not need to intervene, if it is not, then judging that the job morale attended a banquet is abnormal, need to intervene attending a banquet.
In order to remind related personnel in time, unsuitable working condition is corrected, this programme is judging corresponding seat personnel
After producing improper mood, alarm signal is sent out to the manager of call center, and indicates the seat for generating improper mood
Seat personnel.This programme sends out alarm signal in person after judging that corresponding seat personnel produces improper mood to attending a banquet.
The alarm level range of this programme includes first level alarm, second level alarm and third level alarm, first level alarm
Parameter area be that facial emotions are improper and voice mood is improper while occur, the parameter area of second level alarm is face
Portion's mood is normally and voice mood is improper, the parameter area of third level alarm be facial emotions it is improper with voice mood just
Often.The alarm of different stage represents the severity for mood swing of attending a banquet, while represents the intervention degree of manager.This programme
Default characteristic parameter includes facial nozzle type, facial expression degreeof tortuosity, volume, speech frequency, the word speed of audio and sound
Tone of frequency etc..
This programme discloses a kind of novel monitored by personnel's system and method for call center, based on face recognition technology
Carry out the mobility status attended a banquet under the professional air of detection of call center with speech recognition technology, attend a banquet and whether occur practising fraud for class and be
It is no abnormal emotion occur.It is monitored compared to existing access card, account number cipher, artificial to spot-check quality inspection and attend a banquet the method for mood, we
Monitored by personnel's efficiency the degree of automation higher of case, Emotion identification accuracy rate higher, and accurately detecting seat personnel
After improper mood, alarm can be sent to manager in real time and attend a banquet me, it is instructed to carry out reasonably self adjustment and peace
Behavior is comforted, seat personnel is helped preferably to cope with improper mood.It is based on more than feature, the people for call center of this programme
Member's monitoring system and method has prominent substantive distinguishing features and significant progress compared to existing scheme.
This programme is not limited to interior disclosed in specific embodiment for the personnel monitoring system and method for call center
Hold, the technical solution occurred in embodiment can be extended based on the understanding of those skilled in the art, those skilled in the art's root
The simple replacement scheme made according to this programme combination common knowledge also belongs to the range of this programme.
Claims (10)
1. a kind of personnel monitoring system for call center, it is characterized in that including face sample pattern memory module of attending a banquet, sitting
Seat speech samples model memory module, monitoring module, face recognition module, sound identification module, model fitting module, result are sentenced
Cover half block, result output module,
The real-time face information of the current seat personnel for carrying out customer service work of acquisition is sent to the people by the monitoring module
Face identification module, the face recognition module are sent after the real-time face information processing received is obtained real-time face information model
To the model fitting module, the model fitting module transfers the face sample of the face sample pattern memory module of attending a banquet
Model information is compared to judge whether the real-time face information model is located with the real-time face information model received
In the range of the similarity threshold of corresponding face sample pattern information,
The Instant audio messages of the current seat personnel for carrying out customer service work of acquisition are sent to institute's predicate by the monitoring module
The Instant audio messages received are handled to send after obtaining Instant audio messages model by sound identification module, the sound identification module
To the model fitting module, the model fitting module transfers the speech samples of the speech samples model memory module of attending a banquet
Model information is compared to judge whether the Instant audio messages model is located with the Instant audio messages model received
In the range of the similarity threshold of corresponding speech samples model information,
The result judgement module generates judgement conclusion information, the knot according to the comparing result that the model fitting module exports
Fruit output module is according to the judgement conclusion information output monitoring information data.
2. the personnel monitoring system according to claim 1 for call center, which is characterized in that the face sample of attending a banquet
This model memory module includes seat personnel face nozzle type property data base, seat personnel facial expression feature database, described
Speech samples model memory module of attending a banquet includes seat personnel speech volume property data base, seat personnel speech frequency characteristic
According to library, seat personnel voice word speed property data base, seat personnel speech tone property data base.
3. the personnel monitoring system according to claim 1 for call center, which is characterized in that the monitoring module packet
Video photography module, calling record module are included, the face recognition module includes face characteristic extraction module, faceform models
Module, the sound identification module include pronunciation extracting module, speech model modeling module,
The real-time face information of the current seat personnel for carrying out customer service work of acquisition is sent to institute by the video photography module
Face characteristic extraction module is stated, the face characteristic extraction module is sent to after extracting the characteristic parameter of the real-time face information
Faceform's modeling module, faceform's modeling module are built according to the real-time face information characteristics parameter received
The real-time face information model is found,
The Instant audio messages of the current seat personnel for carrying out customer service work of acquisition are sent to institute by the calling record module
Pronunciation extracting module is stated, the pronunciation extracting module is sent to after extracting the characteristic parameter of the Instant audio messages
The speech model modeling module, the speech model modeling module are built according to the Instant audio messages characteristic parameter received
Found the Instant audio messages model.
4. the personnel monitoring system according to claim 1 for call center, which is characterized in that the result exports mould
Block includes cheating signal projector, standby signal transmitter, alarm signal transmitter, and the result output module is sentenced according to
Disconnected conclusion information is that the cheating driving cheating signal projector sends out alarm signal, the result to the manager of call center
Output module is that improper mood drives the standby signal transmitter to seat personnel according to the judgement conclusion information
Standby signal is sent out, the result output module is that improper mood drives the alarm signal according to the judgement conclusion information
Transmitter sends out alarm signal to the manager of call center.
5. a kind of personnel monitoring's method for call center is based on for personnel monitoring's method of call center in calling
The personnel monitoring system of the heart, for call center personnel monitoring system include attend a banquet face sample pattern memory module, attend a banquet
Speech samples model memory module, monitoring module, face recognition module, sound identification module, model fitting module, result judgement
Module, result output module, it is characterized in that including step：
(1) face sample information during acquisition seat personnel progress customer service work in advance under normal and abnormal emotion state, voice
Sample information；
(2) face sample information, the voice sample information of the seat personnel to acquiring in advance pre-process；
(3) the characteristic parameter of face sample information, voice sample information of the extraction by pretreatment；
(4) using the Acoustic Modeling method of DNN- deep neural networks and the wound of the face identification system modeling method based on PCA methods
Attend a banquet face sample pattern memory module, language of attending a banquet are stored in after building the face sample pattern of seat personnel, speech samples model
In sound sample pattern memory module；
(5) the real-time face information of the seat personnel of monitoring module acquisition disengaging workplace, monitoring module acquires to work on station of attending a banquet
The real-time face information of seat personnel, Instant audio messages；
Face recognition module, sound identification module to the real-time face information of the seat personnel of acquisition, Instant audio messages into
Row pretreatment；
(7) face recognition module, sound identification module extraction are by the real-time face information pre-processed, the spy of Instant audio messages
Levy parameter；
(8) face recognition module, sound identification module use the Acoustic Modeling method of DNN- deep neural networks, based on PCA methods
Face identification system modeling method create seat personnel real-time face information model, Instant audio messages model；
(9) model fitting module is by real-time face information model, Instant audio messages model and face sample pattern, speech samples
Model obtains matching result after being matched；
(10) result judgement module judges the state of seat personnel disengaging call center workplace according to matching result；
(11) result judgement module judges whether the personnel of login system are to attend a banquet me according to matching result；
(12) result judgement module judges the current emotional state of seat personnel according to matching result；
(13) judgement conclusion information output monitoring information data of the result output module according to step (10), (11), (12).
6. personnel monitoring's method according to claim 5 for call center, which is characterized in that step (2), (6) in,
The process of pretreatment includes：
The interference noise in picture is smoothed using median filter method, it is assumed that original image is, original graph
The size of picture is a X b,
The vertical gray-level projection function of image is：,
Vertically gray scale smoothing processing function is：,
The horizontal environmental well function of image is：,
Horizontal gray scale smoothing processing function is：,
The noise in recording is eliminated using the denoising method of two benches Meier-Wiener filtering,
Filter amplitudes calculation formula is：, in filter amplitudes meter
It calculates in formula,,
Smoothing processing coefficient is：,
Smoothing processing formula is：。
7. personnel monitoring's method according to claim 5 for call center, which is characterized in that step (3), (7) in,
The process of extraction characteristic parameter includes：
Preemphasis is carried out to voice signal using HTK speech feature extractions algorithm, is filtered using Meier wave filter group instead of Bark
Device group carries out critical band analysis, extracts mel-frequency PLP speech characteristic parameters, is realized using cascading linear regression model algorithm
The acquisition of faceform's parameter of 128 characteristic points.
8. personnel monitoring's method according to claim 5 for call center, which is characterized in that step (4), (8) in,
DNN- deep neural network Acoustic Modeling models are：
, in above-mentioned Acoustic Modeling model, N is the number of training sample,It is 1 in mark state point value for the destination probability of mark, the value of other output state points is 0,For
The reality output probability of DNN,
It is based on PCA method face identification system modeler models：
, in above-mentioned face modeler model, sample number a, feature
Number is b, and subtracting the head portrait sample matrix after mean value is, covariance matrix is c*c, and b face of selection is special
Levying the matrix that vector forms is。
9. personnel monitoring's method according to claim 5 for call center, which is characterized in that
As a result output module include cheating signal projector, step (11) in, result judgement module according to matching result whether
It is operated in the range of the matching threshold of system setting,
If in threshold range, what the judgement of result judgement module was worked using system is to attend a banquet me, there is no cheating,
If not in threshold range, what the judgement of result judgement module was worked using system is not to attend a banquet me, there are cheating,
System forces the people to work at present using system offline, and cheating signal projector sends out warning letter to the manager of call center
Number；
As a result output module include standby signal transmitter, alarm signal transmitter, step (12) in, result judgement module according to
With result whether system setting matching threshold in the range of operate,
If the matching result of faceform and speech model, in the threshold range of setting, result judgement module judges the people that attends a banquet
The job morale of member is normal, does not need to intervene seat personnel,
If the matching result of faceform or speech model, not in the threshold range of setting, result judgement module judges the people that attends a banquet
The job morale of member is abnormal, needs to intervene seat personnel, standby signal transmitter is sent out to seat personnel to be carried
Show signal, alarm signal transmitter sends out alarm signal to the manager of call center.
10. personnel monitoring's method according to claim 9 for call center, which is characterized in that alarm signal emits
Device sends out Level 1Alarming signal, two level alarm signal, three-level alarm signal, the level-one according to the parameter area of abnormal mood
The corresponding parameter area of alarm signal is that facial emotions are improper and voice mood is improper while occurs, the two level alarm letter
Number corresponding parameter area is that facial emotions are normal and voice mood is improper, the corresponding parameter area of the three-level alarm signal
It is that facial emotions are improper and voice mood is normal, the manager of call center is adjusted according to the alarm signal of different stage to be intervened
The degree of related seat personnel.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711107062.2A CN108174046A (en) | 2017-11-10 | 2017-11-10 | A kind of personnel monitoring system and method for call center |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711107062.2A CN108174046A (en) | 2017-11-10 | 2017-11-10 | A kind of personnel monitoring system and method for call center |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108174046A true CN108174046A (en) | 2018-06-15 |
Family
ID=62526939
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711107062.2A Pending CN108174046A (en) | 2017-11-10 | 2017-11-10 | A kind of personnel monitoring system and method for call center |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108174046A (en) |
Cited By (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109147529A (en) * | 2018-09-29 | 2019-01-04 | 上海嘉奕医学科技有限公司 | The standard patient simulation system of achievable humane examination based on artificial intelligence |
| CN109451190A (en) * | 2018-10-23 | 2019-03-08 | 平安科技（深圳）有限公司 | Attend a banquet station management method and device based on condition monitoring |
| CN109688274A (en) * | 2018-08-21 | 2019-04-26 | 平安科技（深圳）有限公司 | It attends a banquet monitoring alarm method, apparatus, equipment and computer readable storage medium |
| CN109994102A (en) * | 2019-04-16 | 2019-07-09 | 上海航动科技有限公司 | A kind of outer paging system of intelligence based on Emotion identification |
| CN110163655A (en) * | 2019-04-15 | 2019-08-23 | 中国平安人寿保险股份有限公司 | Distribution method of attending a banquet, device, equipment and storage medium based on gradient boosted tree |
| CN111209852A (en) * | 2020-01-05 | 2020-05-29 | 南通市中心血站 | Blood donor service system based on intelligent voice calling technology |
| CN111241883A (en) * | 2018-11-29 | 2020-06-05 | 百度在线网络技术（北京）有限公司 | Method and device for preventing remote detected personnel from cheating |
| CN111274460A (en) * | 2020-01-20 | 2020-06-12 | 重庆百事得大牛机器人有限公司 | Semantic information acquisition system and method based on posture control |
| CN111311041A (en) * | 2018-12-12 | 2020-06-19 | 马上消费金融股份有限公司 | Data extraction method and equipment and related device |
| CN111382608A (en) * | 2018-12-28 | 2020-07-07 | 广州盈可视电子科技有限公司 | Intelligent detection system with emotion recognition function |
| CN112002331A (en) * | 2020-08-05 | 2020-11-27 | 广州景瑞智能科技有限公司 | Method and system for recognizing emotion based on conversation voice |
| CN112508536A (en) * | 2020-12-21 | 2021-03-16 | 深圳供电局有限公司 | Monitoring system of electric power system monitoring room |
| CN112738347A (en) * | 2020-12-10 | 2021-04-30 | 南方电网数字电网研究院有限公司 | Information safety operation monitoring and early warning calling system |
| CN113343850A (en) * | 2021-06-07 | 2021-09-03 | 广州市奥威亚电子科技有限公司 | Method, device, equipment and storage medium for checking video character information |
| CN113422876A (en) * | 2021-06-24 | 2021-09-21 | 广西电网有限责任公司 | AI-based auxiliary management method, system and medium for power customer service center |
| CN113438375A (en) * | 2021-05-24 | 2021-09-24 | 商客通尚景科技（上海）股份有限公司 | Method for maintaining seat state |
| CN113724738A (en) * | 2021-08-31 | 2021-11-30 | 平安普惠企业管理有限公司 | Voice processing method, decision tree model training method, device, equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2005002189A1 (en) * | 2003-06-25 | 2005-01-06 | Alai Limited | Apparatus and method for load balancing contact communications |
| CN102625005A (en) * | 2012-03-05 | 2012-08-01 | 广东天波信息技术股份有限公司 | Call center system with function of real-timely monitoring service quality and implement method of call center system |
| CN103701999A (en) * | 2012-09-27 | 2014-04-02 | 中国电信股份有限公司 | Method and system for monitoring voice communication of call center |
| CN106024015A (en) * | 2016-06-14 | 2016-10-12 | 上海航动科技有限公司 | Call center agent monitoring method and system |
| CN106797417A (en) * | 2014-09-03 | 2017-05-31 | 互联企信公司 | Liaison centre is counter to cheat monitoring, detection and prevention scheme |
| CN106845751A (en) * | 2016-11-07 | 2017-06-13 | 成都科曦科技有限公司 | A kind of wisdom hotel is to customer service business decision commanding control system |
-
2017
- 2017-11-10 CN CN201711107062.2A patent/CN108174046A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2005002189A1 (en) * | 2003-06-25 | 2005-01-06 | Alai Limited | Apparatus and method for load balancing contact communications |
| CN102625005A (en) * | 2012-03-05 | 2012-08-01 | 广东天波信息技术股份有限公司 | Call center system with function of real-timely monitoring service quality and implement method of call center system |
| CN103701999A (en) * | 2012-09-27 | 2014-04-02 | 中国电信股份有限公司 | Method and system for monitoring voice communication of call center |
| CN106797417A (en) * | 2014-09-03 | 2017-05-31 | 互联企信公司 | Liaison centre is counter to cheat monitoring, detection and prevention scheme |
| CN106024015A (en) * | 2016-06-14 | 2016-10-12 | 上海航动科技有限公司 | Call center agent monitoring method and system |
| CN106845751A (en) * | 2016-11-07 | 2017-06-13 | 成都科曦科技有限公司 | A kind of wisdom hotel is to customer service business decision commanding control system |
Cited By (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109688274A (en) * | 2018-08-21 | 2019-04-26 | 平安科技（深圳）有限公司 | It attends a banquet monitoring alarm method, apparatus, equipment and computer readable storage medium |
| CN109147529A (en) * | 2018-09-29 | 2019-01-04 | 上海嘉奕医学科技有限公司 | The standard patient simulation system of achievable humane examination based on artificial intelligence |
| CN109451190A (en) * | 2018-10-23 | 2019-03-08 | 平安科技（深圳）有限公司 | Attend a banquet station management method and device based on condition monitoring |
| CN109451190B (en) * | 2018-10-23 | 2021-04-20 | 平安科技（深圳）有限公司 | Seat station management method and device based on state monitoring |
| CN111241883A (en) * | 2018-11-29 | 2020-06-05 | 百度在线网络技术（北京）有限公司 | Method and device for preventing remote detected personnel from cheating |
| CN111241883B (en) * | 2018-11-29 | 2023-08-25 | 百度在线网络技术（北京）有限公司 | Method and device for preventing cheating of remote tested personnel |
| CN111311041A (en) * | 2018-12-12 | 2020-06-19 | 马上消费金融股份有限公司 | Data extraction method and equipment and related device |
| CN111311041B (en) * | 2018-12-12 | 2024-02-20 | 马上消费金融股份有限公司 | Data extraction method and device and related device |
| CN111382608A (en) * | 2018-12-28 | 2020-07-07 | 广州盈可视电子科技有限公司 | Intelligent detection system with emotion recognition function |
| CN110163655A (en) * | 2019-04-15 | 2019-08-23 | 中国平安人寿保险股份有限公司 | Distribution method of attending a banquet, device, equipment and storage medium based on gradient boosted tree |
| CN110163655B (en) * | 2019-04-15 | 2024-03-05 | 中国平安人寿保险股份有限公司 | Agent distribution method, device and equipment based on gradient lifting tree and storage medium |
| CN109994102A (en) * | 2019-04-16 | 2019-07-09 | 上海航动科技有限公司 | A kind of outer paging system of intelligence based on Emotion identification |
| CN111209852A (en) * | 2020-01-05 | 2020-05-29 | 南通市中心血站 | Blood donor service system based on intelligent voice calling technology |
| CN111274460A (en) * | 2020-01-20 | 2020-06-12 | 重庆百事得大牛机器人有限公司 | Semantic information acquisition system and method based on posture control |
| CN112002331A (en) * | 2020-08-05 | 2020-11-27 | 广州景瑞智能科技有限公司 | Method and system for recognizing emotion based on conversation voice |
| CN112738347A (en) * | 2020-12-10 | 2021-04-30 | 南方电网数字电网研究院有限公司 | Information safety operation monitoring and early warning calling system |
| CN112508536B (en) * | 2020-12-21 | 2023-10-13 | 深圳供电局有限公司 | Monitoring system of power system monitoring room |
| CN112508536A (en) * | 2020-12-21 | 2021-03-16 | 深圳供电局有限公司 | Monitoring system of electric power system monitoring room |
| CN113438375A (en) * | 2021-05-24 | 2021-09-24 | 商客通尚景科技（上海）股份有限公司 | Method for maintaining seat state |
| CN113438375B (en) * | 2021-05-24 | 2022-09-27 | 商客通尚景科技（上海）股份有限公司 | Method for maintaining seat state |
| CN113343850A (en) * | 2021-06-07 | 2021-09-03 | 广州市奥威亚电子科技有限公司 | Method, device, equipment and storage medium for checking video character information |
| CN113422876A (en) * | 2021-06-24 | 2021-09-21 | 广西电网有限责任公司 | AI-based auxiliary management method, system and medium for power customer service center |
| CN113422876B (en) * | 2021-06-24 | 2022-05-10 | 广西电网有限责任公司 | AI-based auxiliary management method, system and medium for power customer service center |
| CN113724738A (en) * | 2021-08-31 | 2021-11-30 | 平安普惠企业管理有限公司 | Voice processing method, decision tree model training method, device, equipment and storage medium |
| CN113724738B (en) * | 2021-08-31 | 2024-04-23 | 硅基(昆山)智能科技有限公司 | Speech processing method, decision tree model training method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108174046A (en) | A kind of personnel monitoring system and method for call center | |
| CN107818798B (en) | Customer service quality evaluation method, device, equipment and storage medium | |
| CN105632501B (en) | A kind of automatic accent classification method and device based on depth learning technology | |
| US8861779B2 (en) | Methods for electronically analysing a dialogue and corresponding systems | |
| CN110458154A (en) | Face identification method, device and computer readable storage medium | |
| CN108888281A (en) | State of mind appraisal procedure, equipment and system | |
| CN108427916A (en) | A kind of monitoring system and monitoring method of mood of attending a banquet for customer service | |
| CN112149638B (en) | Personnel identity recognition system construction and use method based on multi-modal biological characteristics | |
| JP2019531792A (en) | Classifier ensemble for detection of abnormal heart sounds | |
| CN108229441A (en) | A kind of classroom instruction automatic feedback system and feedback method based on image and speech analysis | |
| CN111353338B (en) | Energy efficiency improvement method based on business hall video monitoring | |
| CN109829691B (en) | C/S card punching method and device based on position and deep learning multiple biological features | |
| CN112784239B (en) | Electronic sports operation information management system based on artificial intelligence | |
| CN112037788A (en) | Voice correction fusion technology | |
| Guarin et al. | The effect of improving facial alignment accuracy on the video-based detection of neurological diseases | |
| EP3627505B1 (en) | Real-time speaker identification with diarization | |
| JP2019192092A (en) | Conference support device, conference support system, conference support method, and program | |
| CN111803031A (en) | Non-contact type drug addict relapse monitoring method and system | |
| WO2021217979A1 (en) | Voiceprint recognition method and apparatus, and device and storage medium | |
| CN112182537A (en) | Monitoring method, device, server, system and storage medium | |
| CN111339972A (en) | Face image detection method and device, electronic equipment and storage medium | |
| CN106599764A (en) | Lip-shape-characteristic-based living body determination method and equipment | |
| CN110353703A (en) | Autism based on language paradigm behavioural analysis of repeating the words of others like a parrot assesses apparatus and system | |
| CN116886819B (en) | Multi-dimensional telephone traffic data monitoring method, device and storage medium | |
| Garcia et al. | Development of a Non-contact Cough Recognition System with Face Mask Detection and Distance Assistance using Convolutional Neural Networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180615 |
|
| RJ01 | Rejection of invention patent application after publication |