CN110135800A - A kind of artificial intelligence video interview method and system - Google Patents
A kind of artificial intelligence video interview method and system Download PDFInfo
- Publication number
- CN110135800A CN110135800A CN201910329587.3A CN201910329587A CN110135800A CN 110135800 A CN110135800 A CN 110135800A CN 201910329587 A CN201910329587 A CN 201910329587A CN 110135800 A CN110135800 A CN 110135800A
- Authority
- CN
- China
- Prior art keywords
- interview
- video
- result
- analysis
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000013473 artificial intelligence Methods 0.000 title claims abstract description 28
- 238000004458 analytical method Methods 0.000 claims abstract description 74
- 230000008569 process Effects 0.000 claims abstract description 25
- 230000008901 benefit Effects 0.000 claims abstract description 9
- 238000011156 evaluation Methods 0.000 claims description 49
- 230000008859 change Effects 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 16
- 230000001755 vocal effect Effects 0.000 claims description 14
- 238000010801 machine learning Methods 0.000 claims description 10
- 238000004141 dimensional analysis Methods 0.000 claims description 7
- 239000000284 extract Substances 0.000 claims description 7
- 230000004927 fusion Effects 0.000 claims description 7
- 230000000694 effects Effects 0.000 claims description 5
- 208000019901 Anxiety disease Diseases 0.000 claims description 3
- 230000036506 anxiety Effects 0.000 claims description 3
- 238000013210 evaluation model Methods 0.000 claims description 3
- 238000007500 overflow downdraw method Methods 0.000 claims description 3
- 238000003860 storage Methods 0.000 claims description 3
- 230000005540 biological transmission Effects 0.000 claims description 2
- 239000012141 concentrate Substances 0.000 claims description 2
- 230000002996 emotional effect Effects 0.000 claims description 2
- 230000014759 maintenance of location Effects 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 abstract description 8
- 238000010195 expression analysis Methods 0.000 abstract description 3
- 238000012216 screening Methods 0.000 abstract 1
- 238000006243 chemical reaction Methods 0.000 description 4
- 238000000605 extraction Methods 0.000 description 4
- 238000004088 simulation Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 235000013399 edible fruits Nutrition 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000000354 decomposition reaction Methods 0.000 description 2
- 230000007812 deficiency Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000004321 preservation Methods 0.000 description 2
- 238000007619 statistical method Methods 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 230000002776 aggregation Effects 0.000 description 1
- 238000004220 aggregation Methods 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 230000006998 cognitive state Effects 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000013499 data model Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000002045 lasting effect Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000036651 mood Effects 0.000 description 1
- 238000005086 pumping Methods 0.000 description 1
- 238000012797 qualification Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000014616 translation Effects 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06398—Performance of employee with respect to a job function
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
- G06Q10/105—Human resources
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/26—Speech to text systems
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/72—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for transmitting results of analysis
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Human Resources & Organizations (AREA)
- Strategic Management (AREA)
- Entrepreneurship & Innovation (AREA)
- Physics & Mathematics (AREA)
- Economics (AREA)
- Educational Administration (AREA)
- Development Economics (AREA)
- Acoustics & Sound (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Computational Linguistics (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Game Theory and Decision Science (AREA)
- Signal Processing (AREA)
- Data Mining & Analysis (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The present invention provides a kind of artificial intelligence video interview method and system, the intelligent video interviews method and system by the way of video interview, Expression analysis in interview process is provided by artificial intelligence technology for job hunter, phonetic analysis and analysis for interview question answer, it can be to the ability to express of job hunter, expression, it is automatic with the carry out such as the matching degree in post, accurately, it is comprehensive, efficient assessment, it can either help employing unit and carry out according to comprehensive assessment result the screening of job candidates, also it can pass through the feedback of comprehensive assessment, job hunter is allowed to fully realize the advantage and disadvantage in oneself interview process, carry out specific aim raising, the job hunter that helps improves self diathesis, increase interview success rate.
Description
Technical field
The present invention relates to a kind of field of artificial intelligence, in particular to a kind of to carry out video face using artificial intelligence system
The method and system of examination.
Background technique
Job hunting employment is a key node of everyone life, is directly related to the future of job hunter, has every year more
Carry out more university students to graduate into job hunting market, and in job hunting process, the quality of interview result is that job hunting is successfully crucial
Place.For the university student just to have graduated, missing interview experience and interview are taught before formally interviewing in face of enterprise, mostly
The interview experience of number job hunter is all in each interview kind accumulation, this also has led to job hunter can be because interview performance
The not perfect working opportunity missed, there has been no the products that analysis is interviewed to job hunter currently on the market.Mostly predominantly under line
Interview teach.
The defect and deficiency of the prior art:
(1) efficiency: the interview guidance under line needs the person of guidance and job hunter while having time enough that can complete, and occupies that
This time.Although video interview can stay indoors using the computer for being connected to internet, pass through video camera and headset
Mode into voice, video, text carries out instant communication exchange, but needs to appoint the same time, waste of manpower and when
Between.
(2) effect: different interviewers has different evaluations, same interviewer to problem, because different workplaces passes through
It tests, interview under the cognitive state at technical ability and interview scene and also have different judgements.Additionally while from international manpower resources and
The country competency(that Talent Management field introduces is translated as competency, quality, ability) concept, but from different translations
From the point of view of application with shortage knowledge system, generates very more different evaluation criterias and can hardly be avoided naturally, eventually lead to assessment
The uncertainty of effect and it is unfavorable for lasting scientific research.
Recently as the fast development of artificial intelligence (AI) technology, interview is simulated using AI technology and then to interview people
Member carries out subsequent guidance and is possibly realized.
Summary of the invention
In view of the foregoing deficiencies of prior art, the purpose of the present invention is to provide it is a kind of using artificial intelligence technology come
The method of video interview.
In order to achieve the above objects and other related objects, the present invention provides a kind of artificial intelligence video interview method, including
Following steps:
1) interview question is given according to application post；
2) it allows applicant to answer given problem, and acquires the interview video of answer；
3) after converting text information for the audio-frequency information for interviewing video identification, to content of text messages and the progress of application post
It is calculated with degree, exports matching degree calculated result；
4) audio-frequency information in video will be interviewed and carries out vocal print evaluation, export vocal print evaluation result；
5) key frame that given quantity is extracted from interview video, identifies the expression of each key frame, and combine all key frames
Expression Recognition result carry out interview performance evaluation, export evaluation result；
6) by step 3), 4), 5) in evaluation result the relevant overall merit in post is carried out by comprehensive evaluation model, export
The comprehensive evaluation result of current interview topic；
7) judge whether interview terminates, if being not finished, repeat step 1) to 6), if interview terminates, count the every problem of interviewee
Comprehensive evaluation result, and calculated according to the importance of application post feature and every problem and export interview total score；
8) according to step 3) to 7) output result generate marking report, interview analysis report and to the interview of interviewee it is anti-
Feedback.
Preferably, step 2 specifically comprises the following steps: that, for given interview question, interviewee can be by local or remote
Journey video answers interview topic online, can also upload answer video, be answered.
Preferably, step 4) specifically comprises the following steps: to extract the audio-frequency information in interview video, and is directed to interviewee's sound
Word speed, volume, tone, anxiety, the smooth characteristic shape of interviewee is successively assessed in the variation of sound by end-to-end machine learning model
At the assessment result of five dimensions, and to each dimensional analysis, the Oral Activities of comprehensive assessment interviewee.
Preferably, step 5) specifically comprises the following steps: the expression shape change in the interview video to interviewee, is given
The key frame of quantity intercepts, and identifies the expression of interviewee in each key frame, and the requirement according to different posies to expression shape change,
By different Weighted Fusion methods, comprehensive all key frames are as a result, calculate the expression and emotional change of interviewee.
Preferably, identify the expression of interviewee as a result, interview expression knot by end-to-end machine learning model in step 5)
Fruit includes: happy, exciting, nervous, dull, dejected, self-confident, angry, face value.
Preferably, step 6) specifically comprises the following steps: according to claim 3), 4), 5) processing obtain multi dimensional analysis
As a result, carrying out post comprehensive assessment, fusion interviewee assesses in the expression of current question, oracy assessment, Post Match Degree
As a result, using machine learning method generation end to end according to scoring model and combination post feature, to interviewee, this is interviewed back
The comprehensive assessment answered.
Preferably, in step 8), analysis report exports interview result and interview explanation and goes out in interviewee's interview process
Existing problem and advantage, which is concentrated, to be illustrated, is presented with report manner, and report content includes that sound assessment result is commented with suggestion, expression
Estimate result and suggestion, answer content assessment result and suggestion and Comprehensive analysis results and suggestion.
This patent also discloses a kind of system that can be realized above-mentioned artificial intelligence video interview method comprising: interview
Video acquisition module, the interview video acquisition module are used for interviewee's display surface examination question mesh, and the answer to interviewee
Cheng Jinhang videograph and transmission；Video flowing decomposing module, the video flowing decomposing module are used for the audio in videograph
Information is decomposed, and resolves into voice messaging, and voice messaging is converted to text information；Content evaluation module, the content
Evaluation module is used to text information and post carrying out the matching analysis comparison, obtains text analyzing result and stores to server
It is interior；Vocal print evaluation module, the vocal print evaluation module are used to carry out voice messaging multi dimensional analysis and will analysis result storages
In server；Key frame of video extracts identification module, and the key frame of video extracts identification module and is used for face in interview video
The expression shape change point key frame of examination person is extracted, and carries out Expression Recognition, recognition result combination post class to each key frame
Type is stored to the interview expression shape change comprehensive analysis in interviewee's interview process, and by analysis result to server；Post is comprehensive
Analysis module, the post comprehensive analysis module be used for text evaluation module analysis result, vocal print evaluation module analysis result,
Video evaluations module analysis result carries out the comprehensive evaluation result that comprehensive analysis obtains current question；Interview result fusion assessment mould
Block, the interview result fusion evaluation module are used for after interview, count the comprehensive evaluation result of the every problem of interviewee, and
It is calculated according to the importance of application post feature and every problem and exports interview total score；Interview report generation module, the interview
Report generation module is to all interview status analyzings of above-mentioned post comprehensive analysis module as a result, progress united analysis, obtains
Interview reporting analysis results.
Preferably, the video flowing decomposing module includes audio processing modules and speech recognition module, the audio processing
For proposing and handling the audio-frequency unit interviewed in video, the speech recognition module will interview the voice in video to be believed module
Breath is converted into recognizable text.
Preferably, the video flowing decomposing module further includes analysis module, and the analysis module is used for face
Examination person's video carries out secondary treatment, is convenient for later period Expression Recognition.
As described above, this artificial intelligent video interview method and system has the advantages that the artificial intelligent video
Video information is separately converted to text information, voiceprint and picture and is believed by interview method by the way of video simulation interview
Three kinds of forms are ceased, the interview process of interviewee is analyzed from different angles.The artificial video simulation interview exam system, it is convenient
Fast, job hunter can operate whenever and wherever possible, do not need special messenger and carry out preservation operation, which passes through artificial intelligence skill
Art provides Expression analysis, phonetic analysis and the analysis for interview question answer in interview process for job hunter, can allow job hunting
Person fully realizes the advantage and disadvantage in oneself interview process, carries out specific aim raising, and help job hunter improves interview success rate.
Detailed description of the invention
Fig. 1 is the system architecture diagram of artificial intelligence of the present invention interview.
Fig. 2 is the flow chart of artificial intelligence of the present invention interview.
Fig. 3 is key frame of video of the present invention extraction and analysis flow chart diagram.
Fig. 4 is that video of the present invention turns voice module flow chart.
Fig. 5 is audio conversion text module flow chart of the present invention.
Fig. 6 is text evaluation module flow diagram of the present invention.
Fig. 7 is vocal print evaluation module flow chart of the present invention.
Specific embodiment
Embodiments of the present invention are illustrated by particular specific embodiment below, those skilled in the art can be by this explanation
Content disclosed by book is understood other advantages and efficacy of the present invention easily.
Fig. 1 is please referred to Fig. 7.It should be clear that only to cooperate specification revealed depicted in this specification institute accompanying drawings
Content is not intended to limit the invention enforceable qualifications, therefore does not have so that those skilled in the art understands and reads
Technical essential meaning, rather than to limit the scope of the invention, relativeness is altered or modified, in no reality
Under qualitative change more technology contents, when being also considered as the enforceable scope of the present invention.
As shown in Figure 1, this patent discloses a kind of artificial intelligence video interview system, wherein artificial intelligence video interview system
System includes interview video acquisition module 1, video flowing decomposing module 13, content evaluation module 6, vocal print evaluation module 7, key frame pumping
Take identification module 5, expression evaluation module 9, post comprehensive analysis module 10, interview result fusion evaluation module 11 and interview report
Generation module 12.It interviews video acquisition module 1 to be used for interviewee's display surface examination question mesh, and the answer process of interviewee is carried out
Videograph, interview video acquisition module 1 can for mobile phone, computer or other can carry out the communication terminal of man-machine call.
In the interview videograph that video flowing decomposing module decomposing module 13 is used to acquire interview video acquisition module 1
Audio-frequency information is decomposed, and is resolved into voice messaging, and voice messaging is converted to text information, is mainly included speech recognition
Module 2, audio processing modules 3, video processing module 4, wherein speech recognition module 2 will be for that will interview the voice messaging in video
It is converted into text information and is stored.The phonological component interviewed in video is extracted into mp3 file by audio processing modules user
Convenient for later analysis.Video processing module 4, which is used to that the video information interviewed in video to be extracted and be decomposed, does video optimized decomposition,
Key Frame Extraction identification module 5 is used to mark and intercept identification to interviewee's expression shape change point in interview video, assesses interviewee
Expression shape change in interview process is simultaneously assessed according to post information.
Content evaluation module 6 is used to carry out voice messaging analysis and analysis result is stored in server.Vocal print is commented
Module 7 is estimated for carrying out audio analysis to the audio-frequency information decomposed in audio processing modules 3, obtains audio analysis as a result, expression
9 user of evaluation module is for statistical analysis to all analysis results in Expression Recognition module 8, post comprehensive analysis module 10
For text information analysis and assessment result, the voiceprint analysis and assessment result, expression key feature to storage in the server
Analysis and assessment result carries out comprehensive analysis, obtains the level of aggregation analysis of the secondary answer.Interview result, which merges evaluation module 11, to be used
In after interview, the comprehensive evaluation result of the every problem of interviewee is counted, and according to the weight of application post feature and every problem
The property wanted calculates and exports interview total score.12 user of report generation module is interviewed to carry out multiple interview answer Comprehensive analysis results
It analyzes again, changes the performance level and advantage and disadvantage of this interview of interviewee by machine learning model data assessment, and provide conjunction
Reason is suggested.
As shown in Fig. 2, a kind of above-mentioned artificial intelligence video interview method includes the following steps: first through video acquisition mould
Block 1 obtains interview topic, starts to carry out video interview after obtaining interview topic, by interview is video information in interview process
Plug-flow is to server.Judge whether interview terminates, if being not finished, continue video interview, after interviewing, will store
Video information be divided into three kinds of processing modes and handled.For given interview question, interviewee can pass through Local or Remote
Video answers interview topic online, can also upload answer video, be answered.
The voice messaging in video information can be extracted by speech recognition module 2, and converts text envelope for voice messaging
Breath, there may be mistake since voice is converted into text information, can first judge whether the text in text information needs
It modifies, can directly modify if necessary to modify, be such as not required to modify or can directly pass through content evaluation after the completion of modifying
Module 6 carries out analysis comparison to text information.
The audio-frequency information in videograph can be interviewed by 3 pairs of audio processing modules simultaneously to extract, and pass through vocal print
The audio-frequency information of 7 pairs of evaluation module decomposition carries out audio analysis, obtains audio analysis result.Extract the audio letter in interview video
Breath, and the word speed, volume, sound of interviewee is successively assessed in the variation for being directed to interviewee's sound by end-to-end machine learning model
It adjusts, is nervous, the assessment result of five dimensions of smooth characteristic formation, and to each dimensional analysis, the spoken language of comprehensive assessment interviewee
Ability to express.
And video processing module 4 interview videograph can be done in video information extract and decompose and do video optimized point
Solution is being used to mark and intercept identification to interviewee's expression shape change point in interview video by Key Frame Extraction identification module 5, commented
Estimate the expression shape change in interviewee's interview process and is assessed according to post information.To the expression in the interview video of interviewee
Variation carries out the key frame interception of given quantity, identifies the expression of interviewee in each key frame, and according to different posies to table
The requirement of end of love, by different Weighted Fusion methods, comprehensive all key frames are as a result, calculate the expression and mood of interviewee
Variation.The expression of interviewee can be identified by end-to-end machine learning model as a result, interview expression result includes but is not limited to: being opened
The heart, excitement, anxiety, dull, dejected, self-confident, angry, face value.
After being fully completed etc. above-mentioned three kinds of analysis modes (text information, voiceprint and pictorial information), pass through comprehensive point
Analysis module 10 carries out the relevant overall merit in post by comprehensive evaluation model, exports the overall merit knot of current interview topic
Fruit obtains multi dimensional analysis by processing as a result, carrying out post comprehensive assessment, and fusion interviewee comments in the expression of current question
Estimate, oracy assessment, Post Match Degree is as a result, use machine learning end to end according to scoring model and combination post feature
Method generates the comprehensive assessment to this interview answer of interviewee.
If interview terminates, the comprehensive evaluation result of the every problem of interviewee is counted, and according to application post feature and every problem
Importance calculate and export interview total score；Finally output result generates marking report, interview analysis report and to interviewee
Interview feedback.Analysis report exports interview result and interview explanation and to the problem and advantage in interviewee's interview process
Concentrate explanation, presented with report manner, report content include sound assessment result and suggestion, expression assessment result and suggestion,
Answer content assessment result and suggestion and Comprehensive analysis results and suggestion.On content in analysis report includes but is not limited to
State several contents.
As shown in figure 3, being Key Frame Extraction identification process, in the process, interview video is obtained first from server
Information is then randomly selected 1000 therein and is used as and divide then using every frame picture in maven technique intercepts video information
Sample is analysed, then the information in picture is analyzed by the Expression Recognition algorithm in Expression Recognition module 7, is passing through statistics
Parser obtains analysis as a result, and storing the analysis data literal for analyzing result into server.
As shown in figure 4, the process to convert video information to voice messaging and storing: first and first from service
Interviewee is obtained in device to interview video and convert, and the voice messaging in video information is generated into mp3 file, finally by mp3 file
In storing data library.
As shown in figure 5, to convert audio-frequency information to the process of text information: it has been turned by server acquisition first
Audio file is converted to text information with TTS technology, finally by the text envelope after conversion by the audio information files changed
In breath, text data store to server.It, can be directly to the information of conversion due to being easy to appear mistake in the conversion process
It modifies, and is stored to server.
As shown in fig. 6, being the flow chart of text analysis model: it obtains inverted good text by server first
Then the information content is compared with data model, show that text answers are analyzed as a result, and will analyze result by text data side
Formula is stored to server.
As shown in fig. 7, being the flow chart of voiceprint analysis module, pass through the asynchronous acquisition server sound of server background first
Frequency evidence then by audio file cutting process, and every section audio file process is analyzed, final to statistically analyze audio parsing knot
Fruit, and the structure of statistical analysis is stored into server.
Above steps all can greatly improve the speed of service of system by backstage asynchronous execution in this way.
Video information is separately converted to text by the way of video simulation interview by the artificial intelligent video interview method
Three kinds of information, voiceprint and pictorial information forms, from different angles analyze the interview process of interviewee.AI view
Frequency simulation interview exam system, convenient and efficient, job hunter can operate whenever and wherever possible, do not need special messenger and carry out preservation operation, the system
Expression analysis in interview process, phonetic analysis are provided by AI technology for job hunter with method and for interview question answer
Analysis can allow job hunter to fully realize the advantage and disadvantage in oneself interview process, carry out specific aim raising, and help job hunter improves face
Try success rate.So the present invention effectively overcomes various shortcoming in the prior art and has high industrial utilization value.
The above-described embodiments merely illustrate the principles and effects of the present invention, and is not intended to limit the present invention.It is any ripe
The personage for knowing this technology all without departing from the spirit and scope of the present invention, carries out modifications and changes to above-described embodiment.Cause
This, institute is complete without departing from the spirit and technical ideas disclosed in the present invention by those of ordinary skill in the art such as
At all equivalent modifications or change, should be covered by the claims of the present invention.
Claims (10)
1. a kind of intelligent video interviews method, which is characterized in that it includes the following steps:
1) interview question is given according to application post；
2) it allows applicant to answer given problem, and acquires the interview video of answer；
3) after converting text information for the audio-frequency information for interviewing video identification, to content of text messages and the progress of application post
It is calculated with degree, exports matching degree calculated result；
4) audio-frequency information in video will be interviewed and carries out vocal print evaluation, export vocal print evaluation result；
5) key frame that given quantity is extracted from interview video, identifies the expression of each key frame, and combine all key frames
Expression Recognition result carry out interview performance evaluation, export evaluation result；
6) by step 3), 4), 5) in evaluation result the relevant overall merit in post is carried out by comprehensive evaluation model, export
The comprehensive evaluation result of current interview topic；
7) judge whether interview terminates, if being not finished, repeat step 1) to 6), if interview terminates, count the every problem of interviewee
Comprehensive evaluation result, and calculated according to the importance of application post feature and every problem and export interview total score；
8) according to step 3) to 7) output result generate marking report, interview analysis report and to the interview of interviewee it is anti-
Feedback.
2. artificial intelligence video interview method according to claim 1, which is characterized in that step 2 specifically includes following step
Rapid: for given interview question, interviewee can answer online interview topic by Local or Remote video, can also upload answer
Video is answered.
3. artificial intelligence video interview method according to claim 1, which is characterized in that step 4) specifically includes following step
It is rapid: extract interview video in audio-frequency information, and for interviewee's sound variation by end-to-end machine learning model successively
The word speed of interviewee, volume, tone, anxiety, smooth characteristic are assessed, forms the assessment result of multiple dimensions, and by each dimension
Analysis result integrates, and assesses the Oral Activities of interviewee.
4. artificial intelligence video interview method according to claim 1, which is characterized in that step 5) specifically includes following step
It is rapid: to the expression shape change in the interview video of interviewee, to carry out the key frame interception of given quantity, and pass through end-to-end engineering
Learning method identifies the expression of interviewee in each key frame, the requirement according to different posies to expression shape change, by Weighted Fusion
Method integrates all key frames as a result, assessing the expression and emotional change of interviewee, provides expression assessment result.
5. artificial intelligence video interview method according to claim 14, which is characterized in that end-to-end machine learning method
Identify the expression of interviewee, interview Expression Recognition result includes: happy, exciting, nervous, dull, dejected, self-confident, angry, face
Value.
6. artificial intelligence video interview method according to claim 1, which is characterized in that step 6) specifically includes following step
It is rapid: according to claim 3), 4), 5) processing obtain multi dimensional analysis as a result, carry out post comprehensive assessment, method particularly includes: knot
Post feature is closed, expression assessment result, spoken energy using the fusion interviewee of machine learning method end to end in current question
Force estimation result, Post Match Degree assessment result generate the comprehensive assessment to this interview answer of interviewee.
7. artificial intelligence video interview method according to claim 1, which is characterized in that in step 8), analysis report is defeated
Interview result and interview illustrate and concentrate the problem in interviewee's interview process and advantage to illustrate out, are in report manner
Existing, report content includes sound assessment result and suggestion, expression assessment result and suggestion, answer content assessment result and build
View and Comprehensive analysis results and suggestion.
8. a kind of artificial intelligence video interview system, characterized in that it comprises:
Video acquisition module is interviewed, the interview video acquisition module is used for interviewee's display surface examination question mesh, and to interviewee
Answer process carry out videograph and transmission；
Video flowing decomposing module, the video flowing decomposing module are decomposed for decomposing to the audio-frequency information in videograph
Text information is converted at voice messaging and audio-frequency information, and by voice messaging；
Content evaluation module, the content evaluation module are used to text information and post carrying out the matching analysis comparison, obtain text
This analysis result is simultaneously stored to server；
Vocal print evaluation module, the vocal print evaluation module are used to carry out audio-frequency information multi dimensional analysis and will analysis result storages
In server；
Key frame of video extracts identification module, and the key frame of video extracts identification module and is used for interviewee in interview video
Expression shape change point key frame is extracted, and carries out Expression Recognition, recognition result combination post type, opposite to each key frame
Interview expression shape change comprehensive analysis in examination person's interview process, and analysis result is stored to server；
Post comprehensive analysis module, the post comprehensive analysis module are used to assess text evaluation module analysis result, vocal print
Module analysis result, video evaluations module analysis result carry out the comprehensive evaluation result that comprehensive analysis obtains current question；
Interview result merges evaluation module, and the interview result fusion evaluation module is used for after interview, counts interviewee
The comprehensive evaluation result of every problem, and calculated according to the importance of application post feature and every problem and export interview total score；
Report generation module is interviewed, the interview report generation module is all interview hilllocks to above-mentioned post comprehensive analysis module
Position analysis obtains interview reporting analysis results as a result, progress united analysis.
9. artificial intelligence video interview system according to claim 8, which is characterized in that the video flowing decomposing module packet
Audio processing modules and speech recognition module are included, the audio-frequency unit that the audio processing modules are used to interview in video proposes simultaneously
The voice messaging interviewed in video is converted recognizable text by processing, the speech recognition module.
10. artificial intelligence video interview system according to claim 8, which is characterized in that the video flowing decomposing module
It further include analysis module, the analysis module is used to interviewee's video carrying out secondary treatment, is convenient for later period expression
Identification.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910329587.3A CN110135800A (en) | 2019-04-23 | 2019-04-23 | A kind of artificial intelligence video interview method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910329587.3A CN110135800A (en) | 2019-04-23 | 2019-04-23 | A kind of artificial intelligence video interview method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110135800A true CN110135800A (en) | 2019-08-16 |
Family
ID=67570882
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910329587.3A Withdrawn CN110135800A (en) | 2019-04-23 | 2019-04-23 | A kind of artificial intelligence video interview method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110135800A (en) |
Cited By (27)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110705958A (en) * | 2019-09-11 | 2020-01-17 | 何敏 | AI-based simulation interview platform |
| CN110782221A (en) * | 2019-09-19 | 2020-02-11 | 丁玥 | Intelligent interview evaluation system and method |
| CN110827796A (en) * | 2019-09-23 | 2020-02-21 | 平安科技（深圳）有限公司 | Interviewer determination method and device based on voice, terminal and storage medium |
| CN110874716A (en) * | 2019-09-23 | 2020-03-10 | 平安科技（深圳）有限公司 | Interview evaluation method and device, electronic equipment and storage medium |
| CN111144853A (en) * | 2020-01-16 | 2020-05-12 | 南京思特齐科技有限公司 | Multi-mode recognition-based interviewing system for human resources |
| CN111209817A (en) * | 2019-12-25 | 2020-05-29 | 深圳壹账通智能科技有限公司 | Assessment method, device and equipment based on artificial intelligence and readable storage medium |
| CN111222854A (en) * | 2020-01-15 | 2020-06-02 | 中国平安人寿保险股份有限公司 | Interview method, device and equipment based on interview robot and storage medium |
| CN111259862A (en) * | 2020-02-20 | 2020-06-09 | 深圳壹账通智能科技有限公司 | User information analysis method and system |
| CN111275401A (en) * | 2020-01-20 | 2020-06-12 | 上海近屿智能科技有限公司 | Intelligent interviewing method and system based on position relation |
| CN111401828A (en) * | 2020-02-28 | 2020-07-10 | 上海近屿智能科技有限公司 | Dynamic intelligent interviewing method, device and equipment for strengthening sorting and computer storage medium |
| CN111429267A (en) * | 2020-03-26 | 2020-07-17 | 深圳壹账通智能科技有限公司 | Face examination risk control method and device, computer equipment and storage medium |
| CN111540440A (en) * | 2020-04-23 | 2020-08-14 | 深圳市镜象科技有限公司 | Psychological examination method, device, equipment and medium based on artificial intelligence |
| CN111612324A (en) * | 2020-05-15 | 2020-09-01 | 深圳看齐信息有限公司 | Multi-dimensional assessment method based on oral English examination |
| CN111612352A (en) * | 2020-05-22 | 2020-09-01 | 北京易华录信息技术股份有限公司 | Student expression ability assessment method and device |
| CN111723180A (en) * | 2020-06-08 | 2020-09-29 | 中国建设银行股份有限公司 | Interviewing method and device |
| CN112001628A (en) * | 2020-08-24 | 2020-11-27 | 河北冀联人力资源服务集团有限公司 | Recommendation method of intelligent interview video |
| CN112261485A (en) * | 2020-09-11 | 2021-01-22 | 五八到家有限公司 | Audio and video processing method, server, terminal and medium |
| CN112884326A (en) * | 2021-02-23 | 2021-06-01 | 无锡爱视智能科技有限责任公司 | Video interview evaluation method and device based on multi-modal analysis and storage medium |
| CN113095165A (en) * | 2021-03-23 | 2021-07-09 | 北京理工大学深圳研究院 | Simulation interview method and device for perfecting interview performance |
| CN113269541A (en) * | 2021-07-21 | 2021-08-17 | 深圳市启程教育科技有限公司 | Talent online interview data analysis system and method based on Internet |
| CN113327104A (en) * | 2021-08-04 | 2021-08-31 | 深圳市启程教育科技有限公司 | Talent interview screening system and method based on image processing |
| CN113469542A (en) * | 2021-07-09 | 2021-10-01 | 上海以善健康管理有限公司 | Self-iteration expression testing system based on language processing and image recognition |
| CN113672698A (en) * | 2021-08-01 | 2021-11-19 | 北京网聘咨询有限公司 | Intelligent interviewing method, system, equipment and storage medium based on expression analysis |
| CN114220055A (en) * | 2021-12-15 | 2022-03-22 | 中国平安人寿保险股份有限公司 | Method, device, computer equipment and storage medium for assisting user in interview |
| CN114418366A (en) * | 2022-01-06 | 2022-04-29 | 北京博瑞彤芸科技股份有限公司 | Data processing method and device for intelligent cloud interview |
| CN115829533A (en) * | 2023-02-15 | 2023-03-21 | 成都萌想科技有限责任公司 | Intelligent online interviewing method, system, equipment and storage medium |
| CN116862277A (en) * | 2023-05-09 | 2023-10-10 | 北京新东方迅程网络科技有限公司 | User's selection information processing method, device, electronic equipment and readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106663383A (en) * | 2014-06-23 | 2017-05-10 | 因特维欧研发股份有限公司 | Method and system for analyzing subjects |
| CN107705090A (en) * | 2017-09-27 | 2018-02-16 | 重庆市智汇人才开发有限公司 | Personnel recruitment system and method |
| CN107862598A (en) * | 2017-09-30 | 2018-03-30 | 平安普惠企业管理有限公司 | Long-range the interview measures and procedures for the examination and approval, server and readable storage medium storing program for executing |
| CN109543007A (en) * | 2018-10-16 | 2019-03-29 | 深圳壹账通智能科技有限公司 | Put question to data creation method, device, computer equipment and storage medium |
-
2019
- 2019-04-23 CN CN201910329587.3A patent/CN110135800A/en not_active Withdrawn
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106663383A (en) * | 2014-06-23 | 2017-05-10 | 因特维欧研发股份有限公司 | Method and system for analyzing subjects |
| CN107705090A (en) * | 2017-09-27 | 2018-02-16 | 重庆市智汇人才开发有限公司 | Personnel recruitment system and method |
| CN107862598A (en) * | 2017-09-30 | 2018-03-30 | 平安普惠企业管理有限公司 | Long-range the interview measures and procedures for the examination and approval, server and readable storage medium storing program for executing |
| CN109543007A (en) * | 2018-10-16 | 2019-03-29 | 深圳壹账通智能科技有限公司 | Put question to data creation method, device, computer equipment and storage medium |
Cited By (39)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110705958A (en) * | 2019-09-11 | 2020-01-17 | 何敏 | AI-based simulation interview platform |
| CN110782221A (en) * | 2019-09-19 | 2020-02-11 | 丁玥 | Intelligent interview evaluation system and method |
| CN110827796B (en) * | 2019-09-23 | 2024-05-24 | 平安科技（深圳）有限公司 | Interviewer judging method and device based on voice, terminal and storage medium |
| CN110827796A (en) * | 2019-09-23 | 2020-02-21 | 平安科技（深圳）有限公司 | Interviewer determination method and device based on voice, terminal and storage medium |
| CN110874716A (en) * | 2019-09-23 | 2020-03-10 | 平安科技（深圳）有限公司 | Interview evaluation method and device, electronic equipment and storage medium |
| CN111209817A (en) * | 2019-12-25 | 2020-05-29 | 深圳壹账通智能科技有限公司 | Assessment method, device and equipment based on artificial intelligence and readable storage medium |
| CN111222854A (en) * | 2020-01-15 | 2020-06-02 | 中国平安人寿保险股份有限公司 | Interview method, device and equipment based on interview robot and storage medium |
| CN111222854B (en) * | 2020-01-15 | 2024-04-09 | 中国平安人寿保险股份有限公司 | Interview robot-based interview method, interview device, interview equipment and storage medium |
| CN111144853A (en) * | 2020-01-16 | 2020-05-12 | 南京思特齐科技有限公司 | Multi-mode recognition-based interviewing system for human resources |
| CN111275401A (en) * | 2020-01-20 | 2020-06-12 | 上海近屿智能科技有限公司 | Intelligent interviewing method and system based on position relation |
| CN111275401B (en) * | 2020-01-20 | 2023-10-24 | 上海近屿智能科技有限公司 | Intelligent interview method and system based on position relation |
| CN111259862A (en) * | 2020-02-20 | 2020-06-09 | 深圳壹账通智能科技有限公司 | User information analysis method and system |
| WO2021164466A1 (en) * | 2020-02-20 | 2021-08-26 | 深圳壹账通智能科技有限公司 | Method and system for analyzing user information |
| CN111401828A (en) * | 2020-02-28 | 2020-07-10 | 上海近屿智能科技有限公司 | Dynamic intelligent interviewing method, device and equipment for strengthening sorting and computer storage medium |
| CN111429267A (en) * | 2020-03-26 | 2020-07-17 | 深圳壹账通智能科技有限公司 | Face examination risk control method and device, computer equipment and storage medium |
| WO2021190086A1 (en) * | 2020-03-26 | 2021-09-30 | 深圳壹账通智能科技有限公司 | Face-to-face examination risk control method and apparatus, computer device, and storage medium |
| CN111540440A (en) * | 2020-04-23 | 2020-08-14 | 深圳市镜象科技有限公司 | Psychological examination method, device, equipment and medium based on artificial intelligence |
| CN111540440B (en) * | 2020-04-23 | 2021-01-15 | 深圳市镜象科技有限公司 | Psychological examination method, device, equipment and medium based on artificial intelligence |
| CN111612324B (en) * | 2020-05-15 | 2021-02-19 | 深圳看齐信息有限公司 | Multi-dimensional assessment method based on oral English examination |
| CN111612324A (en) * | 2020-05-15 | 2020-09-01 | 深圳看齐信息有限公司 | Multi-dimensional assessment method based on oral English examination |
| CN111612352A (en) * | 2020-05-22 | 2020-09-01 | 北京易华录信息技术股份有限公司 | Student expression ability assessment method and device |
| CN111612352B (en) * | 2020-05-22 | 2024-06-11 | 北京易华录信息技术股份有限公司 | Student expression capability assessment method and device |
| CN111723180A (en) * | 2020-06-08 | 2020-09-29 | 中国建设银行股份有限公司 | Interviewing method and device |
| CN112001628A (en) * | 2020-08-24 | 2020-11-27 | 河北冀联人力资源服务集团有限公司 | Recommendation method of intelligent interview video |
| CN112261485A (en) * | 2020-09-11 | 2021-01-22 | 五八到家有限公司 | Audio and video processing method, server, terminal and medium |
| CN112884326A (en) * | 2021-02-23 | 2021-06-01 | 无锡爱视智能科技有限责任公司 | Video interview evaluation method and device based on multi-modal analysis and storage medium |
| CN113095165A (en) * | 2021-03-23 | 2021-07-09 | 北京理工大学深圳研究院 | Simulation interview method and device for perfecting interview performance |
| CN113469542A (en) * | 2021-07-09 | 2021-10-01 | 上海以善健康管理有限公司 | Self-iteration expression testing system based on language processing and image recognition |
| CN113269541A (en) * | 2021-07-21 | 2021-08-17 | 深圳市启程教育科技有限公司 | Talent online interview data analysis system and method based on Internet |
| CN113672698A (en) * | 2021-08-01 | 2021-11-19 | 北京网聘咨询有限公司 | Intelligent interviewing method, system, equipment and storage medium based on expression analysis |
| CN113672698B (en) * | 2021-08-01 | 2024-05-24 | 北京网聘信息技术有限公司 | Intelligent interview method, system, equipment and storage medium based on expression analysis |
| CN113327104A (en) * | 2021-08-04 | 2021-08-31 | 深圳市启程教育科技有限公司 | Talent interview screening system and method based on image processing |
| CN114220055B (en) * | 2021-12-15 | 2024-04-05 | 中国平安人寿保险股份有限公司 | Method, device, computer equipment and storage medium for assisting user interview |
| CN114220055A (en) * | 2021-12-15 | 2022-03-22 | 中国平安人寿保险股份有限公司 | Method, device, computer equipment and storage medium for assisting user in interview |
| CN114418366B (en) * | 2022-01-06 | 2022-08-26 | 北京博瑞彤芸科技股份有限公司 | Data processing method and device for intelligent cloud interview |
| CN114418366A (en) * | 2022-01-06 | 2022-04-29 | 北京博瑞彤芸科技股份有限公司 | Data processing method and device for intelligent cloud interview |
| CN115829533B (en) * | 2023-02-15 | 2023-04-18 | 成都萌想科技有限责任公司 | Intelligent online interviewing method, system, equipment and storage medium |
| CN115829533A (en) * | 2023-02-15 | 2023-03-21 | 成都萌想科技有限责任公司 | Intelligent online interviewing method, system, equipment and storage medium |
| CN116862277A (en) * | 2023-05-09 | 2023-10-10 | 北京新东方迅程网络科技有限公司 | User's selection information processing method, device, electronic equipment and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110135800A (en) | A kind of artificial intelligence video interview method and system | |
| US8676586B2 (en) | Method and apparatus for interaction or discourse analytics | |
| CN103151042B (en) | Full-automatic oral evaluation management and points-scoring system and methods of marking thereof | |
| Bollen et al. | SMS-based discussions-technology enhanced collaboration for a literature course | |
| CN109359215A (en) | Video intelligent method for pushing and system | |
| CN110717018A (en) | Industrial equipment fault maintenance question-answering system based on knowledge graph | |
| CN102708866A (en) | Semantic-computing-based interaction system and method for person with hearing or language disorder | |
| CN108648535A (en) | A kind of tutoring system and its operation method based on the mobile terminals VR technology | |
| CN111027584A (en) | Classroom behavior identification method and device | |
| CN110458591A (en) | Advertising information detection method, device and computer equipment | |
| KR20210001419A (en) | User device, system and method for providing interview consulting service | |
| WO2022012605A1 (en) | Pre-trained deep neural network model-based user churn prediction system | |
| CN112651714A (en) | Interview evaluation method and system based on multi-mode information | |
| CN111048075A (en) | Intelligent customer service system and intelligent customer service robot | |
| CN116010569A (en) | Online answering method, system, electronic equipment and storage medium | |
| CN110675292A (en) | Child language ability evaluation method based on artificial intelligence | |
| Jain et al. | Student’s Feedback by emotion and speech recognition through Deep Learning | |
| WO2021135140A1 (en) | Word collection method matching emotion polarity | |
| JP2017167308A (en) | Program, device and method capable of estimating degree of contribution of participant on the basis of important word | |
| CN116911869A (en) | Intelligent customer service system based on artificial intelligence and method thereof | |
| CN117252259A (en) | Deep learning-based natural language understanding method and AI teaching aid system | |
| CN112506405B (en) | Artificial intelligent voice large screen command method based on Internet supervision field | |
| Gweon et al. | Predicting idea co-construction in speech data using insights from sociolinguistics | |
| CN114138947A (en) | Text processing method and device | |
| Liu | Informatization of Constructive English Learning Platform Based on Improved GLR Algorithm |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20191114 Address after: 200030 Xiaomuqiao Road 528, 201, 203, Xuhui District, Shanghai Applicant after: Shanghai Jinyu Intelligent Technology Co., Ltd. Address before: 210000 Building 3, Guorui Building, No. 2, 359 Jiangdong Middle Road, Jianye District, Nanjing City, Jiangsu Province Applicant before: Nanjing Grape Credit Information Technology Co., Ltd. |
|
| WW01 | Invention patent application withdrawn after publication | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20190816 |