CN105095069B - A kind of artificial intelligence response system detection method and system - Google Patents

A kind of artificial intelligence response system detection method and system Download PDF

Info

Publication number
CN105095069B
CN105095069B CN201510346495.8A CN201510346495A CN105095069B CN 105095069 B CN105095069 B CN 105095069B CN 201510346495 A CN201510346495 A CN 201510346495A CN 105095069 B CN105095069 B CN 105095069B
Authority
CN
China
Prior art keywords
test
carousel
answer
artificial intelligence
case
Prior art date
Application number
CN201510346495.8A
Other languages
Chinese (zh)
Other versions
CN105095069A (en
Inventor
孙胜方
王昌
Original Assignee
北京京东尚科信息技术有限公司
北京京东世纪贸易有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京京东尚科信息技术有限公司, 北京京东世纪贸易有限公司 filed Critical 北京京东尚科信息技术有限公司
Priority to CN201510346495.8A priority Critical patent/CN105095069B/en
Publication of CN105095069A publication Critical patent/CN105095069A/en
Application granted granted Critical
Publication of CN105095069B publication Critical patent/CN105095069B/en

Links

Abstract

The present invention discloses a kind of artificial intelligence response system detection method and system, including:Acquisition include at least one problem and with the associated multiple carousel answers of each problem;The problem of obtaining test case, selecting to be associated with the test case is as test problem and exports;Obtaining artificial intelligence response system, art is as test words art to the test problem, if the test words art and one of them described associated carousel answer matches of test problem, judge that the testing case passes through.The present invention by obtain a problem and with the relevant multiple carousel answers of the problem, and when test artificial intelligence response system, test words art is matched with multiple carousel answers, to be tested under realization dialogue art carousel scene.

Description

A kind of artificial intelligence response system detection method and system
Technical field
The present invention relates to artificial intelligence correlative technology field, particularly a kind of artificial intelligence response system detection method and it is System.
Background technology
In artificial intelligence field, many outstanding products have been emerged at present, these products are all based at natural language The automaticmanual intelligent response system of reason has the automated test frame that some increase income, in order to solve at present for such system The automatic test problem of automaticmanual intelligent response system based on natural language processing, introduce selenium frames+ The automation solutions of python can be realized and put question to input for system in web terminal and answer the automation of output judgement Test, since most of automatic answering system is for uniquely having inputted unique answer, can directly be matched with result Mode judge the result of test execution.
Existing automatic technology has the automatic test that web is realized based on selenium frames, in the defeated of web page Enter frame input problem, and capture returning the result for input frame, problem and result are saved in file, then carry out result judgement, Next problem is performed again.
The data of the problem of due to intelligent response system and answer are very big, by test script and test data point From, test data is split in the form of reading and control preservation are convenient in certain structure preservation in the table with behavior unit, Data line is a session.Different column datas represent different session informations, and following content is included per a line:
1 problem of the prior art of table and answer store method
Problem Scene identity Test account Predict answer
However, for high frequency problem, it is possible to multiple answers occur, that is, words art carousel scene occur.However it is existing from The result that dynamicization testing process and test data preserving type can not meet under words art carousel scene judges.
Invention content
Based on this, it is necessary to which automatic testing process and test data preserving type for the prior art can not meet words The technical issues of result under art carousel scene judges provides a kind of artificial intelligence response system detection method and system.
A kind of artificial intelligence response system detection method, including:
Test data obtaining step, including:Acquisition include at least one problem and with the associated multiple wheels of each problem Broadcast answer;
Test case obtaining step, including:The problem of obtaining test case, selecting to be associated with the test case is as survey Why inscribe and export;
Comparison step, including:Obtaining artificial intelligence response system, art talks about art as test to the test problem, such as Test words art described in fruit and one of them described associated carousel answer matches of test problem, then judge that the testing case leads to It crosses.
A kind of artificial intelligence response system test system, including:
Test data acquisition module, is used for:Acquisition include at least one problem and with the associated multiple wheels of each problem Broadcast answer;
Test case acquisition module, is used for:The problem of obtaining test case, selecting to be associated with the test case is as survey Why inscribe and export;
Comparison module is used for:Obtaining artificial intelligence response system, art talks about art as test to the test problem, such as Test words art described in fruit and one of them described associated carousel answer matches of test problem, then judge that the testing case leads to It crosses.
The present invention by obtain a problem and with the relevant multiple carousel answers of the problem, and test artificial intelligence should When answering system, test words art with multiple carousel answers is matched, is tested under art carousel scene with realizing to talk with.
Description of the drawings
Fig. 1 is a kind of work flow diagram of artificial intelligence response system detection method of the present invention;
Fig. 2 is the work flow diagram of preferred embodiment;
Fig. 3 is a kind of function structure chart of artificial intelligence response system test system of the present invention.
Specific embodiment
The present invention will be further described in detail in the following with reference to the drawings and specific embodiments.
It is as shown in Figure 1 a kind of work flow diagram of artificial intelligence response system detection method of the present invention, including:
Step S101, including:Acquisition include at least one problem and with the associated multiple carousel answers of each problem;
Step S102, including:Obtain test case, the problem of selecting to be associated with the test case as test problem simultaneously Output;
Step S103, including:Obtaining artificial intelligence response system, art talks about art as test to the test problem, such as Test words art described in fruit and one of them described associated carousel answer matches of test problem, then judge that the testing case leads to It crosses.
The each problem got in step S101 is associated with corresponding carousel answer.Since carousel answer is multiple, because This, in step s 103, when art if artificial intelligence response system provides difference, test method of the invention also can be well It adapts to.In art carousel scene is talked about, artificial intelligence response system may provide different answers for same problem, therefore same One test case, the prior art are also likely to will appear test for the first time by and the situation of second of test crash, and this Invention can overcome the prior art institute problem, make well since multiple carousel answers being associated with same problem It obtains test result and stablizes unification in the case where talking about art carousel scene.
Solve the problems, such as a result decision problem for corresponding to multiple answers, this scheme in automatic test by the present invention It can be adapted for all similar scenes, can significantly promote the automatic test of the automatic answering system based on artificial intelligence Use-case coverage rate reduces testing cost.
In one of the embodiments, described problem also with for representing the carousel answer of associated carousel answer quantity Item number is associated preservation.
Carousel answer item number can in step S101 by analyze obtain the problem of and with each problem it is associated more A carousel answer is calculated, and can also together be provided in offer problem and carousel answer by user.
The step S103 in one of the embodiments, specifically includes:Artificial intelligence response system is obtained to the survey Art obtains the associated carousel answer item number of the test problem as test carousel answer item as test words art if why inscribing Number, the test answers with the associated carousel answer of the test problem are compared and matchs, if it is described test talk about art and One of carousel answer matches, then judge the testing case by and terminate the comparison match, the otherwise test Words art is compared with another carousel answer to be matched, if the number of test the words art and carousel answer comparison match is more than The test carousel answer item number, then judge that the testing case fails.
Since problem is associated with preservation with carousel answer item number, it can therefore be appreciated that there is a problem how many carousels to answer Case matches so that test is more accurate so as to which test words art be compared with all carousel answers.
Described problem and associated carousel answer and carousel answer item number are using two dimension in one of the embodiments, Table preserves, and every a line of the bivariate table preserves a problem and associated carousel answer and carousel answer item number.
In one of the embodiments:
The step S101, specifically includes:Obtain test account, acquisition include at least one problem and with each problem Associated multiple carousel answers, described problem and the carousel answer and the test account relating;
The step S102, specifically includes:Artificial intelligence response system is logged in by testing account, obtains test case, The problem of selecting to be associated with the test case is as test problem and exports.
As highly preferred embodiment of the present invention, problem, carousel answer, carousel answer item number preserving type such as the following table 2 institute Show:
The problem of 2 preferred embodiment of table and answer preserving type
Carousel words art data organization form is solved by way of increasing carousel answer item number and multiple carousel answers Problem, when n=1, can be compatible with pervious scheme, n>Answer words art carousel is supported when 1.Wherein, scene end of identification is used to represent End for a test case.
The work flow diagram of preferred embodiment is illustrated in figure 2, including:
Step S201, read test use-case;
Step S202 opens browser;
Step S203 inputs the URL addresses of artificial intelligence response system to be tested;
Step S204 obtains test account from the bivariate table shown in table 2, and artificial intelligence response is logged in by testing account System;
Step S205 obtains problem from the bivariate table shown in table 2, and inputs artificial intelligence response system;
Step S206 captures the result of artificial intelligence response system;
Step S207, the carousel answer item number n corresponding to acquisition problem, by the obtained results of step S206 in n item wheels It broadcasts and matching comparison is carried out in answer, if successful match, perform step S208, else if n carousel answer mismatches Then perform step S209;
Step S208, implementing result are recorded as, by (PASS), performing step S210;
Step S209, implementing result are recorded as failure (FAIL), perform step S210;
Step S210 determines whether the last one problem in table 2, if it is terminates, and otherwise performs step S205.
A kind of function structure chart of artificial intelligence response system test system of the present invention is illustrated in figure 3, including:
Test data acquisition module 301, is used for:Acquisition includes at least one problem and associated multiple with each problem Carousel answer;
Test case acquisition module 302, is used for:Test case is obtained, selects to make the problem of association with the test case For test problem and export;
Comparison module 303, is used for:Obtaining artificial intelligence response system, art is talked about as test to the test problem Art, if the test words art and one of them described associated carousel answer matches of test problem, judge the test case Test passes through.
In one of the embodiments, described problem also with for representing the carousel answer of associated carousel answer quantity Item number is associated preservation.
The comparison module in one of the embodiments, is specifically used for:Artificial intelligence response system is obtained to the survey Art obtains the associated carousel answer item number of the test problem as test carousel answer item as test words art if why inscribing Number, the test answers with the associated carousel answer of the test problem are compared and matchs, if it is described test talk about art and One of carousel answer matches, then judge the testing case by and terminate the comparison match, the otherwise test Words art is compared with another carousel answer to be matched, if the number of test the words art and carousel answer comparison match is more than The test carousel answer item number, then judge that the testing case fails.
Described problem and associated carousel answer and carousel answer item number are using two dimension in one of the embodiments, Table preserves, and every a line of the bivariate table preserves a problem and associated carousel answer and carousel answer item number.
The test data acquisition module in one of the embodiments, is specifically used for:Test account is obtained, obtains packet Include at least one problem and with the associated multiple carousel answers of each problem, described problem and the carousel answer with it is described Test account relating;
The test case acquisition module, is specifically used for:Artificial intelligence response system is logged in by testing account, obtains and surveys The problem of trying out example, selecting to be associated with the test case is as test problem and exports.
Embodiment described above only expresses the several embodiments of the present invention, and description is more specific and detailed, but simultaneously Cannot the limitation to the scope of the claims of the present invention therefore be interpreted as.It should be pointed out that for those of ordinary skill in the art For, without departing from the inventive concept of the premise, various modifications and improvements can be made, these belong to the guarantor of the present invention Protect range.Therefore, the protection domain of patent of the present invention should be determined by the appended claims.

Claims (6)

1. a kind of artificial intelligence response system detection method, which is characterized in that including:
Test data obtaining step, including:Acquisition includes at least one problem and is answered with the associated multiple carousels of each problem Case;
Described problem also with for representing that the carousel answer item number of associated carousel answer quantity is associated preservation;
Described problem and associated carousel answer and carousel answer item number are preserved using bivariate table, the bivariate table it is each Row preserves a problem and associated carousel answer and carousel answer item number;
Test case obtaining step, including:The problem of obtaining test case, selecting to be associated with the test case is asked as test It inscribes and exports;
Comparison step, including:Obtaining artificial intelligence response system, art is as test words art to the test problem, if institute Test words art and one of them described associated carousel answer matches of test problem are stated, then judge that the testing case passes through.
2. artificial intelligence response system detection method according to claim 1, which is characterized in that the comparison step, tool Body includes:Obtaining artificial intelligence response system, art as test words art, obtains the test problem to the test problem Associated carousel answer item number is as test carousel answer item number, by test answers and the associated carousel answer of the test problem Matching is compared, if the test words art and one of carousel answer matches, judge that the testing case passes through And terminate the comparison match, otherwise the test words art is compared with another carousel answer matches, if the test The number for talking about art and carousel answer comparison match is more than the test carousel answer item number, then judges that the testing case loses It loses.
3. artificial intelligence response system detection method according to claim 1, it is characterised in that:
The test data obtaining step, specifically includes:Obtain test account, acquisition include at least one problem and with it is each The associated multiple carousel answers of problem, problem and the carousel answer and the test account relating;
The test case obtaining step, specifically includes:Artificial intelligence response system is logged in by testing account, test is obtained and uses Example, the problem of selecting to be associated with the test case, is as test problem and exports.
4. a kind of artificial intelligence response system test system, which is characterized in that including:
Test data acquisition module, is used for:Acquisition includes at least one problem and is answered with the associated multiple carousels of each problem Case;
Described problem also with for representing that the carousel answer item number of associated carousel answer quantity is associated preservation;
Described problem and associated carousel answer and carousel answer item number are preserved using bivariate table, the bivariate table it is each Row preserves a problem and associated carousel answer and carousel answer item number;
Test case acquisition module, is used for:The problem of obtaining test case, selecting to be associated with the test case is asked as test It inscribes and exports;
Comparison module is used for:Obtaining artificial intelligence response system, art is as test words art to the test problem, if institute Test words art and one of them described associated carousel answer matches of test problem are stated, then judge that the testing case passes through.
5. artificial intelligence response system test system according to claim 4, which is characterized in that the comparison module, tool Body is used for:Obtaining artificial intelligence response system, art as test words art, obtains the test problem to the test problem Associated carousel answer item number is as test carousel answer item number, by test answers and the associated carousel answer of the test problem Matching is compared, if the test words art and one of carousel answer matches, judge that the testing case passes through And terminate the comparison match, otherwise the test words art is compared with another carousel answer matches, if the test The number for talking about art and carousel answer comparison match is more than the test carousel answer item number, then judges that the testing case loses It loses.
6. artificial intelligence response system test system according to claim 4, which is characterized in that
The test data acquisition module, is specifically used for:Obtain test account, acquisition include at least one problem and with it is each The associated multiple carousel answers of problem, problem and the carousel answer and the test account relating;
The test case acquisition module, is specifically used for:Artificial intelligence response system is logged in by testing account, test is obtained and uses Example, the problem of selecting to be associated with the test case, is as test problem and exports.
CN201510346495.8A 2015-06-19 2015-06-19 A kind of artificial intelligence response system detection method and system CN105095069B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510346495.8A CN105095069B (en) 2015-06-19 2015-06-19 A kind of artificial intelligence response system detection method and system

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510346495.8A CN105095069B (en) 2015-06-19 2015-06-19 A kind of artificial intelligence response system detection method and system
HK16105721.7A HK1217790A1 (en) 2015-06-19 2016-05-18 Test method and system of artificial intelligence answering system

Publications (2)

Publication Number Publication Date
CN105095069A CN105095069A (en) 2015-11-25
CN105095069B true CN105095069B (en) 2018-06-08

Family

ID=54575564

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510346495.8A CN105095069B (en) 2015-06-19 2015-06-19 A kind of artificial intelligence response system detection method and system

Country Status (2)

Country Link
CN (1) CN105095069B (en)
HK (1) HK1217790A1 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106202417A (en) * 2016-07-12 2016-12-07 北京光年无限科技有限公司 A kind of man-machine interaction method for intelligent robot and system
CN110908919B (en) * 2019-12-02 2021-03-26 上海市软件评测中心有限公司 Response test system based on artificial intelligence and application thereof

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1744056A (en) * 2004-09-04 2006-03-08 华为技术有限公司 Automatic test method of application
CN101226522A (en) * 2008-02-04 2008-07-23 黄伟才 Request-answering system and method supporting interaction between users
CN103186466A (en) * 2011-12-31 2013-07-03 中国银联股份有限公司 Information interaction device and method based on automatic generation of associated test cases
CN103853842A (en) * 2014-03-20 2014-06-11 百度在线网络技术(北京)有限公司 Automatic question and answer method and system

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7480899B2 (en) * 2004-03-22 2009-01-20 International Business Machines Corporation Method and apparatus for autonomic test case feedback using hardware assistance for code coverage

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1744056A (en) * 2004-09-04 2006-03-08 华为技术有限公司 Automatic test method of application
CN101226522A (en) * 2008-02-04 2008-07-23 黄伟才 Request-answering system and method supporting interaction between users
CN103186466A (en) * 2011-12-31 2013-07-03 中国银联股份有限公司 Information interaction device and method based on automatic generation of associated test cases
CN103853842A (en) * 2014-03-20 2014-06-11 百度在线网络技术(北京)有限公司 Automatic question and answer method and system

Also Published As

Publication number Publication date
CN105095069A (en) 2015-11-25
HK1217790A1 (en) 2017-01-20

Similar Documents

Publication Publication Date Title
CN106296724B (en) Method and system for determining track information of target person and processing server
US9092549B2 (en) Recommendation of search keywords based on indication of user intention
Pearce et al. Climate change on Twitter: Topics, communities and conversations about the 2013 IPCC Working Group 1 report
Katircioglu Investigating the role of oil prices in the conventional EKC model: evidence from Turkey
CN105550184B (en) A kind of information acquisition method and device
JP2019050011A (en) Method and device for compensating erroneous determination of attribution destination of impression data and/or non-coverage by database holder
US9424319B2 (en) Social media based content selection system
KR20160055930A (en) Systems and methods for actively composing content for use in continuous social communication
WO2017045462A1 (en) Method and system for presenting information related to goods in video stream
CN105930363B (en) HTML5 webpage-based user behavior analysis method and device
CN104391979A (en) Malicious web crawler recognition method and device
WO2014143969A2 (en) Methods and apparatus to credit usage of mobile devices
CN104394118B (en) A kind of method for identifying ID and system
US10313523B2 (en) Remote distance assistance system and method
CN103475910B (en) A kind of programs of set-top box for intelligent television end recommends method and system
EP2943925A2 (en) Method and apparatus of identifying a website user
CN107256650B (en) Exercise pushing method and system and terminal equipment
CN103810198B (en) A kind of searching method of merchandise news and device
EP2506522B1 (en) Method and device for pushing data
CN106327605B (en) A kind of method for inspecting
CN103942337B (en) It is a kind of based on image recognition and the video searching system that matches
Sørensen et al. Before and after gdpr: The changes in third party presence at public and private european websites
US10789311B2 (en) Method and device for selecting data content to be pushed to terminal, and non-transitory computer storage medium
CN104618806A (en) Method, device and system for acquiring comment information of video
US20140343704A1 (en) Systems and Methods for Program Identification

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 1217790

Country of ref document: HK

GR01 Patent grant
GR01 Patent grant
REG Reference to a national code

Ref country code: HK

Ref legal event code: GR

Ref document number: 1217790

Country of ref document: HK