CN106777248A - A kind of search engine test evaluation method and apparatus - Google Patents
A kind of search engine test evaluation method and apparatus Download PDFInfo
- Publication number
- CN106777248A CN106777248A CN201611228051.5A CN201611228051A CN106777248A CN 106777248 A CN106777248 A CN 106777248A CN 201611228051 A CN201611228051 A CN 201611228051A CN 106777248 A CN106777248 A CN 106777248A
- Authority
- CN
- China
- Prior art keywords
- search
- search results
- word
- search word
- results
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/951—Indexing; Web crawling techniques
Abstract
The embodiment of the invention discloses a kind of search engine test evaluation method and apparatus, the method includes：Training pattern is determined according to preset model and at least one training data, the training data includes：At least one search word and at least one set of Search Results corresponding with least one search word, each search word correspondence last set result；Obtain at least one first Search Results of first search word in the first search engine, described at least one first Search Results and first search word are compared in the correlation results of the training pattern respectively, obtain at least one relevance degree of at least one first Search Results；Test evaluation is carried out at least one relevance degree by default test evaluation algorithm respectively, the test evaluation result of first search engine is obtained.The embodiment of the present invention can realize accurately and efficiently automatic test.
Description
Technical field
The present invention relates to the search engine evalution technology in web search field, more particularly to a kind of search engine test evaluation
Method and apparatus.
Background technology
In a search engine, it is necessary to scan for quality evaluation to search engine.Existing search engine evaluation method bag
Include Cranfield appraisement systems., it is necessary to can just evaluate search engine after manual testing in Cranfield appraisement systems
Quality, and have reliability, it is necessary to obtain a result just to compare after a large amount of manual testings if to accomplish the accurate of evaluation result, than
Relatively time-consuming effort；For Cranfield appraisement systems, if to accomplish automatic test, it is necessary to according to search word, manually determine
The most relative set of search word, so as to carry out automation evaluation and test according to comparing with most relative set.Therefore, in the method, most
There is subjectivity in relative set, and needs are manually adjusted, safeguarded, updating, while compare labor intensive, and evaluation result standard
Also can there is deviation in true property.
The content of the invention
In order to solve the above technical problems, the embodiment of the present invention provides a kind of search engine test evaluation method and apparatus, can
To realize accurately and efficiently automatic test.
The technical proposal of the invention is realized in this way：
The embodiment of the present invention provides a kind of search engine test evaluation device, and described device includes：Determining unit, acquisition are single
Unit, processing unit, wherein,
The determining unit, for determining training pattern, the training according to preset model and at least one training data
Data include：At least one search word and at least one set of Search Results corresponding with least one search word, each is searched
Rope word correspondence last set result；
The acquiring unit, for obtaining at least one first Search Results of first search word in the first search engine；
The processing unit, for respectively by described at least one first Search Results with first search word described
The correlation results of training pattern are compared, and obtain at least one relevance degree of at least one first Search Results；
It is additionally operable to respectively carry out at least one relevance degree test evaluation by default test evaluation algorithm, obtains described first
The test evaluation result of search engine.
Alternatively, the determining unit, for determining at least one training number according to the search data of at least one user
According to；
The processing unit, at least one training data to be carried out into integration treatment, obtains the first training data；
It is additionally operable to be trained first training data by word2vec models, obtains training pattern.
Alternatively, the processing unit, is processed for the search data at least one user, obtains second
Each Search Results in search word the second Search Results corresponding with second search word and second Search Results
Number of clicks, second search word includes：At least one search word, second Search Results include：At least one set search
As a result, each search word correspondence last set result, at least one Search Results are included per last set result；It is additionally operable to root
All identicals in second search word are searched for according to the number of clicks of each Search Results in second Search Results
The corresponding Search Results of word are arranged, and are additionally operable to be given birth to by second Search Results after arrangement and second search word
Into at least one training data.
Alternatively, the 3rd search word is identical search word in second search word, and the 3rd Search Results are and the 3rd
The corresponding Search Results of search word；
The processing unit, enters from big to small for the number of clicks according to each Search Results in the 3rd Search Results
Row arrangement the 3rd Search Results, are additionally operable to generate described second by the 3rd Search Results after arrangement and the 3rd search word
Training data.
Alternatively, the acquiring unit, the search data for obtaining at least one user, wherein, user's searches
Rope data include：User searches for and click logs.
The embodiment of the present invention provides a kind of search engine test evaluation method, and methods described includes：
Training pattern is determined according to preset model and at least one training data, the training data includes：At least one
Search word and at least one set of Search Results corresponding with least one search word, each search word correspondence last set knot
Really；
At least one first Search Results of first search word in the first search engine are obtained, respectively by described at least one
First Search Results are compared with first search word in the correlation results of the training pattern, at least one described in acquisition
At least one relevance degree of individual first Search Results；
Test evaluation is carried out at least one relevance degree by default test evaluation algorithm respectively, described the is obtained
The test evaluation result of one search engine.
Alternatively, it is described that training pattern is determined according to preset model and at least one training data, including：
Search data according at least one user determine at least one training data；
At least one training data is carried out into integration treatment, the first training data is obtained；
First training data is trained by word2vec models, obtains training pattern.
Alternatively, the search data according at least one user determine at least one training data, including：
Search data at least one user are processed, and obtain the second search word and second search word
The number of clicks of each Search Results, second search word in the second Search Results of correspondence and second Search Results
Including：At least one search word, second Search Results include：At least one set of Search Results, each search word correspondence one
Group searching result, at least one Search Results are included per last set result；
Number of clicks according to each Search Results in second Search Results will own in second search word
The corresponding Search Results of identical search word are arranged, by second Search Results after arrangement and second search
Word generates at least one training data.
Alternatively, the 3rd search word is identical search word in second search word, and the 3rd Search Results are and the 3rd
The corresponding Search Results of search word；
Number of clicks according to each Search Results in the 3rd Search Results is arranged, and obtains the 3rd search word
The second training data, including：
Number of clicks according to each Search Results in the 3rd Search Results carries out arrangement the described 3rd and searches from big to small
Hitch really, second training data is generated by the 3rd Search Results after arrangement and the 3rd search word.
Alternatively, before the search data according at least one user determine at least one training data, also wrap
Include：
The search data of at least one user are obtained, wherein, the search data of user include：User searches for and clicks on
Daily record.
A kind of search engine test evaluation method and apparatus are the embodiment of the invention provides, according to preset model and at least one
Individual training data determines training pattern, and the training data includes：At least one search word and with least one search word
Corresponding at least one set of Search Results, each search word correspondence last set result；The first search word is obtained in the first search
At least one first Search Results of engine, respectively by described at least one first Search Results with first search word in institute
The correlation results for stating training pattern are compared, and obtain at least one degree of correlation of at least one first Search Results
Value；Test evaluation is carried out at least one relevance degree by default test evaluation algorithm respectively, described first is obtained and is searched
The test evaluation result that index is held up.Search engine test evaluation method and apparatus provided in an embodiment of the present invention, use default mould
Type obtains the relevance degree between search word and Search Results, using relevance degree as test search engine search quality point
Value, so that the score value of the Search Results on the search engine is calculated automatically, while automatic test is realized, it is possible to increase
Test result accuracy, and save the testing time.
Brief description of the drawings
Fig. 1 is a hardware architecture diagram for optional mobile terminal for realizing each embodiment of the invention；
Fig. 2 is the wireless communication system schematic diagram of mobile terminal as shown in Figure 1；
Fig. 3 is search engine test evaluation method flow schematic diagram one provided in an embodiment of the present invention；
Fig. 4 is terminal searching interface display exemplary plot provided in an embodiment of the present invention；
Fig. 5 is search engine test evaluation method flow schematic diagram two provided in an embodiment of the present invention；
Fig. 6 is training data exemplary plot provided in an embodiment of the present invention；
Fig. 7 is search engine test evaluation method flow exemplary plot provided in an embodiment of the present invention；
Fig. 8 is search engine test evaluation apparatus structure schematic diagram provided in an embodiment of the present invention.
Specific embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete
Site preparation is described.
The mobile terminal of each embodiment of the invention is realized referring now to Description of Drawings.In follow-up description, use
For represent element such as " module ", " part " or " unit " suffix only for being conducive to explanation of the invention, itself
Not specific meaning.Therefore, " module " can be used mixedly with " part ".
Mobile terminal can be implemented in a variety of manners.For example, the terminal described in the present invention can include such as moving
Phone, smart phone, notebook computer, digit broadcasting receiver, PDA (personal digital assistant), PAD (panel computer), PMP
The mobile terminal of (portable media player), guider etc. and such as numeral TV, desktop computer etc. are consolidated
Determine terminal.Hereinafter it is assumed that terminal is mobile terminal.However, it will be understood by those skilled in the art that, except being used in particular for movement
Outside the element of purpose, construction according to the embodiment of the present invention can also apply to the terminal of fixed type.
Fig. 1 is that the hardware configuration of an optional mobile terminal for realizing each embodiment of the invention is illustrated.
Mobile terminal 1 00 can include wireless communication unit 110, A/V (audio/video) input block 120, user input
Unit 130, sensing unit 140, output unit 150, memory 160, interface unit 170, controller 180 and power subsystem 190
Etc..Fig. 1 shows the mobile terminal with various assemblies, it should be understood that being not required for implementing all groups for showing
Part.More or less component can alternatively be implemented.The element of mobile terminal will be discussed in more detail below.
Wireless communication unit 110 generally includes one or more assemblies, and it allows mobile terminal 1 00 and wireless communication system
Or the radio communication between network.For example, wireless communication unit can include broadcasting reception module 111, mobile communication module
112nd, at least one of wireless Internet module 113, short range communication module 114 and location information module 115.
Broadcasting reception module 111 receives broadcast singal and/or broadcast via broadcast channel from external broadcast management server
Relevant information.Broadcast channel can include satellite channel and/or terrestrial channel.Broadcast management server can be generated and sent
The broadcast singal and/or broadcast related information generated before the server or reception of broadcast singal and/or broadcast related information
And send it to the server of terminal.Broadcast singal can include TV broadcast singals, radio signals, data broadcasting
Signal etc..And, broadcast singal may further include the broadcast singal combined with TV or radio signals.Broadcast phase
Pass information can also be provided via mobile communications network, and in this case, broadcast related information can be by mobile communication mould
Block 112 is received.Broadcast singal can exist in a variety of manners, for example, it can be with the electronics of DMB (DMB)
The form of program guide (EPG), the electronic service guidebooks (ESG) of digital video broadcast-handheld (DVB-H) etc. and exist.Broadcast
Receiver module 111 can receive signal and broadcast by using various types of broadcast systems.Especially, broadcasting reception module 111
Can be wide by using such as multimedia broadcasting-ground (DMB-T), DMB-satellite (DMB-S), digital video
Broadcast-hand-held (DVB-H), Radio Data System, the received terrestrial digital broadcasting integrated service of forward link media (MediaFLO@)
Etc. (ISDB-T) digit broadcasting system receives digital broadcasting.Broadcasting reception module 111 may be constructed such that and be adapted to provide for extensively
Broadcast the various broadcast systems and above-mentioned digit broadcasting system of signal.Via broadcasting reception module 111 receive broadcast singal and/
Or broadcast related information can be stored in memory 160 (or other types of storage medium).
Mobile communication module 112 sends radio signals to base station (for example, access point, node B etc.), exterior terminal
And at least one of server and/or receive from it radio signal.Such radio signal can be logical including voice
Words signal, video calling signal or the various types of data for sending and/or receiving according to text and/or Multimedia Message.
Wireless Internet module 113 supports the Wi-Fi (Wireless Internet Access) of mobile terminal.The module can be internally or externally
It is couple to terminal.Wi-Fi (Wireless Internet Access) technology involved by the module can include WLAN (WLAN) (Wi-Fi), Wibro
(WiMAX), Wimax (worldwide interoperability for microwave accesses), HSDPA (high-speed downlink packet access) etc..
Short range communication module 114 is the module for supporting junction service.Some examples of short-range communication technology include indigo plant
Tooth TM, radio frequency identification (RFID), Infrared Data Association (IrDA), ultra wide band (UWB), purple honeybee TM etc..
Location information module 115 is the module for checking or obtaining the positional information of mobile terminal.Location information module
Typical case be GPS (global positioning system).According to current technology, calculated as the location information module 115 of GPS and come from
The range information and correct time information of three or more satellites and the Information application triangulation for calculating, so that
Calculate according to longitude, latitude and highly accurately three-dimensional current location information.Currently, for calculating the side of position and temporal information
Method corrects the error of the position and temporal information for calculating using three satellites and by using an other satellite.This
Outward, can be by Continuous plus current location information in real time come calculating speed information as the location information module 115 of GPS.
A/V input blocks 120 are used to receive audio or video signal.A/V input blocks 120 can include 121 Hes of photograph
Microphone 122, the static images that photograph 121 pairs is obtained in Video Capture pattern or image capture mode by image capture apparatus
Or the view data of video is processed.Picture frame after treatment may be displayed on display unit 151.Processed through photograph 121
Picture frame afterwards can be stored in memory 160 (or other storage mediums) or sent out via wireless communication unit 110
Send, two or more photograph 121 can be provided according to the construction of mobile terminal.Microphone 122 can be in telephone calling model, note
Sound (voice data) is received via microphone in record pattern, speech recognition mode etc. operational mode, and can be by so
Acoustic processing be voice data.Audio (voice) data after treatment can be converted in the case of telephone calling model can
The form for being sent to mobile communication base station via mobile communication module 112 is exported.Microphone 122 can implement various types of making an uproar
Sound eliminates (or suppression) algorithm to eliminate the noise or dry that (or suppression) produces during reception and transmission audio signal
Disturb.
User input unit 130 can generate key input data to control each of mobile terminal according to the order of user input
Plant operation.User input unit 130 allows the various types of information of user input, and can include keyboard, metal dome, touch
Plate (for example, detection due to being touched caused by resistance, pressure, electric capacity etc. change sensitive component), roller, rocking bar etc.
Deng.Especially, when touch pad is superimposed upon on display unit 151 in the form of layer, touch-screen can be formed.
Sensing unit 140 detects the current state of mobile terminal 1 00, (for example, mobile terminal 1 00 opens or closes shape
State), the presence or absence of the contact (that is, touch input) of the position of mobile terminal 1 00, user for mobile terminal 1 00, mobile terminal
The acceleration or deceleration movement of 100 orientation, mobile terminal 1 00 and direction etc., and generate for controlling mobile terminal 1 00
The order of operation or signal.For example, when mobile terminal 1 00 is embodied as sliding-type mobile phone, sensing unit 140 can be sensed
The sliding-type phone is opened or closed.In addition, sensing unit 140 can detect power subsystem 190 whether provide electric power or
Whether person's interface unit 170 couples with external device (ED).
Interface unit 170 is connected the interface that can pass through with mobile terminal 1 00 as at least one external device (ED).For example,
External device (ED) can include wired or wireless head-band earphone port, external power source (or battery charger) port, wired or nothing
Line FPDP, memory card port, the port for connecting the device with identification module, audio input/output (I/O) end
Mouth, video i/o port, ear port etc..Identification module can be that storage uses each of mobile terminal 1 00 for verifying user
Kind of information and subscriber identification module (UIM), client identification module (SIM), Universal Subscriber identification module (USIM) can be included
Etc..In addition, the device (hereinafter referred to as " identifying device ") with identification module can take the form of smart card, therefore, know
Other device can be connected via port or other attachment means with mobile terminal 1 00.Interface unit 170 can be used for reception and come from
The input (for example, data message, electric power etc.) of the external device (ED) and input that will be received is transferred in mobile terminal 1 00
One or more elements can be used for transmitting data between mobile terminal and external device (ED).
In addition, when mobile terminal 1 00 is connected with external base, interface unit 170 can serve as allowing by it by electricity
Power provides to the path of mobile terminal 1 00 from base or can serve as allowing the various command signals being input into from base to pass through it
It is transferred to the path of mobile terminal.Be can serve as recognizing that mobile terminal is from the various command signals or electric power of base input
The no signal being accurately fitted within base.Output unit 150 is configured to provide defeated with vision, audio and/or tactile manner
Go out signal (for example, audio signal, vision signal, alarm signal, vibration signal etc.).Output unit 150 can include display
Unit 151, dio Output Modules 152 etc..
Display unit 151 may be displayed on the information processed in mobile terminal 1 00.For example, when mobile terminal 1 00 is in electricity
During words call mode, display unit 151 can show and converse or other communicate (for example, text messaging, multimedia file
Download etc.) related user interface (UI) or graphic user interface (GUI).When mobile terminal 1 00 is in video calling pattern
Or during image capture mode, display unit 151 can show the image of capture and/or the image of reception, show video or figure
UI or GUI of picture and correlation function etc..
Meanwhile, when display unit 151 and touch pad in the form of layer it is superposed on one another to form touch-screen when, display unit
151 can serve as input unit and output device.Display unit 151 can include liquid crystal display (LCD), thin film transistor (TFT)
In LCD (TFT-LCD), Organic Light Emitting Diode (OLED) display, flexible display, three-dimensional (3D) display etc. at least
It is a kind of.Some in these displays may be constructed such that transparence to allow user to be watched from outside, and this is properly termed as transparent
Display, typical transparent display can be, for example, TOLED (transparent organic light emitting diode) display etc..According to specific
Desired implementation method, mobile terminal 1 00 can include two or more display units (or other display devices), for example, moving
Dynamic terminal can include outernal display unit (not shown) and inner display unit (not shown).Touch-screen can be used to detect touch
Input pressure and touch input position and touch input area.
Dio Output Modules 152 can mobile terminal be in call signal reception pattern, call mode, logging mode,
It is that wireless communication unit 110 is received or in memory 160 when under the isotypes such as speech recognition mode, broadcast reception mode
The voice data transducing audio signal of middle storage and it is output as sound.And, dio Output Modules 152 can be provided and movement
The audio output (for example, call signal receives sound, message sink sound etc.) of the specific function correlation that terminal 100 is performed.
Dio Output Modules 152 can include loudspeaker, buzzer etc..
Memory 160 can store software program for the treatment and control operation performed by controller 180 etc., Huo Zheke
Temporarily to store the data that exported or will export (for example, telephone directory, message, still image, video etc.).And
And, memory 160 can store the vibration of various modes on being exported when touching and being applied to touch-screen and audio signal
Data.
Memory 160 can include the storage medium of at least one type, and the storage medium includes flash memory, hard disk, many
Media card, card-type memory (for example, SD or DX memories etc.), random access storage device (RAM), static random-access storage
Device (SRAM), read-only storage (ROM), Electrically Erasable Read Only Memory (EEPROM), programmable read only memory
(PROM), magnetic storage, disk, CD etc..And, mobile terminal 1 00 can perform memory with by network connection
The network storage device cooperation of 160 store function.
The overall operation of the generally control mobile terminal of controller 180.For example, controller 180 is performed and voice call, data
Communication, video calling etc. related control and treatment.In addition, controller 180 can be included for reproducing (or playback) many matchmakers
The multi-media module 181 of volume data, multi-media module 181 can be constructed in controller 180, or can be structured as and control
Device 180 is separated.Controller 180 can be with execution pattern identifying processing, the handwriting input that will be performed on the touchscreen or picture
Draw input and be identified as character or image.
Power subsystem 190 receives external power or internal power under the control of controller 180 and provides operation each unit
Appropriate electric power needed for part and component.
Various implementation methods described herein can be with use such as computer software, hardware or its any combination of calculating
Machine computer-readable recording medium is implemented.Implement for hardware, implementation method described herein can be by using application-specific IC
(ASIC), digital signal processor (DSP), digital signal processing device (DSPD), programmable logic device (PLD), scene can
Programming gate array (FPGA), processor, controller, microcontroller, microprocessor, it is designed to perform function described herein
At least one in electronic unit is implemented, and in some cases, such implementation method can be implemented in controller 180.
For software implementation, the implementation method of such as process or function can with allow to perform the single of at least one function or operation
Software module is implemented.Software code can be come by the software application (or program) write with any appropriate programming language
Implement, software code can be stored in memory 160 and performed by controller 180.
So far, mobile terminal is described according to its function.Below, for the sake of brevity, will description such as folded form,
Slide type mobile terminal in various types of mobile terminals of board-type, oscillating-type, slide type mobile terminal etc. is used as showing
Example.Therefore, the present invention can be applied to any kind of mobile terminal, and be not limited to slide type mobile terminal.
Mobile terminal 1 00 as shown in Figure 1 may be constructed such that using via frame or packet transmission data it is all if any
Line and wireless communication system and satellite-based communication system are operated.
The communication system that mobile terminal wherein of the invention can be operated is described referring now to Fig. 2.
Such communication system can use different air interface and/or physical layer.For example, used by communication system
Air interface includes such as frequency division multiple access (FDMA), time division multiple acess (TDMA), CDMA (CDMA) and universal mobile communications system
System (UMTS) (especially, Long Term Evolution (LTE)), global system for mobile communications (GSM) etc..As non-limiting example, under
The description in face is related to cdma communication system, but such teaching is equally applicable to other types of system.
With reference to Fig. 2, cdma wireless communication system can include multiple mobile terminal 1s 00, multiple base station (BS) 270, base station
Controller (BSC) 275 and mobile switching centre (MSC) 280.MSC280 is configured to and Public Switched Telephony Network (PSTN)
290 form interface.MSC280 is also structured to form interface with the BSC275 that can be couple to base station 270 via back haul link.
Back haul link can in some known interfaces any one construct, the interface includes such as E1/T1, ATM, IP,
PPP, frame relay, HDSL, ADSL or xDSL.It will be appreciated that system can include multiple BSC275 as shown in Figure 2.
Each BS270 can service one or more subregions (or region), by multidirectional antenna or the day of sensing specific direction
Each subregion of line covering is radially away from BS270.Or, each subregion can be by two or more for diversity reception
Antenna is covered.Each BS270 may be constructed such that the multiple frequency distribution of support, and the distribution of each frequency has specific frequency spectrum
(for example, 1.25MHz, 5MHz etc.).
What subregion and frequency were distributed intersects can be referred to as CDMA Channel.BS270 can also be referred to as base station transceiver
System (BTS) or other equivalent terms.In this case, term " base station " can be used for broadly representing single
BSC275 and at least one BS270.Base station can also be referred to as " cellular station ".Or, each subregion of specific BS270 can be claimed
It is multiple cellular stations.
As shown in Figure 2, broadcast singal is sent to broadcsting transmitter (BT) 295 mobile terminal operated in system
100.Broadcasting reception module 111 as shown in Figure 1 is arranged at mobile terminal 1 00 to receive the broadcast sent by BT295
Signal.In fig. 2 it is shown that several global positioning system (GPS) satellites 300.Satellite 300 helps position multiple mobile terminals
At least one of 100.
In fig. 2, multiple satellites 300 are depicted, it is understood that be, it is possible to use any number of satellite obtains useful
Location information.It is generally configured to coordinate to obtain with satellite 300 as the location information module 115 of GPS as shown in Figure 1
The location information that must be wanted.Substitute GPS tracking techniques or outside GPS tracking techniques, it is possible to use can track mobile whole
Other technologies of the position at end.In addition, at least one gps satellite 300 can optionally or additionally process satellite dmb biography
It is defeated.
Used as a typical operation of wireless communication system, BS270 receives the reverse link from various mobile terminal 1s 00
Signal.Mobile terminal 1 00 generally participates in call, information receiving and transmitting and other types of communication.Each of the reception of certain base station 270 is anti-
Processed in specific BS270 to link signal.The data of acquisition are forwarded to the BSC275 of correlation.BSC provides call
Resource allocation and the mobile management function of the coordination including the soft switching process between BS270.The number that BSC275 will also be received
According to MSC280 is routed to, it provides the extra route service for forming interface with PSTN290.Similarly, PSTN290 with
MSC280 forms interface, and MSC and BSC275 form interface, and BSC275 correspondingly controls BS270 with by forward link signals
It is sent to mobile terminal 1 00.
Based on above-mentioned mobile terminal hardware configuration and communication system, the inventive method each embodiment is proposed.
The embodiment of the present invention provides a kind of search engine test evaluation method, as shown in figure 3, the method can include：
Step 301, training pattern is determined according to preset model and at least one training data.
Wherein, the training data includes：At least one search word and corresponding at least with least one search word
Last set result, each search word correspondence last set result.
Specifically, in the embodiment of the present invention, determine that training pattern can be with according to preset model and at least one training data
Realized by search engine test evaluation device, i.e., search engine test evaluation device is according to preset model and at least one
Training data determines training pattern, and in actual applications, the search engine test evaluation device is specifically as follows server.
It is described that training pattern is determined according to preset model and at least one training data in a kind of possible implementation,
Including：
Search data according at least one user determine at least one training data；
At least one training data is carried out into integration treatment, the first training data is obtained；
First training data is trained by word2vec models, obtains training pattern.
Wherein, Word2vec is a efficient tool that word is characterized as real number value vector, and it utilizes the think of of deep learning
Think, the vector operation in K gts can be reduced to the treatment to content of text by training, and in vector space
Similarity can be used to represent similarity on text semantic.The term vector of Word2vec outputs can be used to do a lot
NLP related work, such as cluster, look for synonym, part of speech analysis etc..If a thinking is changed, word as feature, then
Feature Mapping to K gts just can be sought more profound character representation by Word2vec for text data.
Word2vec uses the term vector representation of Distributed representation.
Distributed representation basic thoughts are：By train by each word be mapped to K tie up real number vector (K is general
It is the hyper parameter in model), judged by the distance between word (such as cosine similarities, Euclidean distance etc.) between them
Semantic similarity, it uses one three layers of neutral net, input layer-hidden layer-output layer.The technology for having individual core is, root
Encoded with Huffman according to word frequency so that the content of the similar word hidden layer activation of all word frequency is basically identical, and the frequency of occurrences is higher
Word, they activate hiding number of layers it is fewer, so effectively reduce the complexity of calculating.Word2vec has efficient
Property, the unit version of an optimization can train more than one hundred billion word in one day.This three-layer neural network is in itself that language model is carried out
Modeling, but also obtain a kind of expression of word in vector space simultaneously.With latent semantic analysis (Latent Semantic
Index, LSI), potential Di Li Crays distribution (Latent Dirichlet Allocation, LDA) classical processes compare,
Word2vec make use of the context of word, and semantic information more is enriched.
In a kind of possible implementation, the search data according at least one user determine at least one training number
According to, including：
Search data at least one user are processed, and obtain the second search word and second search word
The number of clicks of each Search Results, second search word in the second Search Results of correspondence and second Search Results
Including：At least one search word, second Search Results include：At least one set of Search Results, each search word correspondence one
Group searching result, at least one Search Results are included per last set result；
Number of clicks according to each Search Results in second Search Results will own in second search word
The corresponding Search Results of identical search word are arranged, by second Search Results after arrangement and second search
Word generates at least one training data.
If specifically, one group of identical search word in setting the 3rd search word as second search word, the 3rd search knot
Fruit is Search Results corresponding with the 3rd search word；Then the number of clicks according to each Search Results in the 3rd Search Results is entered
Row arrangement, obtains the second training data of the 3rd search word, Ke Yiwei：According to each search knot in the 3rd Search Results
The number of clicks of fruit carries out arrangement the 3rd Search Results from big to small, is searched by the 3rd Search Results and the 3rd after arrangement
Rope word generates second training data.
Exemplary, as shown in figure 4, the 3rd search word is " weather ", application shop search " weather " on mobile phone, with
" weather " corresponding 3rd Search Results include：Ink marks weather, bright cloud weather, weather are logical, 360 weather etc., wherein, ink marks weather
Number of clicks be 1035, the number of clicks of bright cloud weather is 654, and the logical number of clicks of weather is the click of 892,360 weather
Number of times is 358, and the 3rd Search Results are arranged according to number of clicks, and the 3rd Search Results after arrangement are：Ink marks weather,
Weather is logical, bright cloud weather, 360 weather, as shown in fig. 6, then the second training data is：Weather：Ink marks weather, weather are logical, bright cloud
Weather, 360 weather.
Alternatively, before the search data according at least one user determine at least one training data, also wrap
Include：
The search data of at least one user are obtained, wherein, the search data of user include：User searches for and clicks on
Daily record.
Step 302, the first search word of acquisition, respectively will be described at least one first Search Results of the first search engine
At least one first Search Results are compared with first search word in the correlation results of the training pattern, obtain institute
State at least one relevance degree of at least one first Search Results.
Wherein, search engine (Search Engine) refer to according to certain strategy, with specific computer program from
Information is collected on internet, after information is organized and processed, retrieval service is provided the user, user search is related
The system that information shows user.
Exemplary, the first search engine can be：Baidu search, search dog search etc., can also be answering for terminal installation
Search in, using the search in shop, search of UC browsers etc..
Step 303, test evaluation is carried out at least one relevance degree by default test evaluation algorithm respectively, obtained
Obtain the test evaluation result of first search engine.
Wherein, it can be multiple different test evaluation algorithms to preset test evaluation algorithm, and the test evaluation algorithm can be with
It is test evaluation algorithm of the prior art.
Search engine test evaluation method provided in an embodiment of the present invention, obtains search word and is tied with search using preset model
Relevance degree between fruit, using relevance degree as the score value of test search engine search quality, so as to calculate automatically at this
The score value of Search Results on search engine, while automatic test is realized, it is possible to increase test result accuracy, and save
Testing time.
The embodiment of the present invention provides a kind of search engine test evaluation method, as shown in figure 5, the method can include：
Step 401, search engine test evaluation device obtain the search data of at least one user.
Wherein, the search data of user include：User searches for and click logs.
Here, user's search and click logs can be the search of the user on the internet of acquisition and click logs, also may be used
Think each user's search for applying backstage and the click logs of acquisition, can also be searched for and click logs for other users,
The embodiment of the present invention is not limited this.
Step 402, search engine test evaluation device determine at least one instruction according to the search data of at least one user
Practice data.
Specifically, search engine test evaluation device is processed the search data of at least one user, obtain
Each search knot in second search word the second Search Results corresponding with second search word and second Search Results
The number of clicks of fruit, second search word includes：At least one search word, second Search Results include：It is at least one set of
Search Results, each search word correspondence last set result, at least one Search Results are included per last set result；
Number of clicks according to each Search Results in second Search Results will own in second search word
The corresponding Search Results of identical search word are arranged, by second Search Results after arrangement and second search
Word generates at least one training data.
If specifically, any one group of identical search word in setting the 3rd search word as second search word, the 3rd searches
Hitch fruit is Search Results corresponding with the 3rd search word；Click time then according to each Search Results in the 3rd Search Results
Number is arranged, and obtains the second training data of the 3rd search word, Ke Yiwei：According in the 3rd Search Results each search
The number of clicks of hitch fruit carries out arrangement the 3rd Search Results from big to small, by the 3rd Search Results after arrangement and the
Three search words generate second training data.Similarly, by the corresponding search knot of all identical search words in the second search word
Fruit is classified as a training data.
Exemplary, the 3rd search word is " payment ", application shop search " payment " on mobile phone, corresponding with " weather "
The 3rd Search Results include：Alipay, PayPal pay, the wing pays, easily precious payment, wherein, the number of clicks of Alipay is
3582, the PayPal numbers of clicks for paying are 1654, and the number of clicks that the wing pays is 2892, and the number of clicks of easily precious payment is
1035, the 3rd Search Results are arranged according to number of clicks, the 3rd Search Results after arrangement are：Alipay, the wing pay,
PayPal pays, easily precious payment, as shown in fig. 6, then the second training data is：Pay：Alipay, the wing pay, PayPal pays,
It is easily precious to pay.
At least one training data is carried out integration treatment by step 403, search engine test evaluation device, is obtained first and is instructed
Practice data.
Specifically, arranging and synthesizing a training data, i.e., first by different search words and its corresponding training data
Training data, as shown in fig. 6, the figure includes the corresponding training data of search word " weather ", the corresponding instruction of search word " payment "
Practice data etc..
Step 404, search engine test evaluation device are trained by word2vec models to the first training data, are obtained
Obtain training pattern.
Wherein, word2vec is an instrument that word is converted into vector form.Can be the treatment to content of text
The vector operation in vector space is reduced to, calculates the similarity in vector space to represent the similarity on text semantic.
Step 405, search engine test evaluation device obtain the first search word at least one the of the first search engine
One Search Results, respectively by described at least one first Search Results to first search word in the related of the training pattern
Property result be compared, obtain at least one relevance degree of at least one first Search Results.
Exemplary, the first search engine can be：Baidu search, search dog search etc., can also be answering for terminal installation
Search in, using the search in shop, search of UC browsers etc..
Step 406, search engine test evaluation device are by default test evaluation algorithm respectively at least one phase
Closing angle value carries out test evaluation, obtains the test evaluation result of first search engine.
Here, it can be multiple different test evaluation algorithms to preset test evaluation algorithm, and the test evaluation algorithm can be with
It is test evaluation algorithm of the prior art.
Specifically, search engine test evaluation device by different test evaluation algorithms from different dimensions to the degree of correlation
Value carries out test evaluation, and the search quality of search engine can be known according to different dimension test evaluation results.
Search engine test evaluation method provided in an embodiment of the present invention, according to a large amount of real user's search and click day
Will, the Search Results that search word and user are clicked on, using word2vec training patterns, are counted as training set according to word2vec
Correlation is calculated, so as to draw relative set, for most relative set needed for automatic test, the relevance degree in word2vec is made
It is the corresponding score value of Search Results, so as to realize accurately and efficiently automatic test.
Exemplary, as shown in fig. 7, obtaining user's search and click logs first, data prediction is carried out, obtain multiple
, there is identical search word, root in the plurality of search word in search word and multiple Search Results corresponding with the plurality of search word
According to same search word, by the click result of different user according to number of clicks, it is successively placed on behind correspondence search word, for giving birth to
Into training data, the above-mentioned all training datas obtained according to different search words are integrated into final training data；Use
Word2vec obtains final training data and is trained to above-mentioned, trains word2vec models；According to search word, will search for
The result that engine is found, the correlation results found with the search word in word2vec models are compared, and obtain each search
The relevance degree of result, as the score value of the Search Results；According to different test evaluation algorithms, according to dividing for mentioned above searching results
Value, scans for engine search result test evaluation automatically.
Search engine test evaluation method provided in an embodiment of the present invention, by using real user search and click logs
As training data, using Word2Vec training patterns, the similarity between search word and Search Results, the i.e. degree of correlation are obtained
Value, using similarity as the score value of test search engine search quality, so as to calculate automatically search for knot on the search engine
The score value of fruit, while automatic test is realized, it is possible to increase test result accuracy, and save the testing time.
The embodiment of the present invention provides a kind of search engine test evaluation device 50, as shown in figure 8, described device 50 includes：
Determining unit 501, acquiring unit 502, processing unit 503, wherein,
The determining unit 501, for determining training pattern, the instruction according to preset model and at least one training data
Practicing data includes：At least one search word and at least one set of Search Results corresponding with least one search word, each
Search word correspondence last set result；
The acquiring unit 502, for obtaining at least one first search knots of first search word in the first search engine
Really；
The processing unit 503, for described at least one first Search Results and first search word to exist respectively
The correlation results of the training pattern are compared, and obtain at least one degree of correlation of at least one first Search Results
Value；It is additionally operable to respectively carry out at least one relevance degree test evaluation by default test evaluation algorithm, obtains described
The test evaluation result of the first search engine.
Alternatively, the determining unit 501, at least one training is determined for the search data according at least one user
Data；
The processing unit, at least one training data to be carried out into integration treatment, obtains the first training data；
It is additionally operable to be trained first training data by word2vec models, obtains training pattern.
Alternatively, the processing unit 503, is processed for the search data at least one user, is obtained
Each search knot in second search word the second Search Results corresponding with second search word and second Search Results
The number of clicks of fruit, second search word includes：At least one search word, second Search Results include：It is at least one set of
Search Results, each search word correspondence last set result, at least one Search Results are included per last set result；Also use
In the number of clicks according to each Search Results in second Search Results by all identicals in second search word
The corresponding Search Results of search word are arranged, and are additionally operable to by second Search Results after arrangement and second search
Word generates at least one training data.
Alternatively, the 3rd search word is identical search word in second search word, and the 3rd Search Results are and the 3rd
The corresponding Search Results of search word；
The processing unit 503, for the number of clicks according to each Search Results in the 3rd Search Results from greatly to
It is small to carry out arrangement the 3rd Search Results, it is additionally operable to described by the 3rd Search Results after arrangement and the generation of the 3rd search word
Second training data.
Alternatively, the acquiring unit 502, the search data for obtaining at least one user, wherein, user's
Search data include：User searches for and click logs.
Specifically, the understanding of search engine test evaluation device provided in an embodiment of the present invention may be referred to above-mentioned search drawing
The explanation of test evaluation embodiment of the method is held up, the embodiment of the present invention will not be repeated here.
In actual applications, determining unit 501, acquiring unit 502 and processing unit 503 can be surveyed by positioned at search engine
Central processing unit (Central Processing Unit, CPU), microprocessor (Micro in examination evaluating apparatus
Processor Unit, MPU), digital signal processor (Digital Signal Processor, DSP) or field-programmable
Gate array (Field Programmable Gate Array, FPGA) etc. is realized.
Search engine test evaluation device provided in an embodiment of the present invention, obtains search word and is tied with search using preset model
Relevance degree between fruit, using relevance degree as the score value of test search engine search quality, so as to calculate automatically at this
The score value of Search Results on search engine, while automatic test is realized, it is possible to increase test result accuracy, and save
Testing time.
It should be understood by those skilled in the art that, embodiments of the invention can be provided as method, system or computer program
Product.Therefore, the present invention can be using the shape of the embodiment in terms of hardware embodiment, software implementation or combination software and hardware
Formula.And, the present invention can be used can use storage in one or more computers for wherein including computer usable program code
The form of the computer program product implemented on medium (including but not limited to magnetic disk storage and optical memory etc.).
The present invention is the flow with reference to method according to embodiments of the present invention, equipment (system) and computer program product
Figure and/or block diagram are described.It should be understood that every first-class during flow chart and/or block diagram can be realized by computer program instructions
The combination of flow and/or square frame in journey and/or square frame and flow chart and/or block diagram.These computer programs can be provided
The processor of all-purpose computer, special-purpose computer, Embedded Processor or other programmable data processing devices is instructed to produce
A raw machine so that produced for reality by the instruction of computer or the computing device of other programmable data processing devices
The device of the function of being specified in present one flow of flow chart or multiple one square frame of flow and/or block diagram or multiple square frames.
These computer program instructions may be alternatively stored in can guide computer or other programmable data processing devices with spy
In determining the computer-readable memory that mode works so that instruction of the storage in the computer-readable memory is produced and include finger
Make the manufacture of device, the command device realize in one flow of flow chart or multiple one square frame of flow and/or block diagram or
The function of being specified in multiple square frames.
These computer program instructions can be also loaded into computer or other programmable data processing devices so that in meter
Series of operation steps is performed on calculation machine or other programmable devices to produce computer implemented treatment, so as in computer or
The instruction performed on other programmable devices is provided for realizing in one flow of flow chart or multiple flows and/or block diagram one
The step of function of being specified in individual square frame or multiple square frames.
The above, only presently preferred embodiments of the present invention is not intended to limit the scope of the present invention.
Claims (10)
1. a kind of search engine test evaluation device, it is characterised in that described device includes：Determining unit, acquiring unit, treatment
Unit, wherein,
The determining unit, for determining training pattern, the training data according to preset model and at least one training data
Including：At least one search word and at least one set of Search Results corresponding with least one search word, each search word
Correspondence last set result；
The acquiring unit, for obtaining at least one first Search Results of first search word in the first search engine；
The processing unit, for respectively by described at least one first Search Results with first search word in the training
The correlation results of model are compared, and obtain at least one relevance degree of at least one first Search Results；Also use
In test evaluation is carried out at least one relevance degree respectively by default test evaluation algorithm, first search is obtained
The test evaluation result of engine.
2. device according to claim 1, it is characterised in that
The determining unit, at least one training data is determined for the search data according at least one user；
The processing unit, at least one training data to be carried out into integration treatment, obtains the first training data；Also use
First training data is trained in by word2vec models, obtains training pattern.
3. device according to claim 2, it is characterised in that the processing unit, at least one user
Search data processed, obtain the second search word the second Search Results corresponding with second search word and described second
The number of clicks of each Search Results in Search Results, second search word includes：At least one search word, described second
Search Results include：At least one set of Search Results, each search word correspondence last set result, include per last set result
At least one Search Results；It is additionally operable to described according to the number of clicks of each Search Results in second Search Results
The corresponding Search Results of all identical search words are arranged in two search words, are additionally operable to be searched by described second after arrangement
Hitch fruit and second search word generate at least one training data.
4. device according to claim 3, it is characterised in that the 3rd search word is that identical is searched in second search word
Rope word, the 3rd Search Results are Search Results corresponding with the 3rd search word；
The processing unit, is arranged from big to small for the number of clicks according to each Search Results in the 3rd Search Results
The 3rd Search Results are arranged, is additionally operable to generate second training by the 3rd Search Results after arrangement and the 3rd search word
Data.
5. the device according to any one of claim 2 to 4, it is characterised in that the acquiring unit, for obtain it is described extremely
Few search data of user, wherein, the search data of user include：User searches for and click logs.
6. a kind of search engine test evaluation method, it is characterised in that methods described includes：
Training pattern is determined according to preset model and at least one training data, the training data includes：At least one search
Word and at least one set of Search Results corresponding with least one search word, each search word correspondence last set result；
At least one first Search Results of first search word in the first search engine are obtained, respectively by described at least one first
Search Results are compared with first search word in the correlation results of the training pattern, obtain described at least one the
At least one relevance degree of one Search Results；
Test evaluation is carried out at least one relevance degree by default test evaluation algorithm respectively, described first is obtained and is searched
The test evaluation result that index is held up.
7. method according to claim 6, it is characterised in that described true according to preset model and at least one training data
Determine training pattern, including：
Search data according at least one user determine at least one training data；
At least one training data is carried out into integration treatment, the first training data is obtained；
First training data is trained by word2vec models, obtains training pattern.
8. method according to claim 7, it is characterised in that the search data according at least one user determine to
A few training data, including：
Search data at least one user are processed, and obtain the second search word corresponding with second search word
The number of clicks of each Search Results in second Search Results and second Search Results, second search word includes：
At least one search word, second Search Results include：At least one set of Search Results, each search word correspondence last set
As a result, at least one Search Results are included per last set result；
Number of clicks according to each Search Results in second Search Results will be all identical in second search word
The corresponding Search Results of search word arranged, given birth to by second Search Results after arrangement and second search word
Into at least one training data.
9. method according to claim 8, it is characterised in that the 3rd search word is that identical is searched in second search word
Rope word, the 3rd Search Results are Search Results corresponding with the 3rd search word；
Number of clicks according to each Search Results in the 3rd Search Results is arranged, and obtains the of the 3rd search word
Two training datas, including：
Number of clicks according to each Search Results in the 3rd Search Results carries out arrangement the 3rd search knot from big to small
Really, second training data is generated by the 3rd Search Results after arrangement and the 3rd search word.
10. the method according to any one of claim 7 to 9, it is characterised in that in the searching according at least one user
Before rope data determine at least one training data, also include：
The search data of at least one user are obtained, wherein, the search data of user include：User searches for and clicks on day
Will.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611228051.5A CN106777248A (en) | 2016-12-27 | 2016-12-27 | A kind of search engine test evaluation method and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611228051.5A CN106777248A (en) | 2016-12-27 | 2016-12-27 | A kind of search engine test evaluation method and apparatus |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106777248A true CN106777248A (en) | 2017-05-31 |
Family
ID=58921710
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611228051.5A Withdrawn CN106777248A (en) | 2016-12-27 | 2016-12-27 | A kind of search engine test evaluation method and apparatus |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106777248A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110472034A (en) * | 2019-08-21 | 2019-11-19 | 北京百度网讯科技有限公司 | Detection method, device, equipment and the computer readable storage medium of question answering system |
| WO2020135059A1 (en) * | 2018-12-29 | 2020-07-02 | 深圳Tcl新技术有限公司 | Search engine evaluation method, apparatus and device, and readable storage medium |
| CN112100528A (en) * | 2020-09-09 | 2020-12-18 | 北京三快在线科技有限公司 | Method, device, equipment and medium for training search result scoring model |
| CN114281935A (en) * | 2021-09-16 | 2022-04-05 | 腾讯科技（深圳）有限公司 | Training method, device, medium and equipment for search result classification model |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090119090A1 (en) * | 2007-11-01 | 2009-05-07 | Microsoft Corporation | Principled Approach to Paraphrasing |
| CN104933047A (en) * | 2014-03-17 | 2015-09-23 | 北京奇虎科技有限公司 | Method and device for determining value of search term |
| CN105573887A (en) * | 2015-12-14 | 2016-05-11 | 合一网络技术(北京)有限公司 | Quality evaluation method and device of search engine |
| CN106204053A (en) * | 2015-05-06 | 2016-12-07 | 阿里巴巴集团控股有限公司 | The misplaced recognition methods of categories of information and device |
-
2016
- 2016-12-27 CN CN201611228051.5A patent/CN106777248A/en not_active Withdrawn
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090119090A1 (en) * | 2007-11-01 | 2009-05-07 | Microsoft Corporation | Principled Approach to Paraphrasing |
| CN104933047A (en) * | 2014-03-17 | 2015-09-23 | 北京奇虎科技有限公司 | Method and device for determining value of search term |
| CN106204053A (en) * | 2015-05-06 | 2016-12-07 | 阿里巴巴集团控股有限公司 | The misplaced recognition methods of categories of information and device |
| CN105573887A (en) * | 2015-12-14 | 2016-05-11 | 合一网络技术(北京)有限公司 | Quality evaluation method and device of search engine |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020135059A1 (en) * | 2018-12-29 | 2020-07-02 | 深圳Tcl新技术有限公司 | Search engine evaluation method, apparatus and device, and readable storage medium |
| CN110472034A (en) * | 2019-08-21 | 2019-11-19 | 北京百度网讯科技有限公司 | Detection method, device, equipment and the computer readable storage medium of question answering system |
| CN110472034B (en) * | 2019-08-21 | 2022-11-15 | 北京百度网讯科技有限公司 | Detection method, device and equipment of question-answering system and computer readable storage medium |
| CN112100528A (en) * | 2020-09-09 | 2020-12-18 | 北京三快在线科技有限公司 | Method, device, equipment and medium for training search result scoring model |
| CN114281935A (en) * | 2021-09-16 | 2022-04-05 | 腾讯科技（深圳）有限公司 | Training method, device, medium and equipment for search result classification model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10956771B2 (en) | Image recognition method, terminal, and storage medium | |
| CN107066497A (en) | A kind of searching method and device | |
| CN106156310A (en) | A kind of picture processing apparatus and method | |
| CN107341187A (en) | Search processing method, device, equipment and computer-readable storage medium | |
| CN105719659A (en) | Recording file separation method and device based on voiceprint identification | |
| CN106777248A (en) | A kind of search engine test evaluation method and apparatus | |
| CN106356065A (en) | Mobile terminal and voice conversion method | |
| CN107241212A (en) | A kind of log processing method and device, equipment | |
| CN104991772B (en) | Remote operation bootstrap technique and device | |
| CN106383780A (en) | Monkey test method and device | |
| CN107132931A (en) | A kind of searching method and server | |
| CN106653011A (en) | Voice control method, voice control device and terminal | |
| CN106778887A (en) | The terminal and method of sentence flag sequence are determined based on condition random field | |
| CN106776906A (en) | One kind application clustering method and device | |
| CN106535228A (en) | System upgrading device and method | |
| CN106024013A (en) | Voice data searching method and system | |
| CN106376004A (en) | Information processing method and terminal | |
| CN106527685A (en) | Control method and device for terminal application | |
| CN106547674A (en) | A kind of fingerprint input method, device and terminal | |
| CN106855946A (en) | A kind of image information acquisition method and apparatus | |
| CN106878093A (en) | One kind is without response log analytic method and terminal | |
| CN106777250A (en) | A kind of word segmentation result system of selection and device | |
| CN106569670A (en) | Application processing apparatus and method | |
| CN106777254A (en) | One kind application search method and device | |
| CN106774983A (en) | A kind of input method and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20170531 |
|
| WW01 | Invention patent application withdrawn after publication |