Summary of the invention
In view of the problem above of the prior art, the embodiment of the present invention provides a kind of novel based on UI automation and OCR
Data processing system, method, equipment and device, be no longer limited by the available resources of API and call frequency and number.
For market business, multi-source comparison is to guarantee the continual and steady correct available effective means of market data.With
The development of terminal technology and software technology, current market data multi-source alignment schemes need to analyze other market data
The interface rules of data source on APP (application program), crawl his source data by interface.But it after interface change, needs pair
Logic follow-up modification is crawled, to cause maintenance cost height.In addition, the promotion of the anti-ability of crawling with other systems, can divide
The data source that interface is precipitated is also more and more limited.
On the other hand, OCR (optical character identification) refers to that electronic equipment determines it by mode dark, bright in detection image
Then shape is translated into the process of computword by shape with character identifying method.But OCR technique often relies on use
Family is for the scanning of content to be identified or specifying for images to be recognized.Thus, OCR technique at present can't be automatically to big spirogram
As carrying out Text region.
Automatic test refers to that the automation of software test, software test are exactly operating system or application on a preset condition based
Program assesses operation result, and preparatory condition should include normal condition and exceptional condition.Automatic test includes that unit automation is surveyed
Try (data analysis layer), interface automatic test (Business Logic) and UI automatic test (gui interface layer).
It is all to be presented to use on market data APP that present inventor, which notices the market data to be crawled usually,
Shown data in the user interface at family.In turn, present inventor is proposed first by means of UI automatization simulation user
It operates market data APP and enters user interface to be captured, screenshotss call OCR identification service, identify the text envelope in screenshotss
Breath, then extracts the content for needing to check field, can finally be compared to more market data of crawl.Especially,
Truncated picture is aggregated into cloud, by the powerful data counts ability of server, data grabber can be greatlyd improve
Efficiency.Term " UI automation " refers to by writing program, realizes to the interactive actions such as the electric shock of element, dragging on APP, thus
Realize the process for utilizing machine simulation user's operation APP.In this way, the data grabber of embodiment according to the present invention is no longer limited by
The available resources and calling frequency and number of API.
A kind of data processing system based on UI automation and OCR of embodiment according to the invention, comprising: it is configured
Regularly to be respectively enterd one or more about object of interest by means of UI automatic operation one or more application program
A user interface, and obtain one or more image corresponding with one or more of user interfaces;And processor,
It is configured as that OCR recognizer is called to identify and extract related to the object of interest in the one or more image respectively
Data information.
A kind of data processing method based on UI automation and OCR of embodiment according to the invention, comprising: regularly borrow
Help the one or more application program in UI automatic operation terminal and respectively enters one or more about object of interest
User interface;Obtain one or more image corresponding with one or more of user interfaces;Also, OCR identification is called to calculate
Method identifies respectively and extracts data information relevant to the object of interest in the one or more image.
A kind of data processing equipment based on UI automation and OCR of embodiment according to the invention, comprising: processor;
And memory, it is stored thereon with executable instruction, wherein the executable instruction makes the processor upon being performed
Execute method above-mentioned.
A kind of machine readable storage medium of embodiment according to the invention, is stored thereon with executable instruction, wherein institute
State executable instruction makes machine execute method above-mentioned upon being performed.
A kind of data processing equipment based on UI automation and OCR of embodiment according to the invention, comprising: obtain mould
Block, for regularly being respectively enterd by means of the one or more application program in UI automatic operation terminal about interested right
One or more user interfaces of elephant, and obtain one or more image corresponding with one or more of user interfaces;With
And extraction module, for call OCR recognizer to identify respectively and extract in the one or more image with it is described interested
The relevant data information of object.
From the above, it can be seen that the scheme of the embodiment of the present invention enters specified use using UI automatic operation application program
Family interface, screenshotss can obtain related data, crawl without analyzing his source interface, it is not easy to be prohibited by the anti-system of climbing of other side
Only.Therefore, compared with prior art, what the scheme of the embodiment of the present invention can be no longer limited by API when data crawl can
With resource and call frequency and number.After frame according to an embodiment of the present invention is built simultaneously, it can be adapted for stock, base
The comparison of golden various businesses data, without by the variation of various businesses data-interface greatly and the biggish shadow of data structure mapping
It rings, significantly reduces maintenance cost.In addition, also solve in the prior art can only be from service for the scheme of the embodiment of the present invention
Whether also end checks whether market data correctly can be used, be unable to ensure client and show correct problem.In addition, according to the present invention
Technology can be applied not only to the scene of market comparing, can be also used for intelligent monitoring analyze competing product APP activity and
State.
Specific embodiment
Theme described herein is discussed referring now to example embodiment.It should be understood that discussing these embodiments only
It is in order to enable those skilled in the art can better understand that being not to claim to realize theme described herein
Protection scope, applicability or the exemplary limitation illustrated in book.It can be in the protection scope for not departing from present disclosure
In the case of, the function and arrangement of the element discussed are changed.Each example can according to need, omit, substitute or
Add various processes or component.For example, described method can be executed according to described order in a different order, with
And each step can be added, omits or combine.In addition, feature described in relatively some examples is in other examples
It can be combined.
As used in this article, term " includes " and its modification indicate open term, are meant that " including but not limited to ".
Term "based" indicates " being based at least partially on ".Term " one embodiment " and " embodiment " expression " at least one implementation
Example ".Term " another embodiment " expression " at least one other embodiment ".Term " first ", " second " etc. may refer to not
Same or identical object.Here may include other definition, either specific or implicit.Unless bright in context
It really indicates, otherwise the definition of a term is consistent throughout the specification.
In the following, in conjunction with each embodiment of attached drawing detailed description of the present invention.
Fig. 1 shows the data processing system 10 based on UI automation and OCR of one embodiment according to the invention.Figure
System 10 shown in 1 includes terminal 100, has been equipped with one or more application program APP1, APP2 in advance on the terminal 100
Deng.Terminal 100 can be the client mobile communication end of mobile phone, tablet computer etc., be also possible to that the application can be run
The desktop computer or laptop of program.For stock market scene, the application program may include, but be not limited to pay
The market data APP such as treasured, self-selected stock, snowball, straight flush, rich way ox ox.It is described to apply journey for intelligent customer service robot scene
Sequence may include, but be not limited to the intelligent customer services robots such as Alipay intelligent customer service robot, Jingdone district intelligent customer service robot
APP.Those skilled in the art will appreciate that after reading and understanding present specification other application scenarios of the invention with
And the application program under various application scenarios.
In exemplary scene shown in Fig. 1, terminal 100 is configured as regularly (for example, each day of trade, each transaction
Per every other hour or a few houres during period, trading session, etc.) by means of UI automatic operation one or more application program
APP1, APP2 ... respectively enter one or more user interfaces about object of interest, and obtain with it is one or more of
The corresponding one or more image of user interface.For stock market scene, the object of interest can be user's care
Certain stock or stock code.The common testing tool of UI automatic test have UFT, Robot Framework, Selenium,
Appium etc.." the UI automation " can be programmed for specific market data APP.For example, if described interested
Object is set to " Alibaba ", and after terminal 100 automates login " self-selected stock " APP by means of UI, automatically clicking is most upper
Side search column and input " Alibaba ", obtain stock search result card (a kind of graphical user circle in relation to " Alibaba "
Face).Meanwhile terminal 100 can automate by means of UI and log in " Alipay " APP, after " wealth " of clicking bottom, click most
The search column of top simultaneously automatically enters " Alibaba ", finally obtains the stock search result card in relation to " Alibaba ".It can be with
After the results page of search successfully renders, the corresponding image of search result card is obtained by taking pictures.
In exemplary scene shown in Fig. 1, terminal 100 includes one or more processors 120, is configured as calling
OCR recognizer identifies respectively and extracts data information relevant to the object of interest in the one or more image.
Known OCR recognizer is all available.For example, processor 120 can be identified and be mentioned by available OCR recognizer
It takes in the search result image for acquisition of taking pictures and for example as " Alibaba " of object of interest related market data.
In exemplary scene shown in Fig. 1, terminal 100 further includes memory 110.If search result card by screenshotss,
Memory 110 can store the screenshotss.Thus, screenshotss image in the accessible memory 110 of processor 120 and by available
OCR recognizer the screenshotss image stored is identified and is extracted.
In exemplary scene shown in Fig. 1, data processing system 10 further includes cloud server 200, independently of terminal
100, and including memory 210 and one or more processors 220.Terminal 100 will be obtained by way of wirelessly or non-wirelessly
The one or more image taken sends the memory 210 of server 200 to, and memory 210 stores the image received.Example
Such as, terminal 100 can by screenshotss content uploading to Operation Support System (OSS) and obtain image URL (unified resource position
Symbol), then the URL of image is sent to the memory 210 of server 200.In this case, by the processing of server 200
Device 220 realizes the identification and extraction to described image.For example, processor 220 judges whether this takes turns the time interval of movement images
Meet pre-provisioning request;If meeting pre-provisioning request, available OCR recognizer is just called, converts text information for image information,
Extract the data (for example, current share prices of " Alibaba ") of corresponding field.Thus it is possible to make full use of the powerful of server
Operational capability fast and efficiently identifies and extracts data information relevant to the object of interest in described image.
Under the scene for carrying out multi-source comparison to market data, processor 120,220 be can be configured as to from multiple
Application program (such as Alipay, self-selected stock, snowball, straight flush, rich way ox ox etc.) with the object of interest (such as ST long
It is raw) relevant data information (such as current share prices) is compared.Fig. 4 shows Alipay (ant wealth), rich way ox ox, snow
Five sections of ball, straight flush and self-selected stock market data APP synchronizations are obtained about the long-living stock information of ST.In such case
Under, processor 120,220, which can be configured as, calls available OCR recognizer, comes from Alipay (ant from shown in Fig. 4
Wealth), rich way ox ox, snowball, straight flush and self-selected stock five width images in identify and extract the long-living current share prices of ST, go forward side by side
Row laterally compares, to judge whether various market data APP are identical for the share price of same B shareB.If processor 120,
220 determine that a certain or certain several market data APP (are greater than one with other for such as displaying share price of designated speculative stock
It is a) the displaying share price of market data APP is inconsistent, then signal an alert.For example, if alarm signal indicates Alipay (ant
Ant wealth) long-living about the ST long-living share price of four ST of stock price and other is inconsistent, then exploitation tester can be with
Start the inspection for Alipay APP, where finding the problem.
In addition, although the case where OCR recognition capability is continuously improved, and there are still wrong identifications at present.According to this hair
Bright embodiment, if processor 120,220 passes through to the image for identical object of interest from a plurality of application programs
Recognition result, which is compared, determines that the recognition result greater than a application program is consistent, then processor 120,220 can make
The image recognition result of those inconsistent application programs is corrected with the image recognition result from these application programs.This
Sample can improve the identification quality of OCR recognizer used.
In addition, technology according to the present invention can be applied not only to the scene of market comparing, intelligence can be also used for
The activity and state of the competing product APP of monitoring analysis.For example, terminal 100 can be configured as by the way of UI automation, open all
It such as the application program of Jingdone district finance APP, clicks to enter that bank is selected in homepage, shows one in the selected page of bank of expansion
A little Determined products and its earning rate.In addition, the marketing position of rolling also can use UI automation and be clicked, into page
Screenshot uploads behind the inside of face.Processor 120,220 can call available OCR recognizer to identify sense from the image obtained
The information of interest.Then, processor 120,220 can by recognition result that this is obtained from a application program and it is last from
The recognition result obtained with a application program is compared.If consistent, just without alarm;If inconsistent (for example, competing product
The new product of APP restocking or operation marketing activity), processor 120,220 can send alarm at this time, to notify related personnel
It goes to and carries out checking analysis.
Those skilled in the art will appreciate that, the above-mentioned institute that processor is realized it is functional can by individual processor Lai
Implement, can also be implemented respectively by multiple processors.
Fig. 2 shows the data processing methods 300 based on UI automation and OCR of one embodiment according to the invention
Overview flow chart.The data processing method 300 of Fig. 2 is described in detail below with reference to data processing system 10 shown in FIG. 1.
As shown in Fig. 2, in box 310, regularly by means of the one or more application in UI automatic operation terminal 100
Program respectively enters one or more user interfaces about object of interest.
In box 320, one or more image corresponding with one or more of user interfaces is obtained.
In box 340, if obtaining the one or more image by way of taking pictures, for example, by processor 120,
220, call OCR recognizer to identify respectively and extract number relevant to the object of interest in the one or more image
It is believed that breath.
In box 330, if obtaining the one or more image, acquired one or more by way of screenshotss
Image can be stored locally in the memory 110 of terminal 100, or can be stored remotely server 200 beyond the clouds
In memory 210.Then, in box 340, processor 120,220 calls the OCR recognizer to identify respectively and extracts and stored
Data information relevant to the object of interest in one or more image.
In box 350, for example, by processor 120,220, to from the multiple application program with it is described interested
The relevant data information of object is compared.In a further embodiment, the method can also include by this extract with
The relevant data information of the object of interest and it is last extract from same application domain with the object of interest
Relevant data information is compared.
It is corrected based on the result of the comparison from the multiple in box 360 for example, by processor 120,220
The data information of one or more application program in application program.
In box 370, for example, by processor 120,220, the multiple application is come from the result instruction of the comparison
The data information of one or more application program is different from from other application journey in the multiple application program in program
When the data information of sequence, alarm signal is exported.In a further embodiment, the method can also be included in the comparison
Result when indicating the data information that this is extracted and the inconsistent last data information that extracts, export about institute
State the alarm signal of application program.
It can be seen from the above that the scheme of the present embodiment is using market data APP objectively to market data
Displaying is realized based on UI automation and OCR identification technology and can be carried out multi-source data comparison not through api interface.Meanwhile
Also it can improve the identification quality of existing OCR.Although of the invention above with respect to the embodiment of the present invention of market data description
Design also can be applied to obtain any scenes of other data sources.For example, for intelligent customer service robot scene, it is desirable to comment
The problem of system, can be inputed to such as Alipay intelligent customer service robot and competing by Gu Jingpin intelligent customer service robot response rate
Product intelligent customer service robot, after getting the answer of robot, screenshotss are uploaded, analyses and comparison Alipay intelligent customer service robot
With the answer of Jing Pin intelligent customer service robot, and the knowledge base of Alipay intelligent customer service robot can be expanded, promote product
Competitiveness and user experience.
Other modifications
It will be appreciated by those skilled in the art that although in the above embodiments, data processing system 10 includes terminal
100, however, the present invention is not limited thereto.In some other embodiment of the invention, the number based on UI automation and OCR
It can only include memory 110,210 and processor 120,220 according to processing equipment.It is stored on memory 110,210 executable
Instruction, wherein the executable instruction makes processor 120,220 execute method 300 shown in Fig. 2 upon being performed.It is described
Data processing equipment can be independently of terminal 100.Moreover, processor 120,220, which can be configured as controlling terminal 100, is based on UI
The one or more of application programs of automatic operation.In some embodiments, memory 110,210 be additionally configured to storage by
The one or more image that terminal 100 transmits by way of wirelessly or non-wirelessly, for identifying and extracting.
Fig. 3 shows the data processing equipment 400 based on UI automation and OCR of one embodiment according to the invention
Schematic diagram.Device 400 shown in Fig. 3 can use the mode of software, hardware or software and hardware combining to realize.
As shown in figure 3, device 400 may include obtaining module 410 and extraction module 430.Module 410 is obtained for regular
Ground respectively enterd by means of the one or more application program in UI automatic operation terminal about one of object of interest or
Multiple user interfaces, and obtain one or more image corresponding with one or more of user interfaces.Extraction module 430 is used
It is identified respectively in calling OCR recognizer and extracts data relevant to the object of interest in the one or more image
Information.
In a first aspect, device 400 further includes memory module 420, for an acquired width to be stored locally or remotely
Or multiple image.In this case, the extraction module is also used to that OCR recognizer is called to identify and extract respectively and is stored
One or more image in data information relevant to the object of interest.
In second aspect, device 400 further includes comparison module 440, for from the multiple application program with institute
The relevant data information of object of interest is stated to be compared.In a further embodiment, comparison module 440 can be also used for this
The data information relevant to the object of interest of secondary extraction with last time extract from same application domain with it is described
The relevant data information of object of interest is compared.
In the third aspect, device 400 further includes correction module 450, for being come from based on the result of the comparison to correct
The data information of one or more application program in the multiple application program.
In fourth aspect, device 400 further includes alarm module 460, for the result instruction in the comparison from described
The data information of one or more application program is different from coming from its in the multiple application program in multiple application programs
When the data information of his application program, alarm signal is exported.In a further embodiment, alarm module 460 can also be used
Indicate that the data information that this is extracted and the upper data information once extracted are inconsistent in the result in the comparison
When, export the alarm signal about the application program.
Embodiment according to the invention also provides a kind of machine readable storage medium, is stored thereon with executable instruction,
In, the executable instruction makes machine execute method 300 shown in Fig. 2 upon being performed.
The specific embodiment illustrated above in conjunction with attached drawing describes exemplary embodiment, it is not intended that may be implemented
Or fall into all embodiments of the protection scope of claims." exemplary " meaning of the term used in entire this specification
Taste " be used as example, example or illustration ", be not meant to than other embodiments " preferably " or " there is advantage ".For offer pair
The purpose of the understanding of described technology, specific embodiment include detail.However, it is possible in these no details
In the case of implement these technologies.In some instances, public in order to avoid the concept to described embodiment causes indigestion
The construction and device known is shown in block diagram form.
The foregoing description of present disclosure is provided so that any those of ordinary skill in this field can be realized or make
Use present disclosure.To those skilled in the art, the various modifications carried out to present disclosure are apparent
, also, can also answer generic principles defined herein in the case where not departing from the protection scope of present disclosure
For other modifications.Therefore, present disclosure is not limited to examples described herein and design, but disclosed herein with meeting
Principle and novel features widest scope it is consistent.