CN109800049A - A kind of operating method of function service, device, terminal device and storage medium - Google Patents
A kind of operating method of function service, device, terminal device and storage medium Download PDFInfo
- Publication number
- CN109800049A CN109800049A CN201910063690.8A CN201910063690A CN109800049A CN 109800049 A CN109800049 A CN 109800049A CN 201910063690 A CN201910063690 A CN 201910063690A CN 109800049 A CN109800049 A CN 109800049A
- Authority
- CN
- China
- Prior art keywords
- model
- interface
- function service
- sample
- object run
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Landscapes
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses a kind of operating method of function service, device, terminal device and storage mediums.The described method includes: obtaining current business interface and the on-unit instruction of function service；According to predetermined object run model, the corresponding element to be operated of the operational order is identified from the current business interface；It determines the operating position of the element to be operated, and executes the operational order at the operating position.Using this method, it is capable of the applicability of effective enhanced feature business automation operation.
Description
Technical field
The present embodiments relate to field of computer technology more particularly to a kind of operating methods of function service, device, end
End equipment and storage medium.
Background technique
Computer is borne by critically important role in all trades and professions at present, runs can satisfy use on computers
The application system that person's daily life and working service need.Application system frequently includes multiple function services, and operator can be with
Each function service is operated by the human-computer interaction interface of application system, it specifically can be to being shown in people in function service
Element respectively to be operated on machine interactive interface carries out corresponding operation.
Currently, function service to be treated is more in application system, it is only difficult to realize real by operator's manual operation
When operate, there is the operation that function service is realized by programming automation thus to substitute the manual operation of operator.So
And at present application system all can cross-platform operation, it is existing by programming automation function service to be operated
Mode of operation largely defines function service automatic operation often appropriate only to the application system under specific operation platform
The scope of application.
Summary of the invention
The embodiment of the invention provides a kind of operating method of function service, device, terminal device and storage mediums, with energy
The applicability of enough effective enhanced feature business automation operations.
In a first aspect, the embodiment of the invention provides a kind of operating methods of function service, comprising:
Obtain current business interface and the on-unit instruction of function service；
According to predetermined object run model, identify that the operational order is corresponding from the current business interface
Element to be operated；
It determines the operating position of the element to be operated, and executes the operational order at the operating position.
Optionally, the operating method of function service, further includes:
Operation model is constructed and trained, object run model is obtained.
Optionally, the building and training operation model, obtain object run model, comprising:
Construct the operation model that model structure is convolutional neural networks；
Based on the sample elements group training operation model in determined sample elements library, object run model is obtained.
Optionally, sample elements library is determined based on operations described below:
Obtain history service interface and the instruction of corresponding historical operation of the function service；
Determine that the historical operation instructs corresponding historical operation element from the history service interface；
Based on history service interface, historical operation instruction and the historical operation element, a sample elements group is formed, and
The sample elements group is added to sample elements library.
Optionally, the sample elements group includes: sample business as input interface and sample operations instruction, as right
The sample operations element that should be exported；
Wherein, sample business interface includes: interface name information, routing information, source-information and interface dimensions letter
Breath；The sample operations element includes: element term information and element size information.
Second aspect, the embodiment of the invention also provides a kind of operating devices of function service, comprising:
Module is obtained, for obtaining current business interface and the on-unit instruction of function service；
Identification module, described in being identified from the current business interface according to predetermined object run model
The corresponding element to be operated of operational order；
Determining module, for determining the operating position of the element to be operated, and described in executing at the operating position
Operational order.
Optionally, the operating device of function service, further includes:
Training module obtains object run model for constructing and training operation model.
Optionally, the training module, is specifically used for:
Construct the operation model that model structure is convolutional neural networks；
Based on the sample elements group training operation model in determined sample elements library, object run model is obtained.
The third aspect, the embodiment of the invention also provides a kind of terminal devices, comprising:
One or more processors；
Storage device, for storing one or more programs；
One or more of programs are executed by one or more of processors, so that one or more of processors
Realize the operating method of function service provided in an embodiment of the present invention.
Fourth aspect, the embodiment of the invention also provides a kind of computer readable storage mediums, are stored thereon with computer
Program, the program realize the operating method of function service provided in an embodiment of the present invention when being executed by processor.
The embodiment of the invention provides a kind of operating method of function service, device, terminal device and storage mediums, utilize
Above-mentioned technical proposal can be behind the current business interface and on-unit instruction for obtaining function service, according to predetermined
Object run model, the corresponding element to be operated of operational order is identified from current business interface, and in element to be operated
Operating position at execute the operational order to complete corresponding function service.It avoids and function is realized by programming automation
The operation of business realizes the function of corresponding to current business interface and operational order by predetermined object run model
Business effectively improves the applicability of function service automatic operation.
Detailed description of the invention
Fig. 1 is a kind of flow diagram of the operating method for function service that the embodiment of the present invention one provides；
Fig. 2 a is a kind of flow diagram of the operating method of function service provided by Embodiment 2 of the present invention；
Fig. 2 b shows the mailbox interface schematic diagram of the Firefox browser of Mac operating system；
Fig. 2 c shows the mailbox interface schematic diagram under the IE browser of Windows operating system；
Fig. 2 d shows the schematic diagram annotated to the mailbox interface of the Firefox browser of Mac operating system；
Fig. 2 e shows the schematic diagram that the mailbox interface under the IE browser to Windows operating system is annotated；
Fig. 2 f shows the mailbox recognition result schematic diagram of the Firefox browser of MAC operation system；
Fig. 3 is a kind of structural schematic diagram of the operating device for function service that the embodiment of the present invention three provides；
Fig. 4 is a kind of structural schematic diagram for terminal device that the embodiment of the present invention four provides.
Specific embodiment
The present invention is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched
The specific embodiment stated is used only for explaining the present invention rather than limiting the invention.It also should be noted that in order to just
Only the parts related to the present invention are shown in description, attached drawing rather than entire infrastructure.
It should be mentioned that some exemplary embodiments are described as before exemplary embodiment is discussed in greater detail
The processing or method described as flow chart.Although operations (or step) are described as the processing of sequence by flow chart,
Many of these operations can be implemented concurrently, concomitantly or simultaneously.In addition, the sequence of operations can be pacified again
Row.The processing can be terminated when its operations are completed, it is also possible to have the additional step being not included in attached drawing.Institute
Stating processing can correspond to method, function, regulation, subroutine, subprogram etc..
Embodiment one
Fig. 1 is a kind of flow diagram of the operating method for function service that the embodiment of the present invention one provides, and this method can
The case where suitable for operating to function service, is realized specifically, this method is applicable to by way of automation to function
The case where operation of energy business.This method can be executed by the operating device of function service, wherein the device can by software and/
Or hardware realization, and be typically integrated on terminal device, terminal device includes but is not limited in the present embodiment: computer, platform
The equipment such as formula computer, server.
As shown in Figure 1, a kind of operation for function service that the embodiment of the present invention one provides, includes the following steps:
S101, the current business interface for obtaining function service and on-unit instruction.
In the present embodiment, function service can be understood as the affairs handled.Current business interface can manage
Solution is business interface shown by terminal device.Operational order can be understood as the order operated to current business interface.
On-unit instruction can be understood as the operational order that current business interface will be performed.
Before the automatic operation for carrying out function service, this step can obtain current business interface and pending first
Operational order, in order to carry out automatic operation to function service based on acquired current business interface and operational order.
The specific means for obtaining current business interface and pending operation instruction is not defined herein, art technology
Personnel can choose suitable technological means according to the actual situation and obtain current business interface and operational order.
Illustratively, this step can obtain current business interface by way of screenshotss；This step can pass through detection
The state of input equipment obtains the on-unit instruction of user's input.Wherein, the concrete form of operational order is not construed as limiting,
As long as corresponding with the storage form of corresponding content in predetermined object run model.Corresponding content can be grasped for history
It instructs.Such as, the identification information of the instruction name or operational order of the available user's input of this step.Instruction name can be
" logging in mailbox in the firefox browser under Mac operating system ".Identification information can be understood as being capable of unique identification operation
The information of instruction.Such as with 0 mark " logging in mailbox in the firefox browser under Mac operating system ".
S102, according to predetermined object run model, identify the operational order from the current business interface
Corresponding element to be operated.
In the present embodiment, object run model can be understood as the predetermined mould before the operation for carrying out function service
Type.Element to be operated can be understood as the page elements for the waiting operation that respective operations in current business interface instruct.Wait operate
Element may include element term and operating position.Operating position can be characterized by coordinate value.
The method of determination of object run model is not construed as limiting herein, can be to be in advance based on history service interface, history behaviour
Make the model of instruction and corresponding historical operation element direct construction；Or history service interface, historical operation are instructed
The model of machine learning formation is carried out with corresponding historical operation element.The object run model can be used for based on current business
Interface and operational order determine the element to be operated in current business interface.Wherein, machine learning can be understood as allowing computer
The thing seen is learnt as people, and according to the result of study to object and progress Classification and Identification in kind.
After obtaining current business interface and operational order, this step can load predetermined object run model,
According to the object run model, the corresponding element to be operated of the operational order is identified from the current business interface.Example
Property, if the object run model is the model formed by machine learning, this step can according to the content learnt, from
The element to be operated of respective operations instruction is identified in current business interface.The content of object run model study does not limit
Fixed, those skilled in the art may be set according to actual conditions.It such as, can be by history industry of the function service under each operating platform
Business interface and historical operation instruction are handled, and form object run model with training, so that enhanced feature business automation is grasped
The applicability of work.
It is understood that there may be multiple job applications interfaces, multiple job applications at the current business interface obtained
It include the corresponding business interface of function service in interface.This step can be based on object run model from current business interface
Identify the corresponding element to be operated of operational order.It is then based on element to be operated and executes operational order.
S103, determine described in element to be operated operating position, and the operational order is executed at the operating position.
In the present embodiment, operating position can be understood as the execution position of the operational order of element to be operated.
It is identifying after operating element, this step can be read wait operate the operating position in element, and in the operation
Operational order is executed at position to complete function service.
This step executes operational order at operating position can drive input equipment to execute operational order for terminal device.
Different operational orders can correspond to different input equipments.
Operational order can be instruction set, determine after operating element, in the operative position of corresponding element to be operated
Place is set, corresponding instruction execution can be chosen from operational order.
A kind of operating method for function service that the embodiment of the present invention one provides can obtain function using the above method
Behind the current business interface of business and on-unit instruction, according to predetermined object run model, from current business
The corresponding element to be operated of operational order is identified in interface, and is executed the operation at the operating position of element to be operated and referred to
It enables to complete corresponding function service.The operation that function service is realized by programming automation is avoided, by predetermined
Object run model realizes the function service for corresponding to current business interface and operational order, effectively improves function service
The applicability of automatic operation.
Further, in the present embodiment function service operating method, further includes: construct simultaneously training operation model, obtain
Object run model.
Wherein, the building of operation model is not construed as limiting herein, and those skilled in the art can select according to practical application scene
Different building means are taken to construct operation model.Needs can be chosen according to the demand of practical application scene after operation model building
Sample be trained, obtain object run model, with based on object run model complete function service automatic operation.
Embodiment two
Fig. 2 a is a kind of flow diagram of the operating method of function service provided by Embodiment 2 of the present invention, the present embodiment
Two optimize on the basis of the various embodiments described above.In the present embodiment, it will construct and training operation model, acquisition target are grasped
Make model, be further detailed as: building model structure is the operation model of convolutional neural networks；
Based on the sample elements group training operation model in determined sample elements library, object run model is obtained.
Further, the present embodiment, which also optimizes, determines sample elements library based on operations described below:
Obtain history service interface and the instruction of corresponding historical operation of the function service；
Determine that the historical operation instructs corresponding historical operation element from the history service interface；
Based on history service interface, historical operation instruction and the historical operation element, a sample elements group is formed, and
The sample elements group is added to sample elements library.
The not yet detailed content of the present embodiment please refers to embodiment one.
As shown in Figure 2 a, the operating method of a kind of function service provided by Embodiment 2 of the present invention, includes the following steps:
S201, the history service interface for obtaining the function service and the instruction of corresponding historical operation.
In the present embodiment, history service interface can be understood as in object run model construction stage, the function service
Corresponding business interface.Historical operation instruction can be understood as object run model construction stage, the corresponding behaviour of the function service
It instructs.
This step can obtain history service interface and the instruction of corresponding historical operation of the function service first.Wherein,
History service interface can be formed the corresponding business interface screenshot of function service for user.Historical operation instruction can be user
In the operational order that object run model training stage is inputted by input equipment.
After obtaining history service interface and historical operation instruction, it can be used for being formed sample elements library, to be based on sample
Element database obtains object run model.
S202, determine that the historical operation instructs corresponding historical operation element from the history service interface.
In the present embodiment, historical operation element can be understood as the behaviour that historical operation instruction is corresponded in history service interface
Make element.Corresponding function industry can be completed by executing historical operation instruction at the historical operation position of historical operation element
Business.
It is understood that historical operation element in history service interface can for user according to function service need into
Rower note.This step can identify the historical operation member for determining that historical operation instruction is corresponding from history service interface
Element, to be used to form sample elements library, to obtain object run model based on sample elements library.
It is understood that historical operation instruction can be instruction set, correspondingly, identified from history service interface
The number of historical operation element can be identical as the number instructed in instruction set.Illustratively, historical operation, which instructs, may include
It inputs user name, input password and clicks and log in, correspondingly, historical operation element, which can correspond to user name, obtains frame, password
Obtain frame and login frame.Historical operation instruction can be not construed as limiting herein with historical operation element there are certain corresponding relationship.
Element to be operated can be effectively determined when application target operation model executes the operation of function service based on the corresponding relationship
Operating position at need execute operational order.
S203, it is based on history service interface, historical operation instruction and the historical operation element, forms a sample elements
Group, and the sample elements group is added to sample elements library.
In the present embodiment, sample elements group can be understood as based on history service interface, historical operation instruction and history
The element group for being used to train operation model of operation element building.Sample elements library is understood that be formed by sample elements group
Element database.Sample elements library can store the corresponding various forms of history service interfaces of the function service, historical operation refers to
It enables and historical operation element.As historical operation instructs corresponding historical operation element in various sizes of history service interface.
Further, the sample elements group includes: that sample business as input interface and sample operations instruct, as
The sample operations element of corresponding output；
Wherein, sample business interface includes: interface name information, routing information, source-information and interface dimensions letter
Breath；The sample operations element includes: element term information and element size information.
Wherein, business interface when sample business interface can be understood as being formed sample elements group.Sample operations instruction can
With the operational order for being interpreted as being formed when sample elements group.Sample operations element can be understood as being formed behaviour when sample elements group
Make element.Interface name information can be understood as the title at the sample business interface.Routing information can be understood as the sample industry
The path at business interface.Routing information may include local path and network path.Source-information can be understood as the sample business
The source at interface.Interface dimensions information can be understood as the size at sample business interface.Herein interface dimensions letter can be determined with unlimited
The concrete form of breath, as interface dimensions information can be characterized by width, height and depth.
Specifically, the specific forming process of sample elements group can be that sample business interface and sample operations are instructed conduct
Input forms object run model using sample operations element as output in order to which operation model is based on the training of sample elements group.
To which object run model can determine corresponding element to be operated based on current business interface and on-unit instruction.
S204, the operation model that building model structure is convolutional neural networks.
In the present embodiment, convolutional neural networks are a kind of comprising convolutional calculation and with the Feedforward Neural Networks of depth structure
Network.
After having determined sample elements library, this step constructs the operation model that model structure is convolutional neural networks.Convolution mind
Level mechanism through network may include data input layer, convolutional calculation layer, excitation layer, pond layer and full articulamentum.
S205, the operation model is trained based on sample elements group in determined sample elements library, obtains object run mould
Type.
After having constructed operation model, this step can train operation model based on sample elements group in sample elements library,
Obtain object run model.
Specifically, in this step in operation model learning sample element database when sample elements group, it can be first by sample
It practises each sample elements group in library to classify, carries out classification learning for operation model.
During operation model learning sample element group, it can use gradient descent algorithm and determine the optimal of learning effect
Value, and using training set and verifying collection come the effect of verification operation model learning, operation is finally determined come final according to test set
The accuracy of model learning, to determine whether operation model has been completed sample elements group study in sample elements library.It will learn
Operation model after the completion of habit is as object run model.
S206, the current business interface for obtaining function service and on-unit instruction.
When application target operation model carries out the operation of function service, the current industry of the available function service of this step
Business interface and on-unit instruction, to determine the element to be operated in current business interface.
S207, according to predetermined object run model, identify the operational order from the current business interface
Corresponding element to be operated.
Object run model in this step can obtain in sample elements library training according to, can be in sample elements library
Sample is formed by including the corresponding each history service interface of function service, historical operation instruction and the historical operation element
Plain group.Based on the object run model can identify the function service current business interface and can be from current business interface
Identify the corresponding element to be operated of the operational order.
S208, determine described in element to be operated operating position, and the operational order is executed at the operating position.
The operating method of function service provided by the invention is described in detail below:
The operating method of function service in the present embodiment may be considered the real-time man-machine interface intelligence based on machine learning
It can identify, and the method operated based on the result after identification.
Existing application system in order to preferably be operator service, it will usually for operator provide close friend man-machine friendship
Mutual interface operates function service consequently facilitating operator is based on human-computer interaction interface.Operator is in order to complete function
The operation of energy business, can be manual operation to the mode of operation of application system.Operator is exported by computer screen
Window interface finds the element to be operated for needing to operate, and treats operation element by keyboard and mouse and carry out corresponding operation.
For small-scale function service by manual operation mode can deal with, but when need to handle it is thousands of very
When to more business operations, especially large-scale or oversized enterprise group is manually operated by operator and carries out function
It is right that the operation of business can not just answer.Thus occur realizing that the operation of function service replaces operator by programming automation
Manual operation.And the mode of operation that programming automation operates function service is commonly available to the application of specific function business
System targetedly handles correlation function business by way of programming automation.Before the operation for carrying out function service, need really
Determine the element to be operated of different function business and the operating method of these elements to be operated, these needs of work operator passes through
The mode of code is write to complete.Which results in there are following defects:
First: due to current application system all can across operating platform operation, exist on the market various
Operating system has the various operating systems at the end PC, also there is the operating system of mobile terminal, different platform end and different operating system
Technical system is all entirely different.In order to determine the operating method of element and these elements to be operated to be operated, technical staff needs
Technical research work is carried out for different platform end and operating system, such workload is very big, and is directed to different platform ends
Also different with the technical system of operating system, learning cost is very high；
Second: the requirement in order to meet end user, application system can frequently be become for the demand of user
More.Once completing the change of application system, application system can not just be carried out by the mode of operation of programming automation correct
It operates, so operator has in time change wait operate correlation function in application system after application system changes
The operating method of element and element to be operated realizes the modification of code and perfect.If this change cannot be completed in time, own
Programming automation realize function service operation just all stagnate, cause the mode of operation of programming automation to respond so stagnant
Property is very strong afterwards, and the cost of operation and maintenance is also very high；
Third: a present huge application system, in addition to that cross-platform can run, frequently, there are also another very for change
Fatal problem, all business application systems can introduce many third party technologies, by merging these technologies come finishing service
The realization of business function in application system.So operator also needs targetedly to be carried out according to different third party technologies
Study, the final automatic operation realized to correlation function business.This results in a very serious drawback, programming automation
The realization of mode of operation is completely dependent on the maturity of third party technology, if not mature technical interface, R&D work can not
It realizes, the automatic operation of application system also just cannot achieve.
Human-computer interaction interface on computer is provided to meet the operational design of people mostly.People can pass through computer
The defeated window interface of screen very easily finds the element to be operated for needing to operate, just change of the element to be operated due to demand at last
Change and quantity or operational change occurs, operator can also very easily find difference, and be wanted according to the design of the page
It asks and is correctly operated.The design of these application systems be all targetedly made according to the vision system of the mankind at
Fruit, purpose are exactly the page exported by computer screen, the vision system of the mankind is allowed easily efficiently to receive, track,
Classification, identification, and it is finally reached the purpose being convenient to operation.
The technology of the present embodiment when being exactly the operation for allowing terminal device to automate function service can as people,
By the content of observation computer screen output, the content of screen output is received, tracked, classified, is identified.To get rid of
The defect of the mode of operation of programming automation.To reach this practical effect, the work step of the present embodiment is divided into two
It is most of: machine recognition study and machine recognition.
One, machine recognition learns
All pages (i.e. history service interface) for the function service for needing automatic operation are acquired, and acquisition
To information give learning model (i.e. operation model), allow computer automatically according to acquisition information carry out all pages of function service
The study in face and page operation element (i.e. historical operation element).The result of study is saved as object run model simultaneously, is supplied
It is used when practical operation.The process flow of this part is as follows:
1. system image pattern acquires
Obtain the history service interface of the function service.Determine the function service that terminal device needs to automatically process,
Screenshot processing is carried out to all history service interfaces of function service.Windows operation system can be compatible with terminal device below
The IE browser of system and Firefox browser in Mac operating system, and automated log on mailbox, for being acquired.Fig. 2 b shows
The mailbox interface schematic diagram of the Firefox browser of Mac operating system is gone out.Fig. 2 c shows the IE of Windows operating system
Mailbox interface schematic diagram under browser.As shown in Figure 2 b, the Firefox of Mac operating system is shown in history service interface 1
The mailbox interface of browser.As shown in Figure 2 c, it is shown under the IE browser of Windows operating system in history service interface 2
Mailbox interface.Fig. 2 b and Fig. 2 c are only the example of image pattern acquisition, and those skilled in the art can be according to actual functional capability industry
The demand of business acquires corresponding history service interface.
2. image pattern annotates
It is instructed according to history service interface and historical operation, user can mark in the different location at history service interface and go through
History operates element, and carries out classification naming to historical operation element.Each history service interface and historical operation instruction can
A corresponding sample elements group is generated, to form sample elements library.
Fig. 2 d shows the schematic diagram annotated to the mailbox interface of the Firefox browser of Mac operating system；Such as figure
Shown in 2d, history service interface can be labeled as Page Name information, the i.e. Firefox browser 11 of Mac operating system, path
Information 12, user name 13, password 14 and login 15.Wherein, routing information 12, user 13, password 14 and login 15 can consider
It is the historical operation element in history service interface.Historical operation can be executed at the corresponding position of historical operation element to refer to
It enables, to complete function service.It is understood that different historical operation instructions can be corresponding with different historical operation members
Element is not construed as limiting herein, and those skilled in the art can establish the corresponding relationship of the two according to actual needs.
Fig. 2 e shows the schematic diagram that the mailbox interface under the IE browser to Windows operating system is annotated.Such as
Shown in Fig. 2 e, history service interface can be labeled as Page Name information, the i.e. IE browser 21 of Windows operating system, road
Diameter information 22, user name 23, password 24 and login 25.Wherein, routing information 22, user name 23, password 24 and log in 25 can be with
The historical operation element being considered in history service interface.
In addition, the storage form of sample elements group is not construed as limiting, table 1 is the storage schematic table of sample elements group.
The storage schematic table of 1 sample elements group of table
It as shown in table 1, include sample operations as input instruction and sample business interface in sample elements group, wherein
Sample business interface includes interface name information i.e. Image Name, routing information i.e. image path, source-information i.e. database name, boundary
Face dimension information, that is, size.It should be noted that may include multiple sample operations instructions, each sample in sample elements group
Operational order can be corresponding with the sample operations element as output, and sample operations element includes element term information i.e. tag name
With element size information, that is, coordinate value.Illustrated only in table 1 in sample elements group there are a sample operations instruction example.
It is understood that sample operations element may include multiple element term information and corresponding element size letter
Breath, includes tag name 1 and the corresponding coordinate value of tag name 1, tag name 2 and label as shown in table 1, in the sample operations element
2 corresponding coordinate values of name and tag name n and the corresponding coordinate value of tag name n etc..It is worth noting that, table 1 is only to sample
The storage schematic table of element group, those skilled in the art can adjust the storage mode of base in this present embodiment according to the actual situation
It is whole, it is not construed as limiting herein, as long as operation model can be trained effectively, to obtain object run model.
3. labeling classification of images
It is split classification automatically according to the sample operations element in sample elements library, learns to use for operation model, this
Place is not defined the specific means of classification.
4. constructing operation model
Utilize convolutional neural networks algorithm, defining operation model.According to the needs of operation model, different convolution can be determined
Block, each convolution kernel can define different convolutional layer and pond layer again.The sample that the model of definition learns as needed
Plain library carries out feature extraction, and model finally enters full link sort device.Operation model is established logic and be can be such that
Convolutional layer is defined, data are standardized；Judge whether to need to increase convolutional layer, if so, increasing volume
Lamination；If it is not, then defining pond layer.Judge whether to need to increase convolution block after defining pond layer, if so, further definition volume
Lamination；If it is not, then terminating.
5. machine sort identification learning
After defining operation model, classification learning is carried out to sample elements library according to operation model, and determine learning effect
Optimal value, learning effect and/or accuracy are to obtain object run model.
6. obtaining object run model
It, can be using the best operation model of learning effect as object run model after the completion of operation model study.
Two, machine recognition
Terminal device can use principle of computer vision, carry out automation behaviour to function service using object run model
Make.Such as current business interface is identified, the corresponding element to be operated of operational order is identified from current business interface, and
Operational order is executed at the operating position of element to be operated.Whole work process may be considered the working method according to " people "
It carries out.Wherein, computer vision technique can be understood as that computer is allowed to go to see object and related content as people.Machine
Identify that specific identification process can be such that
1. object run model loads
Predetermined object run model is loaded, function industry is carried out according to object run model in order to subsequent
The operation of business.
2. obtaining on-unit instruction
According to object run model, user can input pending in the knowledge that object run model learns
Operational order.
3. obtaining the current business interface of function service
The content of screen output may be considered the content that user is seen.Using screen export content carry out acquisition as
Current business interface.
4. current business interface identifies
Because there may be multiple working software windows and other contents on computer screen, object run model can be with
According to the content learnt, the element to be operated for carrying out identification work is found.It determines after operating element, can drive defeated
Enter equipment and executes operational order.
By taking the mailbox operation interface of the Firefox browser of MAC operation system as an example, the displaying of tracking and positioning result is carried out,
And execute operational order.It is understood that Fig. 2 d shows the mailbox interface to the Firefox browser of Mac operating system
The schematic diagram annotated can train to obtain object run model based on Fig. 2 d, so as to based on the target behaviour after training
Make model and identification operation is carried out to the mailbox operation interface of the Firefox browser of MAC operation system.Fig. 2 f shows MAC behaviour
Make the mailbox recognition result schematic diagram of the Firefox browser of system.As shown in figure 2f, include in current business operation interface
Multiple job applications interfaces include the corresponding business interface 110 of function service in multiple job applications interfaces.Object run mould
Type identifies the corresponding element to be operated of operational order from current business interface, such as routing information 120, user name 130, password
140 and log in 150.The each corresponding content of element to be operated in Fig. 2 f, can show in operation, can not also show.Knowing
Not after operating element, operational order can be executed at the operating position of element to be operated.Operational order can be in Fig. 2 f
Instruction set, it can to input user name, input password and logging in.Correspondingly, identified from current business interface wait grasp
Make element, includes user name 130, password 140 and login 150.When executing operational order, can be executed at operating position
Corresponding operational order.
A kind of operating method of function service provided by Embodiment 2 of the present invention embodies and obtains object run model
Operation, the operation that optimization increases determining sample elements library embody further sample elements group.It can be based on going through using this method
History business interface and historical operation, which instruct, determines sample elements library.It is convolutional neural networks based on sample elements training pattern structure
Operation model, obtain object run model.When being automatically brought into operation of function service is being carried out based on object run model, from current
The corresponding element to be operated of operational order is identified in business interface.Then operation is executed at the operating position of element to be operated to refer to
It enables, to complete function service.Object run model is obtained by way of machine learning relative under specific operation platform
For writing program, the workload of developer and the requirement to exploitation personnel specialty technical ability are reduced.Carrying out function service
Operation when, based on trained object run model carry out function service automatic operation, the trained object run
Model is obtained by the training of sample elements library, with strong applicability.
Embodiment three
Fig. 3 is a kind of structural schematic diagram of the operating device for function service that the embodiment of the present invention three provides, which can
The case where suitable for operating to function service, is realized specifically, this method is applicable to by way of automation to function
The case where operation of energy business.Wherein the device can be implemented by software and/or hardware, and be typically integrated on terminal device.
As shown in figure 3, the device includes: to obtain module 31, identification module 32 and determining module 33；
Wherein, module 31 is obtained, for obtaining current business interface and the on-unit instruction of function service；
Identification module 32, for identifying institute from the current business interface according to predetermined object run model
State the corresponding element to be operated of operational order；
Determining module 33 for determining the operating position of the element to be operated, and executes institute at the operating position
State operational order.
In the present embodiment, which passes through the current business interface for obtaining the acquisition function service of module 31 and first wait hold
Capable operational order；Then by identification module 32 according to predetermined object run model, from the current business interface
It is middle to identify the corresponding element to be operated of the operational order；The operation of the element to be operated is determined finally by determining module 33
Position, and the operational order is executed at the operating position.
A kind of operating device of function service is present embodiments provided, it can be at the current business interface for obtaining function service
And after on-unit instruction, according to predetermined object run model, identify that operation refers to from current business interface
Corresponding element to be operated is enabled, and executes the operational order at the operating position of element to be operated to complete corresponding function
Business.The operation for realizing function service by programming automation is avoided, predetermined object run model, realization pair are passed through
The applicability of function service automatic operation should be effectively improved in the function service at current business interface and operational order.
Further, the operating device of function service, further includes: training module is obtained for constructing and training operation model
Obtain object run model.
On the basis of above-mentioned optimization, training module is specifically used for:
Construct the operation model that model structure is convolutional neural networks；
Based on the sample elements group training operation model in determined sample elements library, object run model is obtained.
Based on the above-mentioned technical proposal, the operating device of function service, further includes: library determining module is used for: obtaining the function
The history service interface of energy business and the instruction of corresponding historical operation；
Determine that the historical operation instructs corresponding historical operation element from the history service interface；
Based on history service interface, historical operation instruction and the historical operation element, a sample elements group is formed, and
The sample elements group is added to sample elements library.
Further, the optimization of sample elements group includes: sample business as input interface and sample in the determining module of library
Operational order, the sample operations element as corresponding output；
Wherein, sample business interface includes: interface name information, routing information, source-information and interface dimensions letter
Breath；The sample operations element includes: element term information and element size information.
The operation side of function service provided by any embodiment of the invention can be performed in the operating device of above-mentioned function service
Method has the corresponding functional module of execution method and beneficial effect.
Example IV
Fig. 4 is a kind of structural schematic diagram for terminal device that the embodiment of the present invention four provides.As shown in figure 4, the present invention is real
The terminal device for applying the offer of example four includes: one or more processors 41 and storage device 42；Processor in the terminal device
41 can be one or more, in Fig. 4 by taking a processor 41 as an example；Storage device 42 is for storing one or more programs；
One or more of programs are executed by one or more of processors 41, so that one or more of processors 41 are realized
The operating method of function service as described in any one of embodiment of the present invention.
The terminal device can also include: input unit 43 and output device 44.
Processor 41, storage device 42, input unit 43 and output device 44 in terminal device can by bus or
Other modes connect, in Fig. 4 for being connected by bus.
Storage device 42 in the terminal device is used as a kind of computer readable storage medium, can be used for storing one or more
A program, described program can be software program, computer executable program and module, such as the embodiment of the present invention one or two institutes
Corresponding program instruction/the module of operating method of function service is provided (for example, the operating device of attached function service shown in Fig. 3
In module, comprising: obtain module 31, identification module 32 and determining module 33).Processor 41 is stored in storage dress by operation
Software program, instruction and the module in 42 are set, thereby executing the various function application and data processing of terminal device, i.e., in fact
The operating method of function service in existing above method embodiment.
Storage device 42 may include storing program area and storage data area, wherein storing program area can storage program area,
Application program needed at least one function；Storage data area, which can be stored, uses created data etc. according to terminal device.
It can also include nonvolatile memory in addition, storage device 42 may include high-speed random access memory, for example, at least one
A disk memory, flush memory device or other non-volatile solid state memory parts.In some instances, storage device 42 can
It further comprise the memory remotely located relative to processor 41, these remote memories can be by network connection to setting
It is standby.The example of above-mentioned network includes but is not limited to internet, intranet, local area network, mobile radio communication and combinations thereof.
Input unit 43 can be used for receiving the number or character information of input, and generate the user setting with terminal device
And the related key signals input of function control.Output device 44 may include that display screen etc. shows equipment.
Also, when one or more included program of above-mentioned terminal device is held by one or more of processors 41
When row, program is proceeded as follows:
Obtain current business interface and the on-unit instruction of function service；
According to predetermined object run model, identify that the operational order is corresponding from the current business interface
Element to be operated；
It determines the operating position of the element to be operated, and executes the operational order at the operating position.
Embodiment five
The embodiment of the present invention five provides a kind of computer readable storage medium, is stored thereon with computer program, the journey
For executing the operating method of function service when sequence is executed by processor, this method comprises:
Obtain current business interface and the on-unit instruction of function service；
According to predetermined object run model, identify that the operational order is corresponding from the current business interface
Element to be operated；
It determines the operating position of the element to be operated, and executes the operational order at the operating position.
Optionally, it can be also used for executing function provided by any embodiment of the invention when which is executed by processor
The operating method of business.
The computer storage medium of the embodiment of the present invention, can be using any of one or more computer-readable media
Combination.Computer-readable medium can be computer-readable signal media or computer readable storage medium.It is computer-readable
Storage medium for example may be-but not limited to-the system of electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor, device or
Device, or any above combination.The more specific example (non exhaustive list) of computer readable storage medium includes: tool
There are electrical connection, the portable computer diskette, hard disk, random access memory (Random Access of one or more conducting wires
Memory, RAM), read-only memory (Read Only Memory, ROM), erasable programmable read only memory (Erasable
Programmable Read Only Memory, EPROM), flash memory, optical fiber, portable CD-ROM, light storage device, magnetic storage
Device or above-mentioned any appropriate combination.Computer readable storage medium, which can be, any includes or storage program has
Shape medium, the program can be commanded execution system, device or device use or in connection.
Computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal,
Wherein carry computer-readable program code.The data-signal of this propagation can take various forms, including but unlimited
In: electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be that computer can
Any computer-readable medium other than storage medium is read, which can send, propagates or transmit and be used for
By the use of instruction execution system, device or device or program in connection.
The program code for including on computer-readable medium can transmit with any suitable medium, including but not limited to:
Wirelessly, electric wire, optical cable, radio frequency (Radio Frequency, RF) etc. or above-mentioned any appropriate combination.
The computer for executing operation of the present invention can be write with one or more programming languages or combinations thereof
Program code, described program design language include object oriented program language-such as Java, Smalltalk, C++,
It further include conventional procedural programming language-such as " C " language or similar programming language.Program code can be with
It fully executes, partly execute on the user computer on the user computer, being executed as an independent software package, portion
Divide and partially executes or executed on a remote computer or server completely on the remote computer on the user computer.?
Be related in the situation of remote computer, remote computer can pass through the network of any kind --- including local area network (LAN) or
Wide area network (WAN) --- it is connected to subscriber computer, or, it may be connected to outer computer (such as utilize Internet service
Provider is connected by internet).
Note that the above is only a better embodiment of the present invention and the applied technical principle.It will be appreciated by those skilled in the art that
The invention is not limited to the specific embodiments described herein, be able to carry out for a person skilled in the art it is various it is apparent variation,
It readjusts and substitutes without departing from protection scope of the present invention.Therefore, although being carried out by above embodiments to the present invention
It is described in further detail, but the present invention is not limited to the above embodiments only, without departing from the inventive concept, also
It may include more other equivalent embodiments, and the scope of the invention is determined by the scope of the appended claims.
Claims (10)
1. a kind of operating method of function service characterized by comprising
Obtain current business interface and the on-unit instruction of function service；
According to predetermined object run model, identify that the operational order is corresponding wait grasp from the current business interface
Make element；
It determines the operating position of the element to be operated, and executes the operational order at the operating position.
2. the method according to claim 1, wherein further include:
Operation model is constructed and trained, object run model is obtained.
3. according to the method described in claim 2, it is characterized in that, the building and training operation model, obtain object run
Model, comprising:
Construct the operation model that model structure is convolutional neural networks；
Based on the sample elements group training operation model in determined sample elements library, object run model is obtained.
4. according to the method described in claim 3, it is characterized in that, determining sample elements library based on operations described below:
Obtain history service interface and the instruction of corresponding historical operation of the function service；
Determine that the historical operation instructs corresponding historical operation element from the history service interface；
Based on history service interface, historical operation instruction and the historical operation element, a sample elements group is formed, and by institute
It states sample elements group and is added to sample elements library.
5. according to the method described in claim 4, it is characterized in that, the sample elements group includes: sample industry as input
Business interface and sample operations instruction, the sample operations element as corresponding output；
Wherein, sample business interface includes: interface name information, routing information, source-information and interface dimensions information；Institute
Stating sample operations element includes: element term information and element size information.
6. a kind of operating device of function service characterized by comprising
Module is obtained, for obtaining current business interface and the on-unit instruction of function service；
Identification module, for identifying the operation from the current business interface according to predetermined object run model
Instruct corresponding element to be operated；
Determining module for determining the operating position of the element to be operated, and executes the operation at the operating position
Instruction.
7. device according to claim 6, which is characterized in that further include:
Training module obtains object run model for constructing and training operation model.
8. device according to claim 7, which is characterized in that the training module is specifically used for:
Construct the operation model that model structure is convolutional neural networks；
Based on the sample elements group training operation model in determined sample elements library, object run model is obtained.
9. a kind of terminal device characterized by comprising
One or more processors；
Storage device, for storing one or more programs；
When one or more of programs are executed by one or more of processors, so that one or more of processors are real
Now such as the operating method of function service as claimed in any one of claims 1 to 5.
10. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is by processor
The operating method such as function service as claimed in any one of claims 1 to 5 is realized when execution.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910063690.8A CN109800049A (en) | 2019-01-23 | 2019-01-23 | A kind of operating method of function service, device, terminal device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910063690.8A CN109800049A (en) | 2019-01-23 | 2019-01-23 | A kind of operating method of function service, device, terminal device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109800049A true CN109800049A (en) | 2019-05-24 |
Family
ID=66560008
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910063690.8A Withdrawn CN109800049A (en) | 2019-01-23 | 2019-01-23 | A kind of operating method of function service, device, terminal device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109800049A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110399191A (en) * | 2019-06-28 | 2019-11-01 | 奇安信科技集团股份有限公司 | A kind of program graphic user interface automatic interaction processing method and processing device |
| CN110413529A (en) * | 2019-07-31 | 2019-11-05 | 中国工商银行股份有限公司 | Applied to the test method of electronic equipment, device, calculate equipment and medium |
| CN110427935A (en) * | 2019-06-28 | 2019-11-08 | 华为技术有限公司 | A kind of web page element knows method for distinguishing and server |
| CN112052005A (en) * | 2019-06-06 | 2020-12-08 | 阿里巴巴集团控股有限公司 | Interface processing method, device, equipment and storage medium |
| CN113986434A (en) * | 2021-10-21 | 2022-01-28 | 深圳市联谛信息无障碍有限责任公司 | Method and device for operating user interface of mobile terminal and electronic equipment thereof |
-
2019
- 2019-01-23 CN CN201910063690.8A patent/CN109800049A/en not_active Withdrawn
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112052005A (en) * | 2019-06-06 | 2020-12-08 | 阿里巴巴集团控股有限公司 | Interface processing method, device, equipment and storage medium |
| CN110399191A (en) * | 2019-06-28 | 2019-11-01 | 奇安信科技集团股份有限公司 | A kind of program graphic user interface automatic interaction processing method and processing device |
| CN110427935A (en) * | 2019-06-28 | 2019-11-08 | 华为技术有限公司 | A kind of web page element knows method for distinguishing and server |
| CN110413529A (en) * | 2019-07-31 | 2019-11-05 | 中国工商银行股份有限公司 | Applied to the test method of electronic equipment, device, calculate equipment and medium |
| CN113986434A (en) * | 2021-10-21 | 2022-01-28 | 深圳市联谛信息无障碍有限责任公司 | Method and device for operating user interface of mobile terminal and electronic equipment thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109800049A (en) | A kind of operating method of function service, device, terminal device and storage medium | |
| US11176423B2 (en) | Edge-based adaptive machine learning for object recognition | |
| US20190384640A1 (en) | Artificial intelligence based virtual automated assistance | |
| WO2020005728A1 (en) | Artificial intelligence assisted content authoring for automated agents | |
| CN110957012B (en) | Method, device, equipment and storage medium for analyzing properties of compound | |
| CN111160569A (en) | Application development method and device based on machine learning model and electronic equipment | |
| US20200401503A1 (en) | System and Method for Testing Artificial Intelligence Systems | |
| Kholiya et al. | Intelligent process automation: The future of digital transformation | |
| US20230029045A1 (en) | Automatic image classification and processing method based on continuous processing structure of multiple artificial intelligence model, and computer program stored in computer-readable recording medium to execute the same | |
| US20200265353A1 (en) | Intelligent workflow advisor for part design, simulation and manufacture | |
| CN104281615A (en) | Complaint handling method and system | |
| US11074043B2 (en) | Automated script review utilizing crowdsourced inputs | |
| Narkar et al. | Model lineupper: Supporting interactive model comparison at multiple levels for automl | |
| Gupta | The aspects of artificial intelligence in software engineering | |
| US11983653B2 (en) | Machine-learning-based prediction of construction project parameters | |
| CN112967248B (en) | Method, apparatus, medium and program product for generating defect image samples | |
| CN110704803A (en) | Target object evaluation value calculation method and device, storage medium and electronic device | |
| Dzhusupova et al. | Pattern recognition method for detecting engineering errors on technical drawings | |
| Noaeen et al. | Let’s hear it from retta: A requirements elicitation tool for traffic management systems | |
| CN115686280A (en) | Deep learning model management system, method, computer device and storage medium | |
| CN113223011A (en) | Small sample image segmentation method based on guide network and full-connection conditional random field | |
| Metz | Building AI that can build AI | |
| CN116541528A (en) | Labeling method and system for recruitment field knowledge graph construction | |
| CN115631374A (en) | Control operation method, control detection model training method, device and equipment | |
| Biyeme et al. | An analytical model for analyzing the value of information flow in the production chain model using regression algorithms and neural networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20190524 |
|
| WW01 | Invention patent application withdrawn after publication |