CN101071380A - Graphi cuser interface intercepting and reproducing method - Google Patents
Graphi cuser interface intercepting and reproducing method Download PDFInfo
- Publication number
- CN101071380A CN101071380A CN 200710123124 CN200710123124A CN101071380A CN 101071380 A CN101071380 A CN 101071380A CN 200710123124 CN200710123124 CN 200710123124 CN 200710123124 A CN200710123124 A CN 200710123124A CN 101071380 A CN101071380 A CN 101071380A
- Authority
- CN
- China
- Prior art keywords
- window
- information
- drawing function
- user interface
- function
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Landscapes
- Stored Programmes (AREA)
- Processing Or Creating Images (AREA)
Abstract
A graphical user interface interception. Its characteristics is that the use of the graphical user interface of the target window attribute information, as well as graphics window of the target window to redraw the necessary mapping function name and the corresponding action graphics data generation that The graphical user interface of the target window graphics composite graphics data; window described in graphic attribute information contained window size and window attribute; action described in graphics data mapping function parameters logo, for the completion of the mapping function of the corresponding graphics operation data. The invention through the window into the mapping function library mapping function interceptors, and through the window to intercept and record the movements generated graphics graphical user interface includes vector and vector or bitmap data of graphics data. In accordance with the invention generating graphical user interface graphics information and data in small, easy storage and transmission, and is faster.
Description
Technical field
The present invention relates to a kind of graphic user interface intercepting and the replay method of (Graphical User Interfaces is called for short GUI).
Background technology
Computer graphics is divided into draw bitmaps and polar plot two big classes.Wherein: bitmap is the set of pixel, represents with the position and the color value of pixel; Polar plot is also referred to as OO image, then uses line segment and curve description image, has also comprised color and positional information simultaneously.Two kinds of diagrammatic representation modes respectively have relative merits, and the advantage of bitmap is that color change is abundant, can change the color display effect in the zone of the arbitrary shape in the bitmap when editor, accordingly, the effect that realizes is complicated more, and the pixel number that needs is many more, and the size of image file is big more.The advantage of polar plot is easier modification of the shape of profile and control, but for independent object, the realization of color change is directly convenient not as bitmap.Polar plot can change into bitmap at an easy rate, but compares difficulty but bitmap is converted into polar plot, often needs through complex calculations and manual adjustment.
At present, in individual and enterprise computer application, Windows (form) etc. has occupied the most market share based on the operating system of graphic user interface, even in the embedded product field that with the mobile phone is representative, a large amount of operating system based on graphic user interface has also appearred, as Symbian (Saipan) etc.We claim that these operating systems based on graphic user interface are Graphic Operating System.The application program of moving in Graphic Operating System has graphic user interface mostly, and the user is undertaken by graphic user interface and program alternately.The graphic user interface of application program can comprise a plurality of windows, and each window has menu usually, and controls such as various command button are for user's input operation instruction.In addition, also comprise client area (viewing area) in the window usually, be used for the information of the current running status of real-time display program, or draw or inputting word information by the user.Therefore in Graphic Operating System, computer graphics typically refers to the client area of application programs window and the drafting of nonclient-area.
Fig. 1 is the application program running environment structural representation in the Graphic Operating System.As shown in Figure 1, the graphic user interface of application program comprises a plurality of windows, and when the window of application program need repaint (for example the literal of window client area changes), application program sends to the window management unit and redraws message, the change of notification window state; The window management unit receives and redraws a series of drawing functions that call after the message in the window drawing function storehouse, and will pass to the relevant drawing function in window drawing function storehouse with the relevant information of drawing with parametric form; Each invoked drawing function is rewritten the color value (representing a pixel with rgb value usually) of the pixel corresponding with the screen pixels position in the screen display memory block according to the mapping operation of the parameter execution of importing into to respective window; Operating system (figure part omitted) is drawn screen picture according to the color value of each pixel in the screen display memory block.
At present, in the time need carrying out intercept operation to the current graphic user interface that is presented at the application program on the screen, usually adopt the mode of screenshotss, promptly the calling interface that provides by operating system obtains the data bitmap (being the rgb value of each pixel of screen) of whole screen, and data bitmap is stored or transmission with document form.In the screenshotss mode graphic user interface is intercepted that there are the following problems:
1. data volume is big, and the sharpness that requires is high more, and the pixel that bitmap comprises is just many more, and the storage space that needs is also just big more; When carrying out the remote application monitoring, transmit corresponding data bitmap with consuming a large amount of network bandwidths;
2. Flame Image Process is more consuming time, is the data bitmap of whole screen because screenshotss obtain, if only need the image of certain or certain several windows also will reduce.Monitor this application in real time, require response rapidly usually, when especially on monitoring side's screen, needing to show a plurality of monitored sides' window simultaneously, the very difficult real-time requirement of satisfying monitoring of screenshotss mode;
3. bitmap meeting distortion after carrying out scale transformation is inconvenient to handle.
Because the shape of graphic user interface is more regular, relatively be fit to represent with polar plot, the bitmap that therefore more existing graphics softwares obtain screenshotss, be converted to polar plot through complicated pattern analysis, again polar plot is preserved or transmitted, can reduce storage space and Network Transmission bandwidth though do like this, increase the weight of computational load, treatment effeciency is lower.
Summary of the invention
Technical matters to be solved by this invention is the deficiency that overcomes the intercept method of graphic user interface in the prior art, and it is little to propose a kind of data volume, is convenient to storage, transmission, and the intercept method and the corresponding figure replay method of the fast graphic user interface of processing speed.
In order to address the above problem, the invention provides a kind of intercept method of graphic user interface, it is characterized in that, use this graphic user interface target window the graph window attribute information and this target window redrawn the title and the corresponding plotting motion data of required drawing function, generate the compound pattern data of the target window figure of this graphic user interface of expression; Described graph window attribute information comprises window size and window attribute; Described plotting motion data are data of the mapping operation that parameter identified, that be used to finish this drawing function correspondence of drawing function.
In addition, this method comprises following steps:
A) inject the drawing function blocker to the drawing function that is used for the window of graphic user interface is drawn, make when drawing function will be called, corresponding drawing function blocker replaces it to be called;
B) obtain the window ID information of target window;
C) using this window ID information to start operates the sectional drawing of target window; Obtain the graph window attribute information of this target window, it is recorded in the corresponding compound pattern data storage area, and to the redraw message of target window transmission at the window Zone Full;
When the drawing function blocker is called, carry out following steps:
D) obtain the window ID information of calling, and judge according to this window ID information whether corresponding window is in the sectional drawing mode of operation: if, execution in step E then); Otherwise, execution in step F);
E) title of the drawing function of correspondence, the plotting motion data of drawing function parameter correspondence are recorded in this compound pattern data storage area that calls window ID information correspondence;
F) call corresponding drawing function.
In addition, the startup described step C) comprises following substep to the process of the sectional drawing operation of target window:
C1) the mode of operation position of described target window correspondence is set to starting state;
C2) length of the described target window correspondence of startup is the timer of T;
Described step C) comprise following steps afterwards:
G) when the timer expiry of described target window correspondence, the mode of operation position of described target window correspondence is set to halted state, finishes the sectional drawing operation to described target window;
Wherein T redraws the required time of operation more than or equal to finishing a window.
In addition, described step G) also comprise following steps afterwards: storage or transmission or handle the compound pattern data of the plotting motion data that comprise graph window attribute information and drawing function title, drawing function parameter correspondence of preserving in the described compound pattern data storage area.
In addition, judge that according to window ID information the process whether corresponding window is in the sectional drawing mode of operation comprises following substep described step D):
D1) if the mode of operation position of this window ID information correspondence is a starting state, then judged result is for being;
D2) if the mode of operation position of this window ID information correspondence is a halted state, then judged result is for denying;
D3) if do not find the operational status information corresponding with this window ID information, then judged result is for denying.
In addition, in described steps A) in, the method for employing Hook Function realizes injecting described drawing function with Hook Function as the drawing function blocker.
In addition, the exploitation/running environment of described graphic user interface is JAVA, described steps A) comprise following substep:
A1) create the expansion graphics environment, and by the start-up parameter of the target program under the described target window is set, the graphics environment of this target program is set to this expansion graphics environment;
A2) create the expansion graphics device that joins with this expansion figure environmental correclation; Create the expansion figure configuration information object that is associated with this expansion graphics device;
A3) in this expansion figure configuration information object, create the expansion blank image, and rewrite the createGraphics method of this expansion blank image, this method is returned comprise expansion drawing object, finished the injection of drawing function blocker as the drawing practice of drawing function blocker;
Wherein, the above-mentioned expansion drawing object subclass that is drawing object.
In addition, when being called, the dispose of described expansion drawing object method stops the sectional drawing of described target window is operated.
In addition, described graph window attribute information also comprises the heading message of window, the icon resource information of window, the menu resource information of window.
The present invention also provides a kind of replay method of graphic user interface, it is characterized in that, this method is carried out following operation to the compound pattern data that comprise graph window attribute information and drawing function title and corresponding plotting motion data:
H) create the window and the corresponding drawing function storehouse of identical or respective type according to described graph window attribute information; Write down the window ID information of this window;
I) according to each the drawing function title that comprises in the described compound pattern data and corresponding plotting motion data and above-mentioned window ID information, generate the required parameter of corresponding drawing function, and use corresponding drawing function in the described drawing function of this parameter call storehouse, this window is carried out mapping operation;
Described graph window attribute information comprises window size and window attribute; Described plotting motion data are the data that are used to finish the performed mapping operation of corresponding drawing function.
The present invention passes through to inject the drawing function blocker in window drawing function storehouse, and generates the polar plot of graphic user interface or comprise polar plot and the compound pattern data of data bitmap by the plotting motion of interception and record window.The graphical user interface graphics information and data that generates according to method of the present invention is little, is convenient to storage and transmission, and processing speed is very fast.
Description of drawings
Fig. 1 is the application program running environment structural representation in the Graphic Operating System;
Fig. 2 is the intercept method process flow diagram of first embodiment of the invention graphic user interface;
Fig. 3 is the function calling relationship synoptic diagram that injects drawing function blocker front and back;
Fig. 4 is the replay method process flow diagram of first embodiment of the invention graphic user interface;
Fig. 5 is the structural representation of JAVA exploitation/running environment;
Fig. 6 is that second embodiment of the invention adopts the drawing object substitute mode to inject the method flow diagram of drawing function blocker.
Embodiment
Because the figure of graphic user interface is combined by icon and non-icon control usually.Wherein, the common out-of-shape of icon, color change is big, is fit to represent with bitmap; The regular shape of non-icon control, color change is little, is fit to represent with polar plot, the graphical information that therefore can adopt the compound pattern data that comprise bitmap and vector map data to preserve graphic user interface; And above-mentioned compound pattern data can generate by the title of recording drawing function and the mode of the plotting motion data of correspondence when the drawing function that graphic user interface is carried out mapping operation is called.
Below in conjunction with drawings and Examples, the present invention is described in detail.
Fig. 2 is the intercept method process flow diagram of first embodiment of the invention graphic user interface.As shown in Figure 2, the intercepting of graphic user interface (being the window sectional drawing) method comprises following steps:
101: all drawing functions to window drawing function storehouse inject drawing function blockers, and when making the drawing function in drawing function storehouse to be called, corresponding drawing function blocker replaces it to be called.
As shown in Figure 1, when needs carry out window when redrawing, application program sends to the window management unit and redraws message, the window management unit receives the required a series of drawing functions of this window of drafting that call after this message in the window drawing function storehouse, therefore the drawing function blocker can be injected each drawing function in window drawing function storehouse.The drawing function blocker can obtain information such as the title, parameter of invoked drawing function.
Fig. 3 is the function calling relationship synoptic diagram that injects drawing function blocker front and back.Before solid line among the figure is illustrated in and injects the drawing function blocker, the function calling relationship the when call function in the window management unit calls the drawing function in the window drawing function storehouse; Dotted line represents to inject the function calling relationship when call function calls drawing function behind the blocker.
Use Hook Function as the drawing function blocker in the present embodiment.For the application program that operates in operating systems such as Windows, inject the method that the drawing function blocker can adopt Hook Function.Hook Function is a very important system interface in the Windows system, is used to tackle and handle the function call or the message processing procedure of application program.For example, the name of drawing function is called DrawLine (), if need interception to this function calls, can define a Hook Function MyHook (); After this Hook Function is installed, when DrawLine () function is called, program will go to MyHook () function automatically and carry out.Therefore, when adopting this method, Hook Function MyHook () is exactly the drawing function blocker.Concrete using method about Hook Function sees also the related development document.
102: the window ID information of obtaining target window;
Target window is exactly the window that need carry out the sectional drawing operation to it, and the application program under this window is called destination application.
Because the graphic user interface of application program may comprise a plurality of windows, can carry out the sectional drawing operation to one or more window, therefore need obtain the window ID information of the target window that carries out the sectional drawing operation.Window ID information is known as window handle in Windows exploitation/operation platform, and in JAVA exploitation/operation platform, be known as window/and the top layer container quotes.
Window ID information can be by calling API (ApplicationProgramming Interface, application development interface) the function acquisition that exploitation/operation platform provides; The AP I difference that different exploitation/operation platforms provides.For example, in windows platform, can call FindWindow functions such as () and obtain the target window handle.
103: create the compound pattern data storage area of target window, write down the identification information of this memory block;
The identification information of above-mentioned memory block is used to obtain current memory location, compound pattern data storage area.Above-mentioned memory block can be: a continuous storage space in the internal memory, its identification information are the start address+current offset address of this storage space; A file, its identification information are this document title or file handle; A chain table object, its identification information are the linked list head address or the chained list handle of this chain table object.
104: start sectional drawing operation to target window;
Start the sectional drawing operation and be exactly make the drawing function blocker when being called (by drawing function be called cause) can know that the current goal window is in the sectional drawing mode of operation.
Start the sectional drawing operation and comprise following substep:
104a: the mode of operation position of target window correspondence is set, for example, creates Window state table as shown in table 1, the corresponding operational state value of the window ID information (window handle) of target window is set to 1 (starting state);
104b: start the sectional drawing operation timer of target window correspondence, promptly start the timer of the window ID information correspondence of target window in the table 1;
The window ID information of correspondence timer can be passed to as parameter, the window that timer is affiliated can be determined when making its overtime being triggered.
| Window ID information | Mode of operation | Timer ID | The memory block identification information |
| Window handle 1 | 1 | Timer 1 | Memory address 1 |
| Window handle 2 | 0 | Timer 2 | Memory address 2 |
Table 1
Window handle 1 in the table 1, window handle 2 are the window ID information of 2 windows of application program; Operational state value is that 1 expression is carried out the sectional drawing operation to respective window, and value is that the sectional drawing operation is not carried out in 0 (halted state) expression; Memory address 1, memory address 2 are the current memory location of compound pattern data storage area.If preserve the compound pattern data with file mode, the memory block identification information can be filename or file handle.
105: use window ID information to obtain the graph window attribute information of target window, generate the graph window attribute section of compound pattern data, be recorded in the corresponding compound pattern data storage area, and to the message that redraws of target window send window Zone Full;
Target window receive above-mentioned redraw message after, the message processing function of window can be sent to this message the window management unit, simultaneously window ID information is passed to the window management unit; The corresponding drawing function that the window management unit uses window ID information to call in the window drawing function storehouse begins the Zone Full of target window is redrawn, thereby triggers corresponding drawing function blocker operation.
Above-mentioned compound pattern data comprise two parts: graph window attribute and graph window main body.
The graph window attribute section is with the form of type/length/content record window graphic attribute information successively.The graph window main part comprises a plurality of graph window body items, and each graph window body items is used for the title and the corresponding plotting motion data of recording drawing function.
Above-mentioned graph window attribute information comprises: the menu resource information of the type of the size of window, the heading message of window, window, the icon resource information of window, window etc.For the application program on the Windows exploitation/operation platform, the type of window can be single document, many documents, dialog box etc.; For the application program on the JAVA exploitation/operation platform, window type can be: JFrame, JWindow, Jdialog etc.Wherein, the icon resource information of window is the data bitmap of window icons.
The graph window attribute information can be by calling the api function acquisition that exploitation/operation platform provides.For example, can use GetWindowText () function to obtain the heading message of window at windows platform; Use GetWindowRect () to obtain the size of window.
When generating the graph window attribute section of compound pattern data, except that the type of the size of window, window, other can omit.
When generating the graph window attribute section of compound pattern data according to the graph window attribute information, usually need carry out data-switching, for example according to the bitmap object pointer of the icon of the target window that obtains/quote and obtain corresponding data bitmap, and the data that this data bitmap is compressed into JPEG (Joint PhotographicExperts Group, JPEG (joint photographic experts group)) form are stored.
May need to revise corresponding memory block identification information after the compound pattern data are kept at the compound pattern data storage area, for example, if come the storage of complex graph data with a continuous storage space in the internal memory, then after data write, need to revise current offset address, make follow-up data write correct position.If with file mode storage of complex graph data, then need not to carry out the adjustment of memory block identification information.
When being called, carries out the drawing function blocker following steps 106～108:
106: invoked drawing function blocker obtains the window ID information (abbreviation calls window ID information) that causes the window that corresponding drawing function calls, and inquires about the current sectional drawing state that whether is in of this window according to this window ID information from table 1:
If then execution in step 107;
Otherwise, execution in step 108.
Inquiring about the current sectional drawing state that whether is in is exactly that the drawing function blocker is inquired about corresponding operational state value according to window ID information from table 1; If being 1 expression, the respective operations state value is in the sectional drawing state; If value is 0 or does not exist corresponding window ID information to represent that then this window is not in the sectional drawing state.
Can to be the window management unit pass to drawing function in the mode of parameter to window ID information, so the drawing function blocker can directly obtain by reading this parameter; Perhaps can extract the pointer of call function by the drawing function blocker, obtain the corresponding described class/object information of call function (class title for example, parent under such etc.), by such/object information obtains to call window ID information such as the handle of window/quote.
107: the drawing function blocker obtains corresponding memory block identification information according to window ID information, and the title of invoked drawing function and the pairing plotting motion data of parameter thereof the graph window body items as the compound pattern data is recorded in the indicated compound pattern data storage area of memory block identification information;
Above-mentioned plotting motion data be drawing function parameter identified, irrelevant with target program context (Context), be actually used in the data of finishing the corresponding mapping operation of this drawing function.The employed parameter of drawing function is to point to the pointer of a certain region of memory or the handle of a certain object/quote sometimes; And above-mentioned pointer, handle, information such as quote have promptly lost meaning after having broken away from current program run context, can't be reused.Therefore, need the record plotting motion data that are used for actual plotting motion that this parameter identified.For example, the bitmap function parameters comprises the pointer that points to a bitmap object, need be converted into this pointer data bitmap that is actually used in draw bitmaps pointed (plotting motion data).Certainly, for some drawing function, its part or all of parameter is identical with the plotting motion data.
Drawing function can be divided into two big classes: bitmap function, phasor function.The rgb value that the pairing plotting motion data of bitmap function parameters are each pixel in the corresponding drawing area; The pairing plotting motion data of the parameter of phasor function are the required characteristic parameter of corresponding plotting motion.Generally, the pairing plotting motion data of bitmap function parameters are more much bigger than the data volume of the pairing plotting motion data of the parameter of phasor function.With a rectilinear figure is example, if represent in the bitmap mode, and the pairing plotting motion data of the bitmap function parameters rgb value (or gray-scale value) that is all pixels of this straight line then; If represent that with polar plot the pairing plotting motion data of the parameter of phasor function then are the positional informations of two end points of this straight line.
108: the drawing function blocker is according to the title and the corresponding drawing function of parameter call thereof of the drawing function of being tackled.
109: if above-mentioned timer expiry, then the operational state value corresponding with target window is set to 0 in the table 1, finishes the sectional drawing operation to this target window;
The length of timer should be more than or equal to once redrawing target window the required time of operation, and its value is by the complexity decision of system performance and figure.
110: the compound pattern data of the target window correspondence that storage or transmission are generated.
Fig. 4 is reproduction (being that the window reappears) method flow diagram of first embodiment of the invention graphic user interface.The reproduction of graphic user interface is exactly the information that comprises according in the above-mentioned compound pattern data, repaints graph of a correspondence user interface (window).As shown in Figure 4, the reproduction of graphic user interface comprises following steps:
201: read the graph window attribute section in the compound pattern data, call corresponding api function, create the window and the corresponding drawing function storehouse of identical or respective type according to the graph window attribute information that wherein comprises; The window ID information (for example, window handle) of the record window of creating;
202: read each the graph window body items in the compound pattern data, according to the function name that wherein comprises and plotting motion data and above-mentioned window ID information, generate the required parameter of corresponding drawing function successively, and use this drawing function of this parameter call, this window is carried out mapping operation;
Before drawing function is called, the plotting motion data-switching that writes down in the compound pattern data need be become the drawing function parameter in the corresponding drawing function storehouse.For example, the plotting motion data of the bitmap function that writes down in the compound pattern data comprise the view data of JPEG compressed format, need convert thereof into the data bitmap of BMP form, and generate the bitmap object comprise this data bitmap, the pointer of this bitmap object/quote as parameter is passed to corresponding bitmap function.
In addition, the reproduction of graphic user interface need be in identical exploitation/running environment with intercepting usually.That is to say, to the compound pattern data of the graphic user interface of the application program in a certain exploitation/running environment, moved, need be to its reproduction in same exploitation/running environment.For example, the graphic user interface of window application is intercepted the compound pattern data that generated,, need to rely on Windows exploitation/running environment equally if will reappear to it.Produce this limitation and mainly contain following reason:
One, the graphic user interface in each exploitation/running environment has different attributes, if put into another environment, then is difficult to use the graph window attribute information that comprises in the compound pattern data to create the identical window of outward appearance;
Two, the drawing function title that comprises of the window drawing function storehouse in different exploitation/running environment may be different with desired parameters.
Certainly solve or partly address the above problem by the compound pattern data are changed, for example, when in different exploitation/running environment, reappearing the window of compound pattern data correspondence, though can not create the identical window of type/outward appearance, can create the similar window of type/outward appearance; In addition, can also change to adapt to new exploitation/running environment drawing function title and plotting motion data.
To be further described the present invention by second embodiment below.This embodiment intercepts the graphic user interface of the application program using SWING (waving) developing instrument in reply JAVA (pawl low-lying area) platform and develop.
Fig. 5 is the structural representation of JAVA exploitation/running environment.As shown in Figure 5, the graphic user interface of the application program of exploitation/operation (being called for short the JAVA program) all is to be plotted to terminal device (being exactly indicator screen usually) by the Graphics2D function library in the JAVA platform.Wherein, SWING is the graphical interfaces development kit that generally adopts in the JAVA platform, has wherein comprised the window management unit that graphic user interface is managed.Comprise Graphics2D window drawing function storehouse in the JAVA2D shape library.
In this embodiment, to graphic user interface intercept and the step and the last embodiment that reappear basic identical, just adopted the mode of replacing the Graphics2D drawing object to realize injecting the drawing function blocker.Because the JAVA procedure development is based on Object-oriented Technique, so the window management unit of the window in the JAVA program (being called the top layer container), window correspondence and Graphics2D window drawing function storehouse all are that form with object (Object)/class (Class) exists.And all drawing functions in the Graphics2D window drawing function storehouse all exist with the form of the method for Graphics2D drawing object.
Under default situations, system can draw operation by this drawing object to the top layer container for each top layer container configuration Graphics2D drawing object (being called system's drawing object) of JAVA application program.That is to say, when creating a new top layer container for application program and making it be presented on the screen, when perhaps certain in the top layer container or whole zone need repaint, will call the correlation method (being drawing function) of the Graphics2D drawing object corresponding with this top layer container.As long as therefore system's drawing object of target top layer container can be replaced to self-defining expansion drawing object, all drawing functions that can tackle this top layer container call.
Fig. 6 is that second embodiment of the invention adopts the drawing object substitute mode to inject the method flow diagram of drawing function blocker.As shown in Figure 6, inject the drawing function blocker and comprise following steps:
301: create expansion graphics environment (note is made ExtGe) by the mode of inheriting graphics environment class (GraphicsEnvironment);
Graphics environment among the JAVA is used to define the set of the visible coloring apparatus of JAVA program (Rendering Devices) on the different operating platform.Coloring apparatus comprises display, printer and frame buffer.
302: JAVA program start parameter is set, and the graphics environment of JAVA application program is set to above-mentioned expansion graphics environment ExtGe;
For example, write to order in the startup configuration file of destination application, ExtGe is set to the graphics environment of this application program:
set_JAVA_OPTIONS -Djava.awt.graphicsenv ExtGe
303: in the expansion equipment environment, create expansion graphics device (GraphicsDevice); With the coloring apparatus of display as current appointment, this expansion graphics device is related with these expansion equipment environmental facies.
304: create expansion figure configuration information object (GraphicsConfiguration) in above-mentioned expansion graphics device, this expansion figure configuration information object is associated with this expansion graphics device.
305: in expansion figure configuration information object, create expansion blank image ExtImage.
306: rewrite the createGraphics method of above-mentioned expansion blank image ExtImage, the drawing object that this method is returned is expansion drawing object (note is made ExtGraphics2D); Wherein expand the subclass that drawing object ExtGraphics2D is drawing object Graphics2D; Each drawing practice (drawing function) of expansion drawing object is exactly the drawing function blocker that is injected into.
Through above-mentioned steps 301 to 306, when the top layer container of destination application need carry out plotting motion, the corresponding drawing practice/function in will invoke extensions drawing object (ExtGraphics2D).Therefore only need each drawing function (being the method for drawing object) in the expansion drawing object is rewritten, make it tackle corresponding drawing function and call as the drawing function blocker, and the operation in the execution above-mentioned steps 107; And in step 108,, carry out corresponding mapping operation by calling the drawing function of the same name of its parent (Graphics2D).
In addition, when not using in a period of time owing to above-mentioned expansion drawing object (for example, not redrawing action in its corresponding top layer container a period of time), this expansion drawing object will be deleted by system; When needs redraw the top layer container, this expansion drawing object will be created again again by system.Therefore, can expand in the dispose method of drawing object that the operational state value corresponding with target window is set to 0 in the table 1, finish sectional drawing operation this target window at this.
Below will be to drawing function and the parameter thereof in the Graphics2D drawing function storehouse, and the plotting motion data of parameter correspondence are carried out classified description.Drawing function in the Graphics2D drawing function storehouse comprises following a few class:
1. bitmap function: be used for draw bitmaps, as drawImage () function; Its parameter comprises the coordinate figure in quoting of bitmap object and bitmap zone of living in; Therefore the plotting motion data comprise that this quotes the coordinate figure in data bitmap pointed and bitmap zone of living in;
2. phasor function comprises that location expression function, attribute are provided with function, text is drawn function, shape drafting function, shape stuffing function etc.Wherein:
1) location expression function is used to specify control position or control effective coverage, as: translate () function; Its parameter is a round values, the denotation coordination value; The plotting motion data are identical with parameter;
2) attribute is provided with function, is used to be provided with graphic attribute, as: setColor () function, its parameter are quoting of color object; The plotting motion data are this color object reference color value pointed (rgb value);
3) text is drawn function, is used for rendering text, and as drawString () function, its parameter is quoting of string; The plotting motion data are quoted string value pointed for this;
4) shape is drawn function, is used to draw sealing or open figures such as straight line, broken line, rectangle, circle, and as drawRect (), its parameter is rectangle upper left corner coordinate and rectangle width, rectangle height value, and the plotting motion data are identical with parameter;
5) shape stuffing function is used to use particular color to fill closed figure, and its parameter and plotting motion data and shape are drawn functional similarity.
Above-mentioned drawing function title is kept in the compound pattern data storage area successively with the corresponding plotting motion data graph window body items as the compound pattern data, constitutes the compound pattern data that comprise data bitmap and vector map data with the graph window attribute section.
Based on the principle of the invention, on the basis of the foregoing description, can also do various conversion.For example, for not using the window management unit, and the application program that the drawing function in the window drawing function storehouse is drawn window is directly called in employing, can be when it calls drawing function, the title and the corresponding plotting motion data of drawing function are write the memory block, generate the compound pattern data together with the graph window attribute information of this window, reach the purpose that the graphic user interface to this application program intercepts.
Claims (10)
1, a kind of intercept method of graphic user interface, it is characterized in that, use this graphic user interface target window the graph window attribute information and this target window redrawn the title and the corresponding plotting motion data of required drawing function, generate the compound pattern data of the target window figure of this graphic user interface of expression; Described graph window attribute information comprises window size and window attribute; Described plotting motion data are data of the mapping operation that parameter identified, that be used to finish this drawing function correspondence of drawing function.
2, the intercept method of graphic user interface as claimed in claim 1 is characterized in that, this method comprises following steps:
A) inject the drawing function blocker to the drawing function that is used for the window of graphic user interface is drawn, make when drawing function will be called, corresponding drawing function blocker replaces it to be called;
B) obtain the window ID information of target window;
C) using this window ID information to start operates the sectional drawing of target window; Obtain the graph window attribute information of this target window, it is recorded in the corresponding compound pattern data storage area, and to the redraw message of target window transmission at the window Zone Full;
When the drawing function blocker is called, carry out following steps:
D) obtain the window ID information of calling, and judge according to this window ID information whether corresponding window is in the sectional drawing mode of operation: if, execution in step E then); Otherwise, execution in step F);
E) title of the drawing function of correspondence, the plotting motion data of drawing function parameter correspondence are recorded in this compound pattern data storage area that calls window ID information correspondence;
F) call corresponding drawing function.
3, the intercept method of graphic user interface as claimed in claim 2 is characterized in that,
Described step C) startup in comprises following substep to the process of the sectional drawing operation of target window:
C1) the mode of operation position of described target window correspondence is set to starting state;
C2) length of the described target window correspondence of startup is the timer of T;
Described step C) comprise following steps afterwards:
G) when the timer expiry of described target window correspondence, the mode of operation position of described target window correspondence is set to halted state, finishes the sectional drawing operation to described target window;
Wherein T redraws the required time of operation more than or equal to finishing a window.
4, the intercept method of graphic user interface as claimed in claim 3, it is characterized in that described step G) also comprise following steps afterwards: storage or transmission or handle preserved in the described compound pattern data storage area comprises the compound pattern data of the plotting motion data of graph window attribute information and drawing function title, drawing function parameter correspondence.
5, the intercept method of graphic user interface as claimed in claim 3 is characterized in that, described step D) in judge that according to window ID information the process whether corresponding window is in the sectional drawing mode of operation comprises following substep:
D1) if the mode of operation position of this window ID information correspondence is a starting state, then judged result is for being;
D2) if the mode of operation position of this window ID information correspondence is a halted state, then judged result is for denying;
D3) if do not find the operational status information corresponding with this window ID information, then judged result is for denying.
6, the intercept method of graphic user interface as claimed in claim 2 is characterized in that, in described steps A) in, the method for employing Hook Function realizes injecting described drawing function with Hook Function as the drawing function blocker.
7, the intercept method of graphic user interface as claimed in claim 2 is characterized in that, the exploitation/running environment of described graphic user interface is JAVA, described steps A) comprise following substep:
A1) create the expansion graphics environment, and by the start-up parameter of the target program under the described target window is set, the graphics environment of this target program is set to this expansion graphics environment;
A2) create the expansion graphics device that joins with this expansion figure environmental correclation; Create the expansion figure configuration information object that is associated with this expansion graphics device;
A3) in this expansion figure configuration information object, create the expansion blank image, and rewrite the createGraphics method of this expansion blank image, this method is returned comprise expansion drawing object, finished the injection of drawing function blocker as the drawing practice of drawing function blocker;
Wherein, the above-mentioned expansion drawing object subclass that is drawing object.
8, the intercept method of graphic user interface as claimed in claim 7 is characterized in that, stops the sectional drawing operation to described target window when the dispose of described expansion drawing object method is called.
9, the intercept method of graphic user interface as claimed in claim 1 is characterized in that, described graph window attribute information also comprises the heading message of window, the icon resource information of window, the menu resource information of window.
10, a kind of replay method of graphic user interface is characterized in that, this method is carried out following operation to the compound pattern data that comprise graph window attribute information and drawing function title and corresponding plotting motion data:
H) create the window and the corresponding drawing function storehouse of identical or respective type according to described graph window attribute information; Write down the window ID information of this window;
I) according to each the drawing function title that comprises in the described compound pattern data and corresponding plotting motion data and above-mentioned window ID information, generate the required parameter of corresponding drawing function, and use corresponding drawing function in the described drawing function of this parameter call storehouse, this window is carried out mapping operation;
Described graph window attribute information comprises window size and window attribute; Described plotting motion data are the data that are used to finish the performed mapping operation of corresponding drawing function.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2007101231249A CN100561430C (en) | 2007-06-27 | 2007-06-27 | A kind of intercepting of graphic user interface and replay method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CNB2007101231249A CN100561430C (en) | 2007-06-27 | 2007-06-27 | A kind of intercepting of graphic user interface and replay method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101071380A true CN101071380A (en) | 2007-11-14 |
| CN100561430C CN100561430C (en) | 2009-11-18 |
Family
ID=38898616
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2007101231249A Expired - Fee Related CN100561430C (en) | 2007-06-27 | 2007-06-27 | A kind of intercepting of graphic user interface and replay method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN100561430C (en) |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101853157A (en) * | 2010-05-14 | 2010-10-06 | 中国科学院软件研究所 | Automatic identification method for application software GUI object |
| CN102054280A (en) * | 2010-11-29 | 2011-05-11 | 广东威创视讯科技股份有限公司 | Method and device for rapidly generating vector diagram |
| CN102073973A (en) * | 2009-11-25 | 2011-05-25 | 新奥特（北京）视频技术有限公司 | Method for obtaining and editing user-defined graph and subtitle graph fabricating system |
| CN102396180A (en) * | 2009-02-13 | 2012-03-28 | 三星电子株式会社 | Method and apparatus for providing remote user interface service |
| CN102799432A (en) * | 2012-06-30 | 2012-11-28 | 邱东 | Game video recording and replaying method based on recorded drawing instruction |
| CN102902541A (en) * | 2012-09-27 | 2013-01-30 | 北京国双科技有限公司 | Method and device of capturing system message on IOS (Internetwork Operating System) platform |
| CN103092596A (en) * | 2011-11-08 | 2013-05-08 | 中国移动通信集团陕西有限公司 | Method and device for achieving interface inversion automatically of low-level interface |
| CN103593184A (en) * | 2013-10-31 | 2014-02-19 | 福州瑞芯微电子有限公司 | Image displaying system and image displaying method |
| CN103729174A (en) * | 2012-10-16 | 2014-04-16 | 腾讯科技（深圳）有限公司 | Screen capturing method and device |
| CN103942044A (en) * | 2014-03-18 | 2014-07-23 | 北京领通科技有限公司 | Remote assistance method and system in application of mobile terminal |
| CN104765595A (en) * | 2014-01-08 | 2015-07-08 | 联发科技（新加坡）私人有限公司 | Method and device for displaying graphical user interface |
| CN104823159A (en) * | 2012-11-27 | 2015-08-05 | 欧姆龙株式会社 | Method and device for changing objects in human-machine interface device |
| WO2016082530A1 (en) * | 2014-11-27 | 2016-06-02 | 广州市动景计算机科技有限公司 | Screenshooting method and apparatus for webpage |
| CN106406862A (en) * | 2016-08-31 | 2017-02-15 | 福建省天奕网络科技有限公司 | Screen acquisition method and system |
| CN107221019A (en) * | 2017-03-07 | 2017-09-29 | 武汉唯理科技有限公司 | Chart conversion method and device |
| CN108540852A (en) * | 2015-07-23 | 2018-09-14 | 海信集团有限公司 | A kind of screenshotss method |
| CN109976749A (en) * | 2019-03-28 | 2019-07-05 | 深圳市创联时代科技有限公司 | A kind of inheritance method of drawing course |
| CN113835600A (en) * | 2021-09-24 | 2021-12-24 | 统信软件技术有限公司 | Application window rotating method, computing device and readable storage medium |
-
2007
- 2007-06-27 CN CNB2007101231249A patent/CN100561430C/en not_active Expired - Fee Related
Cited By (33)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102396180B (en) * | 2009-02-13 | 2015-09-09 | 三星电子株式会社 | For providing the method and apparatus of remote user interface service |
| US9160556B2 (en) | 2009-02-13 | 2015-10-13 | Samsung Electronics Co., Ltd. | Method and apparatus for providing remote user interface service |
| CN102396180A (en) * | 2009-02-13 | 2012-03-28 | 三星电子株式会社 | Method and apparatus for providing remote user interface service |
| CN102073973A (en) * | 2009-11-25 | 2011-05-25 | 新奥特（北京）视频技术有限公司 | Method for obtaining and editing user-defined graph and subtitle graph fabricating system |
| CN102073973B (en) * | 2009-11-25 | 2015-06-17 | 新奥特（北京）视频技术有限公司 | Method for obtaining and editing user-defined graph and subtitle graph fabricating system |
| CN101853157B (en) * | 2010-05-14 | 2013-04-10 | 中国科学院软件研究所 | Automatic identification method for application software GUI object |
| CN101853157A (en) * | 2010-05-14 | 2010-10-06 | 中国科学院软件研究所 | Automatic identification method for application software GUI object |
| CN102054280A (en) * | 2010-11-29 | 2011-05-11 | 广东威创视讯科技股份有限公司 | Method and device for rapidly generating vector diagram |
| CN103092596A (en) * | 2011-11-08 | 2013-05-08 | 中国移动通信集团陕西有限公司 | Method and device for achieving interface inversion automatically of low-level interface |
| CN103092596B (en) * | 2011-11-08 | 2016-03-30 | 中国移动通信集团陕西有限公司 | Rudimentary interface development realizes method and the device of interface inversion automatically |
| CN102799432B (en) * | 2012-06-30 | 2015-01-14 | 邱东 | Game video recording and replaying method based on recorded drawing instruction |
| CN102799432A (en) * | 2012-06-30 | 2012-11-28 | 邱东 | Game video recording and replaying method based on recorded drawing instruction |
| CN102902541A (en) * | 2012-09-27 | 2013-01-30 | 北京国双科技有限公司 | Method and device of capturing system message on IOS (Internetwork Operating System) platform |
| CN102902541B (en) * | 2012-09-27 | 2016-02-24 | 北京国双科技有限公司 | The method and apparatus of capturing system message on ios platform |
| CN103729174B (en) * | 2012-10-16 | 2017-10-27 | 腾讯科技（深圳）有限公司 | Screenshotss method and device |
| CN103729174A (en) * | 2012-10-16 | 2014-04-16 | 腾讯科技（深圳）有限公司 | Screen capturing method and device |
| CN104823159A (en) * | 2012-11-27 | 2015-08-05 | 欧姆龙株式会社 | Method and device for changing objects in human-machine interface device |
| CN103593184B (en) * | 2013-10-31 | 2016-08-17 | 福州瑞芯微电子股份有限公司 | Image display system and method for displaying image |
| CN103593184A (en) * | 2013-10-31 | 2014-02-19 | 福州瑞芯微电子有限公司 | Image displaying system and image displaying method |
| US9899004B2 (en) | 2014-01-08 | 2018-02-20 | Mediatek Singapore Pte. Ltd. | Method and device for generating graphical user interface (GUI) for displaying |
| CN104765595A (en) * | 2014-01-08 | 2015-07-08 | 联发科技（新加坡）私人有限公司 | Method and device for displaying graphical user interface |
| CN104765595B (en) * | 2014-01-08 | 2018-05-22 | 联发科技（新加坡）私人有限公司 | A kind of method and device for showing graphic user interface |
| CN103942044A (en) * | 2014-03-18 | 2014-07-23 | 北京领通科技有限公司 | Remote assistance method and system in application of mobile terminal |
| WO2016082530A1 (en) * | 2014-11-27 | 2016-06-02 | 广州市动景计算机科技有限公司 | Screenshooting method and apparatus for webpage |
| CN105701127A (en) * | 2014-11-27 | 2016-06-22 | 广州市动景计算机科技有限公司 | Webpage screen capture processing method and apparatus |
| CN108540852A (en) * | 2015-07-23 | 2018-09-14 | 海信集团有限公司 | A kind of screenshotss method |
| CN106406862A (en) * | 2016-08-31 | 2017-02-15 | 福建省天奕网络科技有限公司 | Screen acquisition method and system |
| CN106406862B (en) * | 2016-08-31 | 2019-10-25 | 福建省天奕网络科技有限公司 | A kind of screen acquiring method and system |
| CN107221019A (en) * | 2017-03-07 | 2017-09-29 | 武汉唯理科技有限公司 | Chart conversion method and device |
| CN109976749A (en) * | 2019-03-28 | 2019-07-05 | 深圳市创联时代科技有限公司 | A kind of inheritance method of drawing course |
| CN109976749B (en) * | 2019-03-28 | 2024-01-16 | 深圳市创联时代科技有限公司 | Inheritance method for drawing process |
| CN113835600A (en) * | 2021-09-24 | 2021-12-24 | 统信软件技术有限公司 | Application window rotating method, computing device and readable storage medium |
| CN113835600B (en) * | 2021-09-24 | 2024-05-14 | 统信软件技术有限公司 | Application window rotation method, computing device and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN100561430C (en) | 2009-11-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100561430C (en) | A kind of intercepting of graphic user interface and replay method | |
| RU2321892C2 (en) | Markup language and object model for vector graphics | |
| RU2324229C2 (en) | Visual and three-dimensional graphic interfaces | |
| JP5374873B2 (en) | Information processing apparatus, information processing system, computer program, and information processing method | |
| US8627217B2 (en) | Window display system, window display method, program development support device, and server device | |
| US10496658B2 (en) | Method and system of visually depicting hierarchical data through selective colorization | |
| CN110378063B (en) | Equipment deployment method and device based on intelligent building space and electronic equipment | |
| JP2008170939A (en) | Electronic device, information processing device, image revision method and information processing method | |
| CN106569783A (en) | Task script executing method and apparatus | |
| CN112231608B (en) | Method for realizing embedded Web configuration graphics primitive | |
| CN108304239B (en) | Rapid interface interaction method and device for remote operation | |
| CN100412902C (en) | Method for extracting graph image data information | |
| CN113368492A (en) | Rendering method and device | |
| US20190080017A1 (en) | Method, system, and device that invokes a web engine | |
| CN104866318A (en) | Method and device for displaying label pages in multiple windows | |
| CN109298806B (en) | Remote quick interface interaction method and device based on object recognition | |
| CN107818588A (en) | The apparatus and method that android system calls Qt drawing based on JNI multithreadings | |
| CN111667199A (en) | Workflow construction method and device, computer equipment and storage medium | |
| KR101888472B1 (en) | Do It Yourself system of Internet of Thinks based on Business Process Model and Its Implementation Method | |
| CN103150198A (en) | Display method of configuration software | |
| CN105446676B (en) | Carry out the method and device that large-size screen monitors are shown | |
| KR101748548B1 (en) | IoT network system | |
| CN114862999A (en) | Dotting rendering method, dotting rendering device, dotting rendering equipment and storage medium | |
| CN110955739B (en) | Plotting processing method, shared image plotting method, and plot reproducing method | |
| CN107832460A (en) | The graphical page control method of real-time display Satellite TT data based on SVG |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20091118 Termination date: 20170627 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |