CN106875408B - Screenshot method and device and terminal equipment - Google Patents
Screenshot method and device and terminal equipment Download PDFInfo
- Publication number
- CN106875408B CN106875408B CN201710109123.2A CN201710109123A CN106875408B CN 106875408 B CN106875408 B CN 106875408B CN 201710109123 A CN201710109123 A CN 201710109123A CN 106875408 B CN106875408 B CN 106875408B
- Authority
- CN
- China
- Prior art keywords
- image
- screenshot
- connected region
- edge
- gray level
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
- Processing Or Creating Images (AREA)
Abstract
The application relates to a method, a device and a terminal device for screenshot, wherein the method comprises the following steps: acquiring a screen image of a terminal in real time; processing the screen image according to the texture information of the screen image, and automatically extracting at least one connected region; and finding a connected region where the operation point is located from the at least one connected region, and performing screenshot according to the connected region. According to the technical scheme, the accurate screenshot of the operation position can be effectively obtained, a large amount of invalid irrelevant background information is avoided, and a large amount of screenshot errors caused by screenshot in a fixed area are remarkably reduced.
Description
Technical Field
The application relates to the technical field of automatic testing, in particular to a screenshot method, a screenshot device and terminal equipment.
Background
In the technical field of software automation test, in the related art, the process of automatically recording a script basically comprises the following steps: and the user operates the equipment, automatically identifies the user operation, generates a script corresponding to the operation in real time, automatically intercepts the image of the operation position and stores the image into the script, and takes the screenshot file path as a script parameter. The principle of the regression script is to perform image matching on the screenshot in the script and the current picture of the device, so as to locate the operation position and execute corresponding operation at the position.
Because the test script may need to be run under the mobile phones with different resolutions respectively, the pictures corresponding to the terminals with different resolutions may correspond to a larger background difference, and if the screenshot contains more backgrounds, the similarity of the matching result is greatly affected when the screenshot is matched with the mobile phone picture.
For two terminal devices with different resolutions, the resolutions and the different scenes where the terminal devices are located result in a larger difference between the pictures, such as the scenes shown in fig. 1 and fig. 2. If a fixed size screenshot image is used, the background of the screenshot for a small button, operation icon, etc. is very different, for example, the screenshot extracted from fig. 1 is fig. 3, and the screenshot extracted from fig. 2 is fig. 4. When the similarity of the screenshots is judged, more background differences can result in lower similarity.
It can be seen that if the automatic screenshot mode during recording the script is set to be the screenshot with the fixed area size, the screenshot may contain more irrelevant areas, and for small buttons and operation icons of text types, the screenshot contains too many irrelevant areas, and the screenshot accuracy cannot be guaranteed. If the automatic screenshot is not appropriate or the screenshot effect is not ideal, the user is required to perform manual screenshot, so that the labor cost is increased, and the test script generation cost is increased.
Disclosure of Invention
The screenshot method can effectively obtain the accurate screenshot of the operation position, avoids a large amount of invalid irrelevant background information, and obviously reduces screenshot errors relative to the screenshot in a fixed area.
Additional features and advantages of the invention will be set forth in the detailed description which follows, or may be learned by practice of the invention.
According to an aspect of the present invention, there is provided a method for screenshot, including:
acquiring a screen image of a terminal in real time;
processing the screen image according to the texture information of the screen image, and automatically extracting at least one connected region; and finding a connected region where the operation point is located from the at least one connected region, and performing screenshot according to the connected region.
According to some embodiments, processing the screen image according to its texture information, automatically extracting at least one connected region, comprises:
converting the screen image into a gray level image, and acquiring an edge binary image of the gray level image;
and acquiring a connected region distribution diagram according to the edge binarization image, and extracting at least one connected region according to the connected region distribution diagram.
According to some embodiments, obtaining the connected component distribution map from the edge binarized image comprises: and carrying out image expansion and image corrosion treatment on the edge binary image to obtain a distribution diagram of the connected region.
According to the technical scheme provided by the application embodiment, the step of extracting at least one connected region according to the connected region distribution diagram comprises the following steps:
and extracting at least one connected region by using at least one of a minimum circumscribed rectangle, a minimum circumscribed ellipse, a minimum circumscribed circle, a maximum inscribed rectangle, a maximum inscribed circle and a maximum inscribed ellipse according to the connected region distribution map.
According to some embodiments, obtaining the edge binarized image of the grayscale image comprises:
and performing edge extraction on the gray level image to obtain edge information, and performing binarization processing on the edge information through a self-adaptive threshold value to obtain an edge binarization image.
According to some embodiments, edge extracting the gray-scale image to obtain edge information comprises: and performing edge extraction on the gray level image by using an edge extraction method based on a Laplacian operator to obtain edge information.
According to some embodiments, edge extracting the gray-scale image to obtain edge information comprises: and carrying out edge extraction on the gray level image by using an edge extraction method based on a Sobel-x operator to obtain edge information.
According to some embodiments, edge extracting the gray-scale image to obtain edge information comprises: and carrying out edge extraction on the gray level image by using an edge extraction method based on a Sobel-y operator to obtain edge information.
According to some embodiments, edge extracting the gray-scale image to obtain edge information comprises: and carrying out edge extraction on the gray level image by using an edge extraction method based on a Sobel-x-y operator to obtain edge information.
According to some embodiments, edge extracting the gray-scale image to obtain edge information comprises: and performing edge extraction on the gray level image by using an edge extraction method based on a Canny operator to obtain edge information.
According to some embodiments, after the screenshot is performed, verifying the obtained screenshot is further included.
According to some embodiments, the verifying comprises: and checking whether the length-width ratio of the screenshot meets a set ratio.
According to some embodiments, the verifying comprises: and checking whether the size of the screenshot meets a set size range.
According to some embodiments, after the verifying, further comprising: and if the screenshot does not accord with the preset rule, re-screenshot by adopting a fixed size or according to the operation of the user.
According to another aspect of the present invention, there is provided an apparatus for screenshot, comprising
Including screen image acquisition element, connected region extraction element and screenshot unit, wherein:
the screen image acquisition unit is used for acquiring a screen image of the terminal in real time;
the connected region extraction unit is used for processing the screen image according to the texture information of the screen image and automatically extracting at least one connected region; and the screenshot unit is used for finding the connected region where the operation point is located from the at least one connected region and screenshot according to the connected region.
According to some embodiments, the connected region extraction unit is to: converting the screen image into a gray level image, and acquiring an edge binary image of the gray level image;
and acquiring a connected region distribution diagram according to the edge binarization image, and extracting at least one connected region according to the connected region distribution diagram.
According to some embodiments, the connected region extraction unit is to: and carrying out image expansion and image corrosion treatment on the edge binary image to obtain a distribution diagram of the connected region.
According to some embodiments, the connected region extraction unit is to: and extracting at least one connected region by using at least one of a minimum circumscribed rectangle, a minimum circumscribed ellipse, a minimum circumscribed circle, a maximum inscribed rectangle, a maximum inscribed circle and a maximum inscribed ellipse according to the connected region distribution map.
According to some embodiments, the connected region extraction unit is to:
and performing edge extraction on the gray level image to obtain edge information, and performing binarization processing on the edge information through a self-adaptive threshold value to obtain an edge binarization image.
According to some embodiments, the connected region extraction unit is to: and performing edge extraction on the gray level image by using an edge extraction method based on a Laplacian operator to obtain edge information.
According to some embodiments, edge extracting the gray-scale image to obtain edge information comprises: and carrying out edge extraction on the gray level image by using an edge extraction method based on a Sobel-x operator to obtain edge information.
According to some embodiments, the connected region extraction unit is to: and carrying out edge extraction on the gray level image by using an edge extraction method based on a Sobel-y operator to obtain edge information.
According to some embodiments, the connected region extraction unit is to: and carrying out edge extraction on the gray level image by using an edge extraction method based on a Sobel-x-y operator to obtain edge information.
According to some embodiments, the connected region extraction unit is to: and performing edge extraction on the gray level image by using an edge extraction method based on a Canny operator to obtain edge information.
According to some embodiments, the apparatus further comprises a verification unit configured to verify the obtained screenshot after the screenshot is performed.
According to some embodiments, the verification unit is to: and checking whether the length-width ratio of the screenshot meets a set ratio.
According to some embodiments, the verification unit is to: and checking whether the size of the screenshot meets a set size range.
According to some embodiments, the screenshot element is further to: and after the verification, if the screenshot does not accord with the preset rule, re-screenshot by adopting a fixed size or re-screenshot according to the operation of a user.
According to another aspect of the present invention, there is provided a terminal device including: a processor; a memory storing instructions for the processor to control the operation of any one of the above described first aspects.
The technical scheme provided by the embodiment of the application can have the following beneficial effects:
the technical scheme provided by the embodiment of the application can effectively acquire the accurate screenshot of the operation position, avoids a large amount of invalid irrelevant background information, and remarkably reduces a large amount of screenshot errors caused by screenshot in a fixed area. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
Drawings
The above and other features and advantages of the present invention will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.
Fig. 1 illustrates a screen shot of a terminal in a scene in the related art;
fig. 2 illustrates a screen shot image of a terminal in another scene in the related art;
FIG. 3 illustrates a related art image automatically captured at a fixed size in the screenshot image shown in FIG. 1;
FIG. 4 illustrates a related art image automatically captured at a fixed size in the screenshot image shown in FIG. 2;
FIG. 5 illustrates a method for screenshot in accordance with an embodiment of the present invention;
FIG. 6 illustrates a method for screenshot in accordance with another embodiment of the present invention;
FIG. 7 illustrates a method of automatic area screenshots according to another embodiment of the present invention;
FIG. 8 shows an original gray scale map according to an example of an embodiment of the invention;
FIG. 9 shows a binarized image according to an example of an embodiment of the present invention;
FIG. 10 shows a graph of results after primary image expansion according to an example of an embodiment of the invention;
FIG. 11 shows a graph of results after image erosion according to an example of an embodiment of the invention;
FIG. 12 shows a graph of results after expansion of a secondary image according to an example of an embodiment of the invention;
FIG. 13 illustrates an individual contour map in a contour extraction captured image according to an example of an embodiment of the present invention;
FIG. 14 shows a profile diagram containing operating points according to an example of an embodiment of the invention;
FIG. 15 illustrates a target screenshot resulting from the operation point of FIG. 14;
FIG. 16 shows a profile diagram containing operating points according to an example of an embodiment of the invention;
FIG. 17 illustrates a target screenshot taken from the operating point of FIG. 16;
FIG. 18 shows a profile diagram containing operating points according to an example of an embodiment of the invention;
FIG. 19 illustrates a target screenshot taken from the operating point of FIG. 18;
FIG. 20 shows a schematic diagram of a manual screenshot according to an example of an embodiment of the invention;
FIG. 21 shows a block diagram of an apparatus for screenshot according to an embodiment of the present invention;
fig. 22 shows a terminal device according to an embodiment of the present invention.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The same reference numerals denote the same or similar parts in the drawings, and thus, a repetitive description thereof will be omitted.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to provide a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that the invention may be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known methods, devices, implementations or operations have not been shown or described in detail to avoid obscuring aspects of the invention.
The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. I.e. these functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor means and/or microcontroller means.
The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and operations/steps, nor do they necessarily have to be performed in the order described. For example, some operations/steps may be decomposed, and some operations/steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
Fig. 5 illustrates a method for screenshot according to an embodiment of the present invention, which may be applied to a case where a screenshot of an operation control is automatically obtained in an automated test, and as shown in fig. 5, the method for screenshot according to this embodiment includes:
in step S510, a screen image of the terminal is acquired in real time.
Namely, when the terminal is operated, a real-time screen image is acquired by shooting the screen of the terminal.
In step S520, the screen image is processed according to the texture information of the screen image, and at least one connected region is automatically extracted.
The step can be realized in various ways, for example, the screen image can be converted into a gray level image, an edge binarization image of the gray level image is obtained, a connected region distribution diagram is obtained according to the edge binarization image, and at least one connected region is extracted according to the connected region distribution diagram.
The edge information obtained by performing edge extraction on the gray-scale image can be obtained by various methods, which are not limited in this embodiment, and for example, the edge information obtained by performing edge extraction on the gray-scale image can be obtained by using an edge extraction method based on a Laplacian operator, an edge extraction method based on a Sobel-x operator, an edge extraction method based on a Sobel-y operator, an edge extraction method based on a Sobel-x-y operator, an edge extraction method based on a Canny operator, or the like.
The edge binarization image of the gray level image can be obtained by various methods, which are not limited in this embodiment, for example, edge information can be obtained by performing edge extraction on the gray level image, and the edge information is subjected to binarization processing by using an adaptive threshold value to obtain the edge binarization image.
For obtaining at least one connected region according to the edge binarization image, obtaining a connected region distribution map according to the edge binarization image can obtain the connected region distribution map by performing image expansion and image erosion processing on the edge binarization image. And for example, performing image expansion, corrosion, secondary expansion and other operations on the edge binary image to extract at least one connected region. The extracting of the at least one connected region according to the connected region distribution map may specifically be extracting the at least one connected region according to the connected region distribution map by using at least one of a minimum circumscribed rectangle, a minimum circumscribed ellipse, a minimum circumscribed circle, a maximum inscribed rectangle, a maximum inscribed circle, and a maximum inscribed ellipse.
In step S530, finding a connected region where the operation point is located from the at least one connected region, and performing screenshot according to the connected region.
It should be noted that, if the operation point is exactly in one connected region, the image in the connected region is taken as a screenshot; if the operation point is surrounded by a plurality of connected areas, taking the image in the connected area with the minimum area as a screenshot; and if the position of the operating point is not in any communication area, performing screenshot from the operating point according to the size of the fixed area.
Further, in order to further improve the accuracy of screenshot, the obtained screenshot can be verified after the screenshot is performed. If the screenshot does not accord with the preset rule, the screenshot can be re-captured by adopting a fixed size or according to the operation of a user. The specific verification may include multiple types, for example, length-width ratio verification, verification whether the length-width ratio of the screenshot meets a set ratio, or size verification, verification whether the size of the screenshot meets a set size range, or both length-width ratio verification and size verification.
In this embodiment, a screen image is converted into a grayscale image, an edge binarization image of the grayscale image is obtained, a connected region distribution map is obtained according to the edge binarization image, at least one connected region is extracted according to the connected region distribution map, a connected region where an operation point is located is found from the at least one connected region, and a screenshot is performed according to the connected region. The method can effectively obtain the accurate screenshot of the operation position, avoid a large amount of invalid irrelevant background information, and remarkably reduce a large amount of screenshot errors caused by screenshot in a fixed area.
Fig. 6 shows a method for screenshot, according to another embodiment of the present invention, the user performs continuous actual operations, and the automatic screenshot function is performed in real time in a background process of the computer according to the user operations. Fig. 6 shows a flowchart of a whole automatic screenshot method, where the method for screenshot according to this embodiment includes:
in step S610, a device operation of a user is received.
In step S620, a real-time screen capture is performed on the screen.
In step S630, an automatic area screenshot is performed on the screenshot according to a user operation.
In step S640, the screenshot is checked. If the screenshot verification is successful, step S660 is executed to obtain the screenshot as a result screenshot, and if the screenshot verification is failed, step S650 is executed.
In step S650, if the screenshot fails to be verified, a screenshot with a fixed area size is performed on the screenshot again according to the user operation.
In step S660, the screenshot is obtained as the resulting screenshot.
The method for automatically screenshot in an area is shown in fig. 7, and the method for automatically screenshot in this embodiment includes:
in step S710, a screen image of the user operation time is acquired.
In step S720, the screen image is converted into a grayscale image.
In operating the device, a live screen is captured by capturing a screen of the device and converting it into a grayscale image, and fig. 8 shows an original grayscale map according to an example of an embodiment of the invention, as shown in fig. 8.
Since the color image is a three-channel image, which is essentially a two-dimensional rgb value matrix, and each pixel point of the color image is an rgb combined value (e.g., (50,250,155), rather than a simple number, it is troublesome to perform image processing, and therefore, the color image is converted into a gray image, which is convenient for the subsequent image algorithm processing.
The gray image is a two-dimensional digital matrix, and each pixel point of the gray image is a value of 0 to 255, which is similar to: [[0,255,120,250],[0,120,255,250],[0,120,255,250]].
In step S730, image edge information is extracted to obtain an edge binarized image.
The image edge information is found from the grayscale image obtained in step S720, and then the image edge information is subjected to a binarization process once.
The purpose of the binarization processing is to: when the edge information is extracted, the edge gray image is still a gray image, the edge gray image is converted into a 0-1 type image (non-black or white) through self-adaptive threshold binarization, and whether the image area has the same attribute can be judged directly through the 0-1 attribute of the image area.
After the binarization processing, the effective operation area and the background area can be distinguished according to the edge information in the original image. In the process, a plurality of image edge extraction methods can be selected, for example, an edge extraction method based on a Laplacian operator, an edge extraction method based on a Sobel-x operator, an edge extraction method based on a Sobel-y operator, an edge extraction method based on a Sobel-x-y operator, an edge extraction method based on a Canny operator and the like can be used for realizing edge extraction on the gray image to obtain edge information.
In this step, the image is analyzed, and the edge and texture information in the image is expressed using a clear black and white region. The better the separation between different edge texture blocks, the better the subsequent processing. In this embodiment, an edge extraction method using a sobel-x operator is selected to perform edge extraction on the grayscale image shown in fig. 8, so as to obtain an edge binarization image shown in fig. 9.
In step S740, the edge binarized image is subjected to image expansion and erosion to obtain a connected region.
Image erosion and image dilation are common methods for processing binarized images. Belongs to a common method of morphology. In step S730, an image edge binary image is already obtained according to the edge information, in this step, the edge information needs to be further highlighted, adjacent areas are merged through image expansion, and the fragmentation condition of the binary image area in step S730 is reduced; isolated white edge points are eliminated through image corrosion, and excessive connected areas are avoided. And acquiring an ideal region connectivity map through multiple times of image erosion-image expansion.
Fig. 10 shows a result chart after primary image expansion is performed on the basis of the edge binarized image shown in fig. 9, fig. 11 shows a result chart after image erosion is performed on the basis of the result chart obtained in fig. 9, and fig. 12 shows a result chart after secondary image expansion is performed on the basis of the result chart obtained in fig. 11.
In the above drawings, it can be seen that the original binarized image in fig. 9 is subjected to primary connectivity of near binarized edge information through primary image expansion, and a basic connectivity image is obtained as shown in fig. 10. The basic connected image shown in fig. 10 is subjected to image erosion once, and some small isolated connected regions are removed, so that an eroded image is obtained as shown in fig. 11. The image expansion is performed again on the image shown in fig. 11 to obtain a more ideal connected component distribution map as shown in fig. 12, and the image shown in fig. 12 is the obtained connected component distribution map and can be regarded as a basic contour connected component map.
The connected regions here all correspond to regions where adjacent edge information in the original device screenshot image is expanded and connected, that is, regions located in the same edge, and are in accordance with the usage scenario described at the beginning of this embodiment, and may be used to acquire a connected region near the operation position.
It should be noted that erosion is a process of eliminating boundary points and shrinking boundaries inward. Can be used to eliminate small and meaningless objects. Dilation is the process of merging all background points in contact with an object into the object, expanding the boundary outward. Can be used to fill in voids in objects.
The process of erosion followed by dilation is called an on operation. For eliminating small objects, separating objects at fine points, smoothing the boundaries of larger objects without significantly changing their area. The process of expansion followed by erosion is called closed-loop operation. The filling material is used for filling fine holes in an object, connecting adjacent objects and smoothing the boundary of the objects without obviously changing the area of the objects.
In step S740, the present embodiment performs image processing using the dilation-erosion-dilation method. In fact, the method performs an operation process of 'closing operation-opening operation'. In the actual use process, the process of 'closing operation-opening operation' can be repeated for a plurality of times to obtain the most appropriate binarization area image. The specific number and sequence of expansions and erosions are not limited in this example and are included within the scope of the present disclosure.
In step S750, a precise region is acquired.
After the basic connected region is acquired in step S740, different individual contours may be framed by morphological contour extraction. The contour extraction of the binary contour can be carried out by using a minimum circumscribed rectangle, a minimum circumscribed ellipse, a minimum circumscribed circle, a maximum inscribed rectangle, a maximum inscribed circle and a maximum inscribed ellipse. According to the use scene of the accurate screenshot, the minimum circumscribed rectangle is selected to extract the connected region.
Fig. 12 is the exact connected graph obtained in step S750, and fig. 13 is the result of extracting the independent outline using the minimum bounding rectangle and drawing on the original equipment screenshot using the same outline (filtering the undersized area and the abnormally-sized area).
In step S760, a screenshot is acquired.
According to the position of the operation point, finding out the outline area where the operation point is located from the rectangular outline area obtained in the step S750, that is, the area of the accurate screenshot, and performing picture capture on the area to obtain the target screenshot.
As shown in fig. 14, in this embodiment, the operation position of the operator in the screen interface of the device is a black point (as shown by a black point in the third box on the left side of fig. 14), and a corresponding region is captured just within a certain extracted rectangular outline, which is the captured image obtained in fig. 15:
in this embodiment, an abnormal situation needs to be considered, and the operation point may not be included in any extracted region. For example, in the case that the operation point (as indicated by the black dot in the second box at the upper left part of fig. 16) in fig. 16 may be surrounded by multiple rectangles, the rectangle with the smallest area is taken as the target screenshot (the innermost region) as shown in fig. 17.
As shown in fig. 18, if the operation point position (indicated by the black dot shown at the upper left of fig. 18) is not within the extracted region, the screen is captured by selecting the fixed region size described above directly from the operation point, and the result is shown in fig. 19.
The above parts are the basic flow of the automatic area screenshot and the detail presentation of the actual processing image. After the automatic area screenshot is completed, it cannot be ensured that the obtained screenshot is a very accurate screenshot, and therefore screenshot verification needs to be performed. The check rule is related to the use scene, and two check rules are set in the invention: aspect ratio anomaly, size anomaly.
The verification rule includes various types, and for example, the following verification methods can be adopted:
checking the length-width ratio, wherein the width-height ratio exceeds 10:1, or the height-width ratio exceeds 10:1, and judging that the length-width ratio is abnormal;
and secondly, performing size verification, wherein the height of the screenshot exceeds 50% of the screenshot of the equipment, or the width of the screenshot exceeds 50% of the screenshot of the equipment, and judging that the size is abnormal.
Checking the processing scheme of rule exception: and (5) converting the fixed size to screenshot. Setting the fixed width of the screenshot as 2w, setting the fixed height of the screenshot as 2h, taking an operation point as a center, respectively extending the distance w from left to right, respectively extending the distance up and down, if the extension exceeds the edge of the screen of the equipment, intercepting the screenshot until the corresponding edge, and taking the rectangular image in the range as a target screenshot.
If the screenshot obtained in the automatic screenshot process is not good, the screenshot needs to be performed in response to manual screenshot operation after the automatic screenshot is finished.
Fig. 20 shows a schematic diagram of a manual screenshot according to an example of an embodiment of the present invention, which takes a rectangular image as an example, and is specifically implemented as follows: the screenshot area has 4 corner points AB C D in the device, and the user has four screenshot operation modes which can be selected:
① pressing down the device from point A and lifting up the device from point C;
② pressing down the device from point C and lifting up the device from point A;
③ pressing down the device from point B and lifting up the device from point D;
④ pressing down the device from point D and lifting up the device from point B;
when the user finishes the screenshot operation, the two operation points are respectively x1 and x2 in abscissa and y1 and y2 in ordinate relative to the upper left corner of the picture, and then rectangular areas in the ranges of x1 to x2 and y1 to y2 are taken as screenshot results.
Through the image processing process, the independent connected region of the whole screen capture is extracted, and then the connected region where the accurate operation point is located is taken out by combining the position of the operation point. And when the automatic acquisition of the accurate communication area fails, a method for screenshot by using a fixed size is used, so that the screenshot can be ensured to be automatically acquired. And in the case that the automatic screenshot acquisition is not ideal, providing a manual screenshot function.
According to the method and the device, the accurate screenshot of the operation position can be effectively obtained in the application scene of automatically generating the screenshot along with the operation through the connected region extraction and screenshot of the operation position, a large amount of invalid irrelevant background information is avoided, a large amount of screenshot errors caused by screenshot of a fixed region are remarkably reduced, and the subsequent processing of the automatic screenshot result is facilitated. The automatic screenshot utilizes the screenshot with the fixed area size under the specified abnormal condition, and the effectiveness of the screenshot is guaranteed to the maximum extent. When the automatic screenshot does not meet the use requirement, screenshot is carried out in a specified manual screenshot mode, and the completeness of the screenshot method and the generation of a final available result are guaranteed.
Fig. 21 is a block diagram illustrating an apparatus for capturing a screenshot according to an embodiment of the present invention, and as shown in fig. 21, the apparatus for capturing a screenshot according to the embodiment includes: a screen image acquiring unit 2110, a connected region extracting unit 2120, and a screen shot unit 2130.
The screen image obtaining unit 2110 is configured to obtain a screen image of the terminal in real time;
the connected component extracting unit 2120 is configured to process the screen image according to texture information of the screen image, and automatically extract at least one connected component;
the screenshot unit 2130 is configured to find a connected region where an operation point is located from the at least one connected region, and perform screenshot according to the connected region.
The connected component extracting unit 2120 is configured to convert the screen image into a grayscale image, and obtain an edge binary image of the grayscale image;
and acquiring a connected region distribution diagram according to the edge binarization image, and extracting at least one connected region according to the connected region distribution diagram.
According to some embodiments of the invention, the connected region extracting unit 2120 is configured to: and carrying out image expansion and image corrosion treatment on the edge binary image to obtain a distribution diagram of the connected region.
According to some embodiments of the invention, the connected region extracting unit 2120 is configured to: and extracting at least one connected region by using at least one of a minimum circumscribed rectangle, a minimum circumscribed ellipse, a minimum circumscribed circle, a maximum inscribed rectangle, a maximum inscribed circle and a maximum inscribed ellipse according to the connected region distribution map.
According to some embodiments of the invention, the connected region extracting unit 2120 is configured to:
and performing edge extraction on the gray level image to obtain edge information, and performing binarization processing on the edge information through a self-adaptive threshold value to obtain an edge binarization image.
According to some embodiments of the invention, the connected region extracting unit 2120 is configured to:
performing edge extraction on the gray level image by using an edge extraction method based on a Laplacian operator to obtain edge information; or
Performing edge extraction on the gray level image by using an edge extraction method based on a Sobel-x operator to obtain edge information; or
Performing edge extraction on the gray level image by using an edge extraction method based on a Sobel-y operator to obtain edge information; or
Performing edge extraction on the gray level image by using an edge extraction method based on a Sobel-x-y operator to obtain edge information; or
And performing edge extraction on the gray level image by using an edge extraction method based on a Canny operator to obtain edge information.
According to some embodiments of the present invention, the apparatus further includes a verification unit (not shown in fig. 21) configured to verify the obtained screenshot after the screenshot is performed.
According to some embodiments of the invention, the verification unit is to: and checking whether the length-width ratio of the screenshot meets a set ratio.
According to some embodiments of the invention, the verification unit is to: and checking whether the size of the screenshot meets a set size range.
According to some embodiments of the invention, the screenshot unit is further configured to: and after the verification, if the screenshot does not accord with the preset rule, re-screenshot by adopting a fixed size or re-screenshot according to the operation of a user.
With regard to the apparatus in the above-described embodiment, the specific manner in which each unit performs the operation has been described in detail in the embodiment related to the method, and will not be described in detail here.
The device for screenshot provided by the embodiment can execute the method for screenshot provided by the first embodiment and the second embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.
Fig. 22 shows a terminal device according to an embodiment of the present invention, and as shown in fig. 22, the terminal device 2200 may include a processor 2210, a memory 2220, a transmitter 2230, and a receiver 2240.
Memory 2220 may store instructions for processor 2210 to control the processing of operations. The memory 2220 may include volatile or nonvolatile memory, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), and the like, which is not limited in this respect.
Processor 2210 may call instructions stored in memory 2220 to control related operations. According to one embodiment, memory 2220 stores instructions for processor 2210 to control the operation of the method described above for the screenshot.
For example, acquiring a screen image of the terminal in real time; converting the screen image into a gray level image, and acquiring an edge binary image of the gray level image; acquiring a connected region distribution diagram according to the edge binarization image, and extracting at least one connected region according to the connected region distribution diagram; and finding a connected region where the operation point is located from the at least one connected region, and performing screenshot according to the connected region.
It will be readily appreciated that memory 2220 may also store instructions for processor 2210 to control other operations according to embodiments of the present invention, which will not be described in detail herein.
Processor 2210 may also control transmitter 2230 and receiver 2240 for signaling, and the like.
Those skilled in the art will readily appreciate from the foregoing detailed description that the systems and methods according to embodiments of the present invention have one or more of the following advantages.
The present invention also provides, according to some embodiments, a non-transitory computer-readable storage medium, such as a memory, including instructions executable by a processor of an apparatus to perform the above-described method. For example, the non-transitory computer readable storage medium may be a ROM, a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like. The instructions in the storage medium, when executed by a processor of the terminal, enable the terminal to perform the method of: transforming the native resource name in the script module into a native resource ID; and the native application module acquires the corresponding native resource through the native resource ID.
It will be appreciated by those skilled in the art that the drawings are merely schematic representations of exemplary embodiments, and that the blocks or flow charts in the drawings are not necessarily required to practice the present invention and are, therefore, not intended to limit the scope of the present invention.
Those skilled in the art will appreciate that the modules described above may be distributed in the apparatus according to the description of the embodiments, or may be modified accordingly in one or more apparatuses unique from the embodiments. The modules of the above embodiments may be combined into one module, or further split into multiple sub-modules.
Exemplary embodiments of the present invention are specifically illustrated and described above. It is to be understood that the invention is not to be limited to the disclosed embodiments, but on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
Claims (12)
1. A method for screenshot, comprising:
acquiring a screen image of a terminal in real time;
analyzing the screen image according to the texture information of the screen image, and automatically extracting at least one connected region;
finding a connected region where an operation point is located from the at least one connected region, and if the operation point is in one connected region, determining that an image in the connected region is a screenshot;
if the operation point is surrounded by the plurality of connected regions, determining the image in the connected region with the minimum area as a screenshot;
and if the position of the operating point is not in any one of the communication areas, performing screenshot from the operating point according to the size of the fixed area.
2. The method of claim 1, wherein processing the screen image according to texture information of the screen image to automatically extract at least one connected region comprises:
converting the screen image into a gray level image, and acquiring an edge binary image of the gray level image;
and acquiring a connected region distribution diagram according to the edge binarization image, and extracting at least one connected region according to the connected region distribution diagram.
3. The method as claimed in claim 2, wherein obtaining a connected component distribution map from the edge binarized image comprises: and carrying out image expansion and image corrosion treatment on the edge binary image to obtain a distribution diagram of the connected region.
4. The method of claim 2, wherein extracting at least one connected region from the connected region profile comprises:
and extracting at least one connected region by using at least one of a minimum circumscribed rectangle, a minimum circumscribed ellipse, a minimum circumscribed circle, a maximum inscribed rectangle, a maximum inscribed circle and a maximum inscribed ellipse according to the connected region distribution map.
5. The method of claim 2, wherein obtaining an edge binarized image of the grayscale image comprises:
and performing edge extraction on the gray level image to obtain edge information, and performing binarization processing on the edge information through a self-adaptive threshold value to obtain an edge binarization image.
6. The method of claim 5, wherein performing edge extraction on the grayscale image to obtain edge information comprises:
performing edge extraction on the gray level image by using an edge extraction method based on a Laplacian operator to obtain edge information; or
Performing edge extraction on the gray level image by using an edge extraction method based on a Sobel-x operator to obtain edge information; or
Performing edge extraction on the gray level image by using an edge extraction method based on a Sobel-y operator to obtain edge information; or
Performing edge extraction on the gray level image by using an edge extraction method based on a Sobel-x-y operator to obtain edge information; or
And performing edge extraction on the gray level image by using an edge extraction method based on a Canny operator to obtain edge information.
7. The method of claim 1, further comprising, after making the screenshot, verifying the resulting screenshot.
8. The method of claim 7, wherein the verifying comprises:
checking whether the length-width ratio of the screenshot meets a set ratio; and/or
And checking whether the size of the screenshot meets a set size range.
9. The method of claim 7, further comprising, after performing the verification: and if the screenshot does not accord with the preset rule, re-screenshot by adopting a fixed size or according to the operation of the user.
10. The device for screenshot is characterized by comprising a screen image acquisition unit, a connected region extraction unit and a screenshot unit, wherein:
the screen image acquisition unit is used for acquiring a screen image of the terminal in real time;
the connected region extraction unit is used for processing the screen image according to the texture information of the screen image and automatically extracting at least one connected region;
the screenshot unit is used for finding a connected region where an operation point is located from the at least one connected region, and if the operation point is in one connected region, determining that an image in the connected region is a screenshot;
if the operation point is surrounded by the plurality of connected regions, determining the image in the connected region with the minimum area as a screenshot;
and if the position of the operating point is not in any one of the communication areas, performing screenshot from the operating point according to the size of the fixed area.
11. The apparatus of claim 10, wherein the connected region extraction unit is further to:
converting the screen image into a gray level image, and acquiring an edge binary image of the gray level image;
and acquiring a connected region distribution diagram according to the edge binarization image, and extracting at least one connected region according to the connected region distribution diagram.
12. A terminal device, comprising: a processor; a memory storing instructions for the processor to control the operations of any of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710109123.2A CN106875408B (en) | 2017-02-27 | 2017-02-27 | Screenshot method and device and terminal equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710109123.2A CN106875408B (en) | 2017-02-27 | 2017-02-27 | Screenshot method and device and terminal equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106875408A CN106875408A (en) | 2017-06-20 |
| CN106875408B true CN106875408B (en) | 2020-03-17 |
Family
ID=59167977
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710109123.2A Active CN106875408B (en) | 2017-02-27 | 2017-02-27 | Screenshot method and device and terminal equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106875408B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107256414A (en) * | 2017-06-27 | 2017-10-17 | 哈尔滨工业大学 | Polarimetric SAR Image convolutional neural networks sorting technique based on spatial feature collection of illustrative plates |
| CN107807769A (en) * | 2017-09-29 | 2018-03-16 | 珠海市魅族科技有限公司 | Sectional drawing control method and device, terminal installation and computer-readable recording medium |
| CN107833227B (en) * | 2017-10-31 | 2021-11-26 | 苏州乐米信息科技股份有限公司 | Drawing method and system for circular cutting area |
| CN110442521B (en) * | 2019-08-02 | 2023-06-27 | 腾讯科技（深圳）有限公司 | Control unit detection method and device |
| CN112862694A (en) * | 2019-11-12 | 2021-05-28 | 合肥欣奕华智能机器有限公司 | Screen position correction method and device, computing equipment and storage medium |
| CN113127349B (en) * | 2021-04-19 | 2021-11-30 | 广州掌动智能科技有限公司 | Software testing method and system |
| CN113805993B (en) * | 2021-09-03 | 2023-06-06 | 四川新网银行股份有限公司 | Method for rapidly and continuously capturing images |
| CN114549551A (en) * | 2022-02-15 | 2022-05-27 | Oppo广东移动通信有限公司 | Image cutting method and device, terminal and readable storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103577817A (en) * | 2012-07-24 | 2014-02-12 | 阿里巴巴集团控股有限公司 | Method and device for identifying forms |
| CN105741300A (en) * | 2016-02-03 | 2016-07-06 | 浙江科澜信息技术有限公司 | Region segmentation screenshot method |
| CN106325745A (en) * | 2016-08-23 | 2017-01-11 | 上海卓易科技股份有限公司 | Screen capturing method and device |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101408942B (en) * | 2008-04-17 | 2011-01-12 | 浙江师范大学 | Method for locating license plate under a complicated background |
| US20100205529A1 (en) * | 2009-02-09 | 2010-08-12 | Emma Noya Butin | Device, system, and method for creating interactive guidance with execution of operations |
| CN103106658A (en) * | 2013-01-23 | 2013-05-15 | 中国人民解放军信息工程大学 | Island or reef coastline rapid obtaining method |
| CN105303189B (en) * | 2014-07-29 | 2019-08-20 | 阿里巴巴集团控股有限公司 | It is a kind of for detecting the method and device of specific identifier image in presumptive area |
| KR102325345B1 (en) * | 2014-12-15 | 2021-11-11 | 삼성전자주식회사 | Interactive image segmentation apparatus and method |
| CN105427320B (en) * | 2015-11-30 | 2018-07-06 | 威海北洋电气集团股份有限公司 | A kind of image division and extracting method |
| CN105389827A (en) * | 2015-12-24 | 2016-03-09 | Tcl集团股份有限公司 | Method and device for acquiring television station logo region |
-
2017
- 2017-02-27 CN CN201710109123.2A patent/CN106875408B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103577817A (en) * | 2012-07-24 | 2014-02-12 | 阿里巴巴集团控股有限公司 | Method and device for identifying forms |
| CN105741300A (en) * | 2016-02-03 | 2016-07-06 | 浙江科澜信息技术有限公司 | Region segmentation screenshot method |
| CN106325745A (en) * | 2016-08-23 | 2017-01-11 | 上海卓易科技股份有限公司 | Screen capturing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106875408A (en) | 2017-06-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106875408B (en) | Screenshot method and device and terminal equipment | |
| US20190347767A1 (en) | Image processing method and device | |
| CN110569341B (en) | Method and device for configuring chat robot, computer equipment and storage medium | |
| CN109753953B (en) | Method and device for positioning text in image, electronic equipment and storage medium | |
| CN111275034B (en) | Method, device, equipment and storage medium for extracting text region from image | |
| US11580665B2 (en) | Image positioning system and image positioning method based on upsampling | |
| CN112101386B (en) | Text detection method, device, computer equipment and storage medium | |
| CN109063598A (en) | Face pore detection method, device, computer equipment and storage medium | |
| US9117132B2 (en) | System and method facilitating designing of classifier while recognizing characters in a video | |
| US9824289B2 (en) | Exploiting color for license plate recognition | |
| CN111507957A (en) | Identity card picture conversion method and device, computer equipment and storage medium | |
| CN111079730A (en) | Method for determining area of sample image in interface image and electronic equipment | |
| CN112381092A (en) | Tracking method, device and computer readable storage medium | |
| CN113283439B (en) | Intelligent counting method, device and system based on image recognition | |
| CN113112511B (en) | Method and device for correcting test paper, storage medium and electronic equipment | |
| CN113627423A (en) | Circular seal character recognition method and device, computer equipment and storage medium | |
| CN110310341B (en) | Method, device, equipment and storage medium for generating default parameters in color algorithm | |
| CN108174091B (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| JP4967045B2 (en) | Background discriminating apparatus, method and program | |
| CN111738961B (en) | Electronic image restoration method, device and equipment and readable storage medium | |
| CN112634382A (en) | Image recognition and replacement method and device for unnatural object | |
| US20150278636A1 (en) | Image processing apparatus, image processing method, and recording medium | |
| CN111435544B (en) | Picture processing method and device | |
| CN107392205B (en) | Code value table generation method and device of remote controller | |
| CN116993654B (en) | Camera module defect detection method, device, equipment, storage medium and product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |