CN107491778B - Intelligent device screen extraction method and system based on positioning image - Google Patents
Intelligent device screen extraction method and system based on positioning image Download PDFInfo
- Publication number
- CN107491778B CN107491778B CN201710731243.6A CN201710731243A CN107491778B CN 107491778 B CN107491778 B CN 107491778B CN 201710731243 A CN201710731243 A CN 201710731243A CN 107491778 B CN107491778 B CN 107491778B
- Authority
- CN
- China
- Prior art keywords
- screen
- image
- positioning
- intelligent
- positioning image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000605 extraction Methods 0.000 title claims abstract description 67
- 238000000034 method Methods 0.000 claims abstract description 38
- 239000000284 extract Substances 0.000 claims abstract description 31
- 238000012937 correction Methods 0.000 claims description 19
- 230000009286 beneficial effect Effects 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 238000010801 machine learning Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012550 audit Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 239000011362 coarse particle Substances 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 239000010419 fine particle Substances 0.000 description 1
- 238000013467 fragmentation Methods 0.000 description 1
- 238000006062 fragmentation reaction Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
- G06V10/242—Aligning, centring, orientation detection or correction of the image by image rotation, e.g. by 90 degrees
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
- H04M1/724—User interfaces specially adapted for cordless or mobile telephones
- H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an intelligent equipment screen extraction method based on a positioning image, which comprises the following steps: 100: displaying the positioning image on a screen of the intelligent equipment; 200: acquiring a screen image of the intelligent equipment and transmitting the screen image to an extraction server; 300: the extraction server side identifies the positioning image to determine the original coordinate of the positioning image; 400: the extraction server side corrects the screen image of the intelligent equipment; 500: the extraction server side searches outwards from the positioning image and extracts an inner frame of the screen of the intelligent equipment displayed in the screen image of the intelligent equipment; 600: and the extraction server extracts the screen image of the intelligent equipment within the frame range in the screen of the intelligent equipment. In addition, the invention also provides a corresponding system and application. The method and the device can automatically and conveniently extract the screen image of the intelligent equipment within the frame range in the screen of the intelligent equipment. And further extracting the screen image of the intelligent device within the range of the outer frame of the screen of the intelligent device.
Description
Technical Field
The invention relates to a method and a system for extracting a device screen, in particular to a method and a system for extracting an intelligent device screen.
Background
At present, intelligent devices such as mobile phones and tablets become an essential part of daily life of people. As smart devices become more and more, services related to device screens, such as device screen insurance, image recognition of device screen content, and the like, also become more and more. These services related to the screen of the device generally need to automatically and conveniently extract the screen image of the device, for example, quickly extract the screen area image of the device by means of machine-associated prompts, and perform machine-checking operations on the screen content in the image, so as to determine the quality of the screen or identify the related content in the screen area.
Generally, extracting a screen image of a device requires taking a picture of the front of the device and then extracting the screen image in the picture. The traditional method for extracting the screen image in the whole shot photo has the following problems:
(1) different intelligent devices have different screen sizes and different length-width ratios, so that the difficulty of identifying the screen range of the device is improved.
(2) Different people and different environments cause the background of the screen of the equipment to be completely different, and the background is not beneficial to determining the quality of the screen or identifying relevant contents in the screen area and needs to be removed. For example, when determining the quality of a screen in a model manner, it is costly to consider these diversified backgrounds when constructing a training sample.
Disclosure of Invention
One of the purposes of the invention is to provide a method for extracting a screen of an intelligent device based on a positioning image, which can automatically and conveniently extract the screen image of the intelligent device within the frame range in the screen of the intelligent device. And further extracting the screen image of the intelligent device within the range of the outer frame of the screen of the intelligent device.
According to the above purpose, the present invention provides a method for extracting a screen of an intelligent device based on a positioning image, which comprises the following steps:
100: displaying a positioning image for determining the position, size and direction of the screen of the intelligent equipment on the screen of the intelligent equipment;
200: acquiring an intelligent equipment screen image containing a positioning image and transmitting the intelligent equipment screen image containing the positioning image to an extraction server;
300: the extraction server side identifies the positioning image to determine the original coordinate of the positioning image in the screen image of the intelligent device;
400: the extraction server corrects the screen image of the intelligent device on the basis of the original coordinates;
500: the extraction server side searches and extracts an inner frame of the screen of the intelligent equipment displayed in the screen image of the intelligent equipment from the positioning image;
600: and the extraction server extracts the screen image of the intelligent device within the frame range in the screen of the intelligent device.
The method for extracting the screen of the intelligent equipment based on the positioning image displays the positioning image on the screen of the intelligent equipment, identifies the positioning image in the acquired screen image of the intelligent equipment and determines the original coordinate of the positioning image, and can correct the screen image of the intelligent equipment based on the original coordinate of the positioning image, search and extract the inner frame of the screen of the intelligent equipment from the positioning image and further extract the screen image of the intelligent equipment within the range of the inner frame of the screen of the intelligent equipment as the position, the size and the direction of the screen of the intelligent equipment can be determined based on the original coordinate of the positioning image. Wherein:
the positioning image in step 100 may be preset in size and displayed at a fixed position in the screen of the smart device, and thus may be used to determine the position, size, and direction of the screen of the smart device. The positioning image generally has different easily recognized features from the screen image of the smart device, so that the positioning image is conveniently recognized.
In step 200, the screen image of the smart device may be captured and transmitted by an image capturing and transmitting device, such as a mobile phone, having an image capturing function and an image transmitting function.
In step 300, the positioning image may be identified by scanning to determine its original coordinates. If the positioning image cannot be found, the user can be prompted to find the positioning image and to adjust the shooting position and angle.
In some embodiments, the smart device screen image may be corrected according to the mapping relationship between the original coordinates and the expected coordinates in step 400. The coordinate system of the desired coordinates is generally a two-dimensional coordinate system perpendicular to the image capturing optical axis of the screen of the smart device.
In step 500 and step 600, because the positioning image is located inside the frame inside the screen of the smart device, the frame inside the screen of the smart device can be searched outwards from the positioning image and the screen image of the smart device within the frame range inside the screen of the smart device can be extracted.
In conclusion, the method can automatically and conveniently extract the screen image of the intelligent equipment within the frame range in the screen of the intelligent equipment.
Further, in the method for extracting a screen of an intelligent device based on a positioning image according to the present invention, step 400 further includes: after the screen image of the intelligent device is corrected, the corrected coordinates of the positioning image in the screen image are determined or acquired.
Further, in the method for extracting a screen of an intelligent device based on a positioning image, the positioning image is a positioning image generated immediately or a positioning image stored in a database.
In the above scheme, the positioning image may use an existing positioning image stored in the database, or may be generated in real time according to a specific business logic. The positioning image stored in the database may use a one-dimensional code, a two-dimensional code, or the like.
Further, in the method for extracting the screen of the intelligent device based on the positioning image, the positioning image is a two-dimensional code image.
In the above scheme, the two-dimensional code image belongs to a two-dimensional positioning image. If the screen image of the intelligent device needs to be completely extracted subsequently, the positioning image is preferably a positioning image with two dimensions, namely a two-dimensional positioning image. Using two-dimensional positioning images generally requires consideration of at least four points of the quadrilateral through which the complete projection correction of the device screen image can be performed. For the two-dimensional code, since the two-dimensional code has three anchor points and one hidden anchor point information, a complete screen correction can be performed through the anchor point information.
Of course, the positioning image may also be selected as a one-dimensional code. Locating an image using a one-dimensional code typically requires attention to the following points:
the one-dimensional code can only support left and right correction of the image, and does not have enough information to support front and back of the image, namely projection correction;
the one-dimensional code cannot calculate the range of the screen through complete deduction logic, and can only find the position of the screen in a heuristic mode, such as a connected region, flooding and approximate parallel finding.
Further, in the method for extracting the screen of the intelligent device based on the positioning image, the extraction server is a client or a cloud, and an extraction program runs on the client or the cloud.
In the above scheme, the extraction server executes the relevant steps through the extraction program. The client or the cloud can be selected as the extraction server according to specific business requirements.
When the extraction server side is the cloud side, interaction can be carried out on the intelligent device of the screen image to be extracted through the cloud side, so that generation of the positioning image is controlled more finely, the fine control is simpler and more convenient for subsequent screen image extraction, and complicated back-end processing logic is avoided. For example, the two-dimensional code is selected as the positioning image, parameters can be returned to the intelligent device of the screen image to be extracted through the cloud service provider, the two-dimensional code positioning image is generated in the middle of the screen, namely the height is 50% and the width is 50%, and the subsequent cloud service provider can extract the screen image by using the parameters.
Further, in the method for extracting a screen of an intelligent device based on a positioning image, the correction includes rotation correction and projection correction.
Further, in the method for extracting a screen of an intelligent device based on a positioning image, the positioning image contains identity information, so that when the extraction server receives the screen image of the intelligent device containing the positioning image, whether the screen image of the intelligent device is the screen image of the intelligent device specified or not is judged.
In the above scheme, since the specific service requirement usually requires to audit the identity of the intelligent device from which the screen image is to be extracted, the identity information is added to the positioning image so as to determine whether the screen image of the intelligent device is the screen image of the intelligent device specified.
For a two-dimensional code positioning image, information encoding is generally required to be supported, and if a plurality of two-dimensional code positioning images exist on a screen image of an intelligent device, which intelligent device belongs to the screen image to be extracted can be determined or not through decoding the two-dimensional code positioning images.
Further, in the method for extracting a screen of an intelligent device based on a positioning image according to the present invention, in step 500, the extraction server reversely deduces the position of the inner frame of the screen of the intelligent device based on the preset size ratio relationship between the positioning image and the screen of the intelligent device or the distance between the edge of the positioning image and the inner frame of the screen of the intelligent device, so as to search and extract the inner frame of the screen of the intelligent device displayed in the screen image of the intelligent device from the positioning image.
In the above scheme, the positioning image may be a quadrangle, such as a two-dimensional code. Taking a two-dimensional code as an example, the two-dimensional code generally has three anchor points and one hidden anchor point information. The positioning point information can be used for displaying based on the position relation, such as the proportional relation or the edge distance, of the preset two-dimensional code and the inner frame of the intelligent device screen, so that the size and the position of the inner frame of the intelligent device screen in the corrected intelligent device screen can be reversely deduced according to the size and the position of the two-dimensional code in the corrected intelligent device screen image, and the intelligent device screen image within the range of the inner frame of the intelligent device screen is extracted.
Further, in the method for extracting a screen of an intelligent device based on a positioning image according to the present invention, in the step 500, an inner frame of the screen of the intelligent device displayed in the screen image of the intelligent device is searched and extracted from the positioning image in a manner of area connectivity determination.
In the above scheme, if the backward derivation calculation cannot be performed according to the positioning image, the inner frame of the screen of the smart device displayed in the screen image of the smart device needs to be searched and extracted from the positioning image in an area connectivity discrimination manner. The specific search mode may be a 'flooding' mode, or a search based on the property that a certain edge of the positioning image is parallel to the inner frame.
Further, the method for extracting the screen of the intelligent device based on the positioning image further comprises the following steps: the extraction server side continuously searches outwards from the inner frame of the screen of the intelligent equipment and extracts the outer frame of the screen of the intelligent equipment displayed in the screen image of the intelligent equipment; and the extraction server extracts the screen image of the intelligent equipment within the range of the outer frame of the screen of the intelligent equipment.
Whether searching and extracting of the intelligent device screen outer frame displayed in the intelligent device screen image and extracting of the intelligent device screen image within the intelligent device screen outer frame range in the scheme are carried out can be selected according to specific business requirements or extraction fineness requirements.
The fineness generally includes both coarse and fine particle sizes. The coarse granularity corresponds to searching and extracting of frames in the screen of the intelligent device displayed in the screen image of the intelligent device, and extracting of the screen image of the intelligent device within the range of the frames in the screen of the intelligent device. And the fine granularity corresponds to searching and extracting of the outer frame of the screen of the intelligent equipment displayed in the screen image of the intelligent equipment, and extracting the screen image of the intelligent equipment within the range of the outer frame of the screen of the intelligent equipment.
Because the outer frames of the screens in various intelligent devices are different in expression form and have no fixed rule to follow, the extraction is usually performed in a machine learning manner, and the extraction is usually performed at a proportional position where the inner frames overflow.
And the outer screen can be omitted for the service which is not sensitive to the outer screen between the inner frame and the outer frame, and the coarse-grained screen image extraction can be directly carried out.
The invention further aims to provide a smart device screen extraction system based on positioning images, which can automatically and conveniently extract smart device screen images within the frame range in the smart device screen. And further extracting the screen image of the intelligent device within the range of the outer frame of the screen of the intelligent device.
Based on the above object, the present invention further provides a system for extracting a screen of an intelligent device based on a positioning image, which executes any one of the above methods for extracting a screen of an intelligent device based on a positioning image.
According to the intelligent equipment screen extraction system based on the positioning image, any intelligent equipment screen extraction method based on the positioning image is executed, so that the intelligent equipment screen image within the frame range in the intelligent equipment screen can be automatically and conveniently extracted. And further extracting the screen image of the intelligent device within the range of the outer frame of the screen of the intelligent device. The method of the present invention has been described in detail, and will not be described herein.
The system generally comprises an intelligent device for acquiring the screen image, an image acquisition and transmission device and an extraction server. The intelligent device of the screen image to be collected executes the step 100, the image collecting and transmitting device executes the step 200, and the extraction server executes the steps 300 to 600.
Still another object of the present invention is to provide a use of a screen image of a smart device, which can effectively utilize the screen image of the smart device extracted by the method of the present invention.
Based on the purpose, the invention also provides the application of the screen image of the intelligent device, and the screen image of the intelligent device extracted by adopting any one of the extraction methods based on the positioning image is used for identifying screen damage or detecting screen display content.
The application of the screen image of the intelligent equipment, which is extracted by adopting any one of the extraction methods based on the positioning image, is used for identifying screen damage or detecting screen display content, so that the screen image of the intelligent equipment extracted by the method is effectively utilized.
The intelligent equipment screen extraction method based on the positioning image has the following advantages and beneficial effects:
(1) the intelligent device screen image within the frame range in the intelligent device screen can be automatically and conveniently extracted, and the intelligent device screen image within the frame range outside the intelligent device screen can also be extracted.
(2) The complexity of screen recognition is greatly reduced, and meanwhile, the original screen information can be efficiently transmitted for the subsequent application for the processing of the subsequent application.
(3) The accuracy of screen image extraction is greatly improved, the interference of unnecessary features to model training is reduced, the amount of processed content information is reduced, and the operating efficiency of the whole automatic process is improved.
(4) By searching the positioning image and then searching the equipment screen according to the positioning image, the difficulty of searching the equipment screen due to different background scenes is avoided.
(5) Through the correction based on the positioning image, the accuracy of screen image extraction is further improved, so that the subsequent application operation can be efficiently carried out.
(6) Some corresponding prompts can be given on the basis of screen image extraction, such as incomplete screen shooting, no positioning image found and other information which is helpful for the user to adjust.
(7) The approximate range of the screen of the equipment is searched from inside to outside by using a positioning image mode, and then an accurate screen position is searched by a machine learning method, so that the influence of the content outside the screen is eliminated, and the input characteristics of training and prediction are provided for subsequent screen content detection and screen fragmentation identification of the screen.
(8) Without loss of generality, in principle, all intelligent devices which are provided with screens and can be connected with the Internet, including but not limited to screen image extraction methods of tablet computers, notebook computers, intelligent watches and the like, are all applicable to the method provided by the invention.
The intelligent equipment screen extraction system based on the positioning image also has the advantages and beneficial effects.
The application of the screen image of the intelligent device also has the advantages and the beneficial effects.
Drawings
Fig. 1 is a schematic flow chart of a method for extracting a screen of an intelligent device based on a positioning image according to an embodiment of the present invention.
Fig. 2 is a flowchart illustrating a working process of the system for extracting a screen of an intelligent device based on a positioning image according to an embodiment of the present invention.
Detailed Description
The method, system and use of smart device screen extraction based on positioning images according to the present invention will be further described in detail with reference to the drawings and specific embodiments, but the detailed description is not to be construed as limiting the invention.
Fig. 1 shows a flow of the method for extracting a screen of an intelligent device based on a positioning image according to an embodiment of the present invention. As shown in fig. 1, the method for extracting a screen of an intelligent device based on a positioning image includes the steps of:
100: displaying a positioning image for determining the position, size and direction of the screen of the intelligent equipment on the screen of the intelligent equipment;
200: acquiring an intelligent equipment screen image containing a positioning image and transmitting the intelligent equipment screen image containing the positioning image to an extraction server;
300: the extraction server side identifies the positioning image to determine the original coordinate of the positioning image in the screen image of the intelligent device;
400: extracting a server side to correct the screen image of the intelligent device based on the original coordinates;
500: the extraction server side searches outwards from the positioning image and extracts an inner frame of the screen of the intelligent equipment displayed in the screen image of the intelligent equipment;
600: and the extraction server extracts the screen image of the intelligent equipment within the frame range in the screen of the intelligent equipment.
In certain embodiments, step 400 further comprises: after the screen image of the intelligent device is corrected, the corrected coordinates of the positioning image in the screen image are determined or acquired.
In some embodiments, the positioning image is a positioning image generated on-the-fly or a positioning image stored in a database.
In some embodiments, the positioning image is a two-dimensional code image.
In some embodiments, the extraction server is a client or a cloud, and the client or the cloud runs an extraction program.
In some embodiments, the correction includes a rotation correction and a projection correction.
In some embodiments, the location image contains identity information, so that when the extraction server receives the smart device screen image containing the location image, it determines whether the smart device screen image is the smart device screen image of the specified smart device.
In some embodiments, in step 500, the extraction server reversely deduces the position of the inner frame of the smart device screen based on the preset size ratio between the positioning image and the smart device screen or the distance between the edge of the positioning image and the inner frame of the smart device screen, so as to search and extract the inner frame of the smart device screen displayed in the screen image of the smart device from the positioning image.
In some embodiments, in step 500, the intra-screen frame of the smart device displayed in the screen image of the smart device is searched and extracted from the positioning image by means of the region connectivity determination.
In certain embodiments, further comprising the step of: the extraction server side continuously searches outwards from the inner frame of the screen of the intelligent equipment and extracts the outer frame of the screen of the intelligent equipment displayed in the screen image of the intelligent equipment; and the extraction server extracts the screen image of the intelligent equipment within the range of the outer frame of the screen of the intelligent equipment.
The intelligent device screen extraction system based on the positioning image comprises a mobile phone A to be detected of the intelligent device used as a screen image to be acquired, a camera mobile phone B used as an image acquisition and transmission device and a cloud C used as an extraction server side, wherein an extraction program runs on the cloud C. And storing a two-dimensional code image D as a positioning image in a database, wherein the two-dimensional code image D is a code of the identity information of the mobile phone A to be detected.
Fig. 2 shows a workflow of a smart device screen extraction system based on a positioning image as an embodiment. As shown in fig. 2, the workflow of the smart device screen extraction system based on positioning images of this embodiment includes:
step 100: the mobile phone A to be tested reads the two-dimensional code image D used for determining the position, size and direction of the screen of the mobile phone A to be tested from the database, and displays the two-dimensional code image D on the screen of the mobile phone A to be tested in a preset size and position according to the size proportional relation between the preset two-dimensional code image D from the cloud C and the screen of the mobile phone A to be tested or a distance instruction of the edge of the two-dimensional code image D from the inner frame of the screen of the mobile phone A to be tested.
Step 200: the camera mobile phone B collects a screen image of the mobile phone A to be detected containing the two-dimension code image D and transmits the screen image of the mobile phone A to be detected containing the two-dimension code image D to the cloud C.
Step 300: the cloud C identifies the two-dimensional code image D through an extraction program running on the cloud C, firstly decodes the two-dimensional code image D and compares the two-dimensional code image D with the identity information of the mobile phone A to be detected so as to judge whether the received screen image is the screen image of the mobile phone A to be detected or not, and then determines the original coordinate of the two-dimensional code image D in the screen image of the mobile phone A to be detected.
In the step, if the positioning image cannot be found, the user is prompted that the positioning image cannot be found, and the user is prompted to adjust the shooting position and angle.
Step 400: and the cloud C corrects the screen image of the mobile phone A to be detected based on the original coordinates through an extraction program running on the cloud C, and the correction comprises rotation correction and projection correction. And after the screen image of the mobile phone A to be detected is corrected, recording the correction coordinate of the two-dimensional code image D.
In this step, the screen image of the mobile phone a to be tested can be corrected according to the mapping relationship between the original coordinates and the expected coordinates. The coordinate system of the desired coordinates is a two-dimensional coordinate system perpendicular to the image capture optical axis of the camera phone B.
Step 500: and the cloud C searches and extracts the inner frame of the screen of the mobile phone A to be detected displayed in the screen image of the mobile phone A to be detected from the two-dimensional code image D through the extraction program running on the cloud C.
In this step, the cloud C reversely deduces the position of the inner frame of the screen of the mobile phone a to be detected based on the preset size proportional relationship between the two-dimensional code image D and the screen of the mobile phone a to be detected or the distance between the edge of the two-dimensional code image D and the inner frame of the screen of the mobile phone a to be detected, so as to search and extract the inner frame of the screen of the mobile phone a to be detected displayed in the screen image of the mobile phone a to be detected from the two-dimensional code image D. For example, if the instruction requires that the two-dimensional code is generated at the center of the screen of the mobile phone a to be tested, and the distance between the upper side and the left side of the screen of the mobile phone a to be tested has a preset proportional relationship, the inner frame of the screen of the mobile phone a to be tested is directly searched and extracted according to the proportional relationship.
Step 600: and extracting a coarse-grained screen image. And the cloud C extracts the screen image of the mobile phone A to be detected within the frame range in the screen of the mobile phone A to be detected through the extraction program running on the cloud C.
Step 700: and extracting the fine-grained screen image. The cloud C continuously searches outwards from the inner frame of the screen of the mobile phone A to be detected and extracts the outer frame of the screen of the mobile phone A to be detected displayed in the screen image of the mobile phone A to be detected; and the cloud C extracts the screen image of the mobile phone A to be detected within the range of the outer frame of the screen of the mobile phone A to be detected.
In the step, a proportion position overflowing from the inner frame is extracted in a machine learning mode.
The screen image of the mobile phone a to be detected extracted by the positioning image-based intelligent device screen extraction system and method in the above embodiments can be used for identifying the screen damage of the mobile phone a to be detected or detecting the screen display content of the mobile phone a to be detected. The method comprises the following specific steps:
step 800: and (3) performing characteristic processing, such as resizing (resize), reshaping (reshape) and marginalization, on the screen image of the mobile phone A to be detected within the range of the outer frame of the screen of the mobile phone A to be detected extracted in the step 700, and using the processed screen image to train and predict a broken screen recognition model.
Step 900: and predicting the broken screen of the mobile phone A to be tested by using the broken screen identification model and updating and training the parameters of the broken screen identification model.
It should be noted that the combination of the features in the present application is not limited to the combination described in the claims or the combination described in the embodiments, and all the features described in the present application may be freely combined or combined in any manner unless contradictory to each other.
It should be noted that the above-mentioned embodiments are only specific examples of the present invention, and obviously, the present invention is not limited to the above-mentioned embodiments, and many similar variations exist. All modifications which would occur to one skilled in the art and which are, therefore, directly derived or suggested from the disclosure herein are deemed to be within the scope of the present invention.
Claims (10)
1. A method for extracting a screen of an intelligent device based on a positioning image is characterized by comprising the following steps:
100: displaying a positioning image for determining the position, size and direction of the screen of the intelligent equipment on the screen of the intelligent equipment;
200: acquiring an intelligent equipment screen image containing a positioning image and transmitting the intelligent equipment screen image containing the positioning image to an extraction server;
300: the extraction server side identifies the positioning image to determine the original coordinate of the positioning image in the screen image of the intelligent device;
400: the extraction server corrects the screen image of the intelligent device on the basis of the original coordinates;
500: the extraction server side searches and extracts an inner frame of the screen of the intelligent equipment displayed in the screen image of the intelligent equipment from the positioning image; in step 500, the extracting server reversely deduces the position of the inner frame of the screen of the smart device based on the preset size proportional relationship between the positioning image and the screen of the smart device, or reversely deduces the position of the inner frame of the screen of the smart device based on the distance between the edge of the positioning image and the inner frame of the screen of the smart device, so as to search and extract the inner frame of the screen of the smart device displayed in the screen image of the smart device from the positioning image; or in the step 500, the inner frame of the screen of the intelligent device displayed in the screen image of the intelligent device is searched and extracted from the positioning image in a region communication judging mode;
600: and the extraction server extracts the screen image of the intelligent device within the frame range in the screen of the intelligent device.
2. The method for extracting a screen of an intelligent device based on a positioning image according to claim 1, wherein the step 400 further comprises: after the screen image of the intelligent device is corrected, the correction coordinates of the positioning image in the screen image are determined.
3. The method for extracting a screen of an intelligent device based on a positioning image as claimed in claim 1, wherein the positioning image is a positioning image generated instantly or a positioning image stored in a database.
4. The intelligent device screen extraction method based on positioning images as claimed in claim 1, wherein the positioning images are two-dimensional code images.
5. The method for extracting a screen of a smart device based on a positioning image as claimed in claim 1, wherein the extracting server is a client or a cloud, and an extracting program runs on the client or the cloud.
6. The method for smart device screen extraction based on scout images of claim 1, wherein the correction comprises a rotation correction and a projection correction.
7. The method for extracting a screen of an intelligent device based on a positioning image as claimed in claim 1, wherein the positioning image contains identity information, so that when the extraction server receives the screen image of the intelligent device containing the positioning image, the extraction server determines whether the screen image of the intelligent device is the screen image of the intelligent device specified.
8. The method for extracting the screen of the intelligent device based on the positioning image as claimed in any one of claims 1 to 7, further comprising the steps of: the extraction server side continuously searches outwards from the inner frame of the screen of the intelligent equipment and extracts the outer frame of the screen of the intelligent equipment displayed in the screen image of the intelligent equipment; and the extraction server extracts the screen image of the intelligent equipment within the range of the outer frame of the screen of the intelligent equipment.
9. A smart device screen extraction system based on positioning images, which executes the method of any one of claims 1-8.
10. Use of a screen image of a smart device, wherein the screen image of the smart device extracted by the extraction method based on the positioning image according to any one of claims 1 to 8 is used for identifying screen breakage or detecting screen display content.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710731243.6A CN107491778B (en) | 2017-08-23 | 2017-08-23 | Intelligent device screen extraction method and system based on positioning image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710731243.6A CN107491778B (en) | 2017-08-23 | 2017-08-23 | Intelligent device screen extraction method and system based on positioning image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107491778A CN107491778A (en) | 2017-12-19 |
| CN107491778B true CN107491778B (en) | 2020-07-03 |
Family
ID=60650837
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710731243.6A Active CN107491778B (en) | 2017-08-23 | 2017-08-23 | Intelligent device screen extraction method and system based on positioning image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107491778B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110245668B (en) * | 2018-03-09 | 2023-06-27 | 腾讯科技（深圳）有限公司 | Terminal information acquisition method, acquisition device and storage medium based on image recognition |
| FR3086436B1 (en) * | 2018-09-21 | 2021-01-22 | Diotasoft | DEVICE, SYSTEM AND METHOD FOR LOCATING, BY A PROCESSING MODULE, AN ACQUISITION MODULE IN RELATION TO AN EQUIPMENT TO BE CONTROLLED |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| FR2833743B1 (en) * | 2001-12-17 | 2004-02-20 | Eldim | LOW ACQUISITION RESOLUTION PROCESS AND DEVICE FOR THE CONTROL OF A DISPLAY SCREEN |
| CN104990689B (en) * | 2015-06-09 | 2017-11-10 | 青岛海信移动通信技术股份有限公司 | Localization method and device in a kind of optic test |
| CN105301810A (en) * | 2015-11-24 | 2016-02-03 | 上海斐讯数据通信技术有限公司 | Screen defect detecting method and screen defect detecting device |
| CN106060126B (en) * | 2016-05-24 | 2019-03-19 | 众安在线财产保险股份有限公司 | A kind of the remote auto recognition methods and system of screen of intelligent device state |
| CN106228541A (en) * | 2016-07-11 | 2016-12-14 | 广州视源电子科技股份有限公司 | The method and device of screen location in vision-based detection |
-
2017
- 2017-08-23 CN CN201710731243.6A patent/CN107491778B/en active Active
Non-Patent Citations (3)
| Title |
|---|
| 《图像中二维码的检测和定位》;Java与Android技术栈;《https://blog.csdn.net/SLFq6OF5O7aH/article/details/78141333》;20170820;全文 * |
| 《基于内容的图像检索技术在手机外形特征检索系统中的应用研究》;吴文超 等;《计算机应用与软件》;20091231;第26卷(第12期);全文 * |
| 《手机屏幕图像缺陷的实时检测》;刘波 等;《计算机工程与应用》;20101231;第46卷(第28期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107491778A (en) | 2017-12-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3152706B1 (en) | Image capturing parameter adjustment in preview mode | |
| EP3358298B1 (en) | Building height calculation method and apparatus, and storage medium | |
| CN101667251B (en) | OCR recognition method and device with auxiliary positioning function | |
| US10943140B2 (en) | Image processing to detect a rectangular object | |
| US9058536B1 (en) | Image-based character recognition | |
| US20080002893A1 (en) | Recognizing text in images | |
| WO2016187888A1 (en) | Keyword notification method and device based on character recognition, and computer program product | |
| EP2973222A1 (en) | Local image enhancement for text recognition | |
| CN107710280B (en) | Object visualization method | |
| US10609293B2 (en) | Real-time glare detection inside a dynamic region of an image | |
| EP2041678A2 (en) | Recognizing text in images | |
| EP2660753A2 (en) | Image processing method and apparatus | |
| US20220130161A1 (en) | Dynamically optimizing photo capture for multiple subjects | |
| CN110463177A (en) | The bearing calibration of file and picture and device | |
| CN107491778B (en) | Intelligent device screen extraction method and system based on positioning image | |
| US10699145B1 (en) | Systems and methods for augmented reality assisted form data capture | |
| CN115546219A (en) | Detection board type generation method, board card defect detection method, device and product | |
| CN111368726B (en) | Construction site operation face personnel number statistics method, system, storage medium and device | |
| WO2017219562A1 (en) | Method and apparatus for generating two-dimensional code | |
| US20230188833A1 (en) | Dynamic minimum focus threshold determination inside a region of interest in an image | |
| JP6833324B2 (en) | Information processing equipment, programs and information processing methods | |
| CN113139144A (en) | Method, device and equipment for acquiring resource picture | |
| KR20220154556A (en) | Photography system for surpporting to picture for mobile terminal and method thereof | |
| CN117132648A (en) | Visual positioning method, electronic equipment and computer readable storage medium | |
| CN117765439A (en) | Target object detection method, vehicle control method, device, chip and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240227 Address after: Room 1179, W Zone, 11th Floor, Building 1, No. 158 Shuanglian Road, Qingpu District, Shanghai, 201702 Patentee after: Shanghai Zhongan Information Technology Service Co.,Ltd. Country or region after: China Address before: 518000 Room 201, building A, No. 1, Qian Wan Road, Qianhai Shenzhen Hong Kong cooperation zone, Shenzhen, Guangdong (Shenzhen Qianhai business secretary Co., Ltd.) Patentee before: ZHONGAN INFORMATION TECHNOLOGY SERVICE Co.,Ltd. Country or region before: China |