CN109886324B - Icon identification method and device - Google Patents
Icon identification method and device Download PDFInfo
- Publication number
- CN109886324B CN109886324B CN201910105196.3A CN201910105196A CN109886324B CN 109886324 B CN109886324 B CN 109886324B CN 201910105196 A CN201910105196 A CN 201910105196A CN 109886324 B CN109886324 B CN 109886324B
- Authority
- CN
- China
- Prior art keywords
- icon
- feature vector
- database
- type
- recognition model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses an icon identification method, which comprises the following steps: acquiring a first icon to be identified; extracting a first feature vector of the first icon based on a pre-trained icon recognition model; querying whether a pre-established database has a target characteristic vector, wherein the target characteristic vector is matched with a first characteristic vector of the first icon; if not, determining first labeling information of the first icon, wherein the first labeling information is used for describing the icon type of the first icon; optimizing the icon recognition model based on the first icon and the icon type of the first icon, so that a second icon is recognized based on the optimized icon recognition model. By adopting the embodiment of the invention, the flexibility of icon identification can be improved.
Description
Technical Field
The invention relates to the field of terminals, in particular to an icon identification method and device.
Background
With the development of communication technology, mobile terminals are increasingly popularized, application programs on the mobile terminals are more and more colorful, icons of the application programs are more diversified, and the icons of the application programs are generally recognized by an icon recognition model based on a machine learning algorithm.
However, due to the diversity of icons, during model training, icons of all styles cannot be collected in a database of the model in advance, so in the icon recognition process, icons which are not collected in the database are encountered, and the icons cannot be recognized.
Therefore, a more reliable icon recognition scheme is needed.
Disclosure of Invention
The embodiment of the invention provides an icon identification method, which aims to solve the problem that an icon cannot be identified.
In order to solve the technical problem, the invention is realized as follows:
in a first aspect, an icon identification method is provided, and the method includes:
acquiring a first icon to be identified;
extracting a first feature vector of the first icon based on a pre-trained icon recognition model;
querying whether a pre-established database has a target characteristic vector, wherein the target characteristic vector is matched with a first characteristic vector of the first icon;
if not, determining first labeling information of the first icon, wherein the first labeling information is used for describing the icon type of the first icon;
optimizing the icon recognition model based on the first icon and the icon type of the first icon, so that a second icon is recognized based on the optimized icon recognition model.
In a second aspect, an icon recognition apparatus is provided, the apparatus comprising:
the acquisition module is used for acquiring a first icon to be identified;
the extraction module is used for extracting a first feature vector of the first icon based on a pre-trained icon recognition model;
the query module is used for querying whether a target characteristic vector exists in a pre-established database, and the target characteristic vector is matched with the first characteristic vector of the first icon;
a first determining module, configured to determine first annotation information of the first icon if the first icon is not the first icon, where the first annotation information is used to describe an icon type of the first icon;
and the optimization module is used for optimizing the icon recognition model based on the first icon and the icon type of the first icon so as to recognize a second icon based on the optimized icon recognition model.
In a third aspect, a terminal device is provided, which includes: a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the steps of the method according to the first aspect.
In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the method according to the first aspect.
In the embodiment of the invention, a first icon to be identified is obtained; extracting a first feature vector of the first icon based on a pre-trained icon recognition model; querying whether a pre-established database has a target characteristic vector, wherein the target characteristic vector is matched with a first characteristic vector of the first icon; if not, determining first labeling information of the first icon, wherein the first labeling information is used for describing the icon type of the first icon; and optimizing the icon recognition model based on the first icon and the icon type of the first icon so as to recognize the second icon based on the optimized icon recognition model, so that the updating and optimizing speed of the icon recognition model can be increased, and the flexibility of icon recognition is improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the invention and not to limit the invention. In the drawings:
FIG. 1 is a schematic diagram of an application scenario provided by an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a method for identifying icons according to an embodiment of the present invention;
fig. 3 is a schematic diagram of icon recognition through a VGG icon recognition model according to an embodiment of the present invention;
FIG. 4 is a flowchart illustrating a method for identifying icons according to another embodiment of the present invention;
FIG. 5 is a schematic structural diagram of an icon recognition apparatus according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of a terminal device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
An application scenario of the present invention is exemplarily illustrated with reference to fig. 1.
The application scene comprises the following steps: the testing method comprises a testing platform and a terminal device 102 bearing a testing object, wherein an application program 104 is installed on the terminal device 102, a first icon 106 and a second icon 108 are displayed on a user interface of the application program 104, and:
in the automatic testing process, the testing platform acquires an interface image of the user interface, identifies the icon in the interface image by adopting an icon identification module, and executes a corresponding testing step on the first icon 106 if the icon type of the first icon 106 is identified; if the first icon 106 is not identified, the icon type of the first icon 106 is labeled in a human or machine labeling manner, and the relevant testing step is executed on the first icon.
Further, when the icon type of the first icon is obtained, the test platform may optimize the icon recognition model based on the first icon and the icon type thereof, and recognize the next icon (e.g., the second icon 108) based on the optimized icon recognition model.
Therefore, the icon can be continuously identified and the icon identification module can be continuously optimized in the automatic test process, and the purpose of continuously optimizing the icon identification accuracy is achieved.
The automatic test generally refers to the automation of software test, the software test is to run a system or an application program under a preset condition, and evaluate a running result, wherein the preset condition comprises a normal condition and an abnormal condition; the icon type may be return, search, close, etc.
The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.
Fig. 2 is a schematic flowchart of an icon recognition method according to an embodiment of the present invention, where the method is executed by a test platform in an application scenario corresponding to fig. 1 based on an icon recognition model, and referring to fig. 2, the method may specifically include the following steps:
The first icon may be an icon to be recognized in an automated testing process, and may be an icon of an application program displayed on the terminal device, for example: camera icon, QQ icon, or an icon displayed on a certain user interface of an application, for example: a search icon, a return icon, a close icon, etc. on the camera browsing interface.
And 204, extracting a first feature vector of the first icon based on the pre-trained icon recognition model.
The icon identification model may be a VGG (Visual Geometry Group Network) icon identification model, or may be a CNN (Convolutional Neural Networks) icon identification model or an RNN (Recurrent Neural Networks) icon identification model; the pre-trained icon recognition model is obtained by pre-collecting a preset number of icons and training corresponding icon types; the icon recognition model comprises a plurality of layers of networks, and the first characteristic vector is a vector corresponding to the last layer of networks of the icon recognition model.
Fig. 3 is a schematic diagram of icon identification through a VGG icon identification model, and as shown in fig. 3, an icon to be identified "<" is obtained, and after passing through a multilayer network of the VGG model, classification probabilities of various icon types can be obtained, for example, the probability that the icon to be identified is a return (Back) icon is 89%, the probability that the icon to be identified is a Close (Close) icon is 8%, the probability that the icon to be identified is a Search (Search) icon is 1%, and the probability that the icon to be identified is Other (Other) icon is 0.1%, then the icon to be identified is a return (Back) icon; and extracting a vector corresponding to the last layer of network of the VGG icon recognition model, and taking the vector as a feature vector of the icon "<".
Based on this, the accuracy of icon identification is improved, and the icon identification is more convenient and faster.
It should be noted that, one implementation manner of step 206 may be:
step S1, determining the similarity between each feature vector in the database and the first feature vector;
the implementation manner of step S1 may be specifically exemplified as:
and determining the similarity between each feature vector in the database and the first feature vector by adopting a similarity algorithm, wherein the similarity algorithm can be a least square method.
And step S2, determining whether the target characteristic vector exists in the database according to the similarity.
Based on this, the determination of whether the target feature vector exists can be made more accurate.
Further, one implementation manner of step S2 may be:
if the feature vector with the similarity larger than or equal to a preset similarity threshold exists, determining that a target feature vector exists in the database; otherwise, determining that the target feature vector does not exist;
the target feature vector is a feature vector of which the similarity with the first feature vector in the database is greater than or equal to a preset similarity threshold.
The preset similarity threshold may be 90%, or 85%, and the target feature vector may be one or multiple.
Based on the method, the similarity of each feature vector in the database is compared with the preset similarity threshold value to determine whether the target feature vector exists in the database, so that the determination of whether the target feature vector exists can be more accurate.
Optionally, in another embodiment of the present invention, a corresponding relationship between the feature vector and the icon type is stored in the database, and the icon identification method further includes:
and step S1, determining the icon type corresponding to the feature vector with the highest similarity in the target feature vectors, and taking the icon type as the icon type of the first icon.
If the similarity of the target feature vector can be 91%, 98% and 95%, respectively, determining the icon type corresponding to the target feature vector with the similarity of 98% as the icon type of the first icon.
Based on the method, when the target characteristic vector exists in the database, the determined icon type of the first icon is more accurate.
And 208, if not, determining first labeling information of the first icon, wherein the first labeling information is used for describing the icon type of the first icon.
The implementation of step 208 may specifically be exemplified by:
if the target characteristic vector does not exist in the database, namely the similarity between each characteristic vector in the database and the first characteristic vector is lower than a preset similarity threshold, the icon type of the first icon is obtained according to the determined first marking information.
Optionally, in another embodiment of the present invention, the icon identification manner further includes:
step S1, when it is determined that the target feature vector does not exist in the database, determining second labeling information of the first icon, where the second labeling information is used to label the first feature vector as a temporary feature vector.
Based on this, the first feature vector in which the target feature vector does not exist can be distinguished from the other feature vectors.
Optionally, in another embodiment of the present invention, the icon identification method further includes:
and step S1, storing the first characteristic vector and the icon type of the first icon into the database for updating the database.
One implementation manner of step S1 may be specifically exemplified by:
and correspondingly storing the icon type of the first icon extracted from the first labeling information and the first characteristic vector labeled as the temporary characteristic vector into a database so as to update the database.
Based on the method, the icon types and the feature vectors in the database can be richer, and the icon identification can be more accurate and convenient.
One implementation of step 210 may specifically be as follows:
and training the icon recognition model according to the first icon and the icon type of the first icon, namely optimizing the icon recognition model to obtain an optimized icon recognition model, and recognizing the second icon according to the optimized icon recognition model.
Optionally, in another embodiment of the present invention, the icon identification method further includes:
step S1, extracting a second feature vector of the first icon based on the optimized icon recognition model;
and step S2, replacing the first feature vector in the database with the second feature vector.
The first characteristic vector is extracted from the icon recognition model when the first icon and the icon type of the first icon are not used for training the icon model; and the second characteristic vector is extracted from the optimized icon recognition model after the icon model is trained by using the first icon and the icon type of the first icon.
Based on this, the second feature vector replaces the first feature vector, so that the feature vector stored in the database is more accurate, and the icon identification is more accurate and convenient.
Further, one implementation manner of step S2 may be:
determining a first feature vector in the database labeled as a provisional feature vector, and replacing the first feature vector with the second feature vector.
Based on this, the first feature vector marked as the temporary feature vector is replaced by the second feature vector, so that the replacement of the feature vector is more accurate, the feature vector stored in the database is more accurate, and the icon identification is more accurate and convenient.
For the embodiment corresponding to fig. 2, the icon recognition model is optimized based on the first icon and the icon type of the first icon when the target feature vector matching the first feature vector of the first icon does not exist in the database, so that a second icon is recognized based on the optimized icon recognition model; the optimization updating speed of the icon recognition model can be improved, so that the icon recognition is more accurate, convenient and flexible.
Fig. 4 is a schematic flowchart of an icon recognition method according to another embodiment of the present invention, where the method may be executed by the test platform in the application scenario corresponding to fig. 1 based on an icon recognition model, and referring to fig. 4, the method may specifically include the following steps:
The first icon may be an icon to be recognized in an automated testing process, and may be an icon of an application program displayed by the terminal device, for example: camera icon, QQ icon, or an icon displayed on a certain user interface of an application, for example: a search icon, a return icon, a close icon, etc. on the camera browsing interface.
Based on the method, whether the target characteristic vector exists in the database or not is determined through comparison between the similarity and a preset similarity threshold, so that the accuracy of the determination process is improved; the first characteristic vector and the first icon type are stored in the database, so that the characteristic vector and the icon type in the database are richer; by storing the corresponding relation between the characteristic vector and the icon type in the database, the icon type corresponding to the characteristic vector with the highest similarity can be used as the icon type of the first icon, so that the accuracy of the first icon type is improved; the icon recognition model is optimized based on the first icon and the first icon type, so that the updating speed of the icon recognition model is increased, and the icon recognition process is more flexible; by replacing the first feature vector labeled as the temporary feature vector in the database with the second feature vector, the accuracy of the feature vector of the first icon is improved, and thus the accuracy of icon identification is improved.
In addition, for simplicity of explanation, the above-described method embodiments are described as a series of acts or combinations, but it should be understood by those skilled in the art that the present invention is not limited by the order of acts or steps described, as some steps may be performed in other orders or simultaneously according to the present invention. Furthermore, those skilled in the art will appreciate that the embodiments described in the specification are presently preferred and that no particular act is required to implement the invention.
Fig. 5 is a schematic structural diagram of an icon identifying apparatus according to an embodiment of the present invention, and referring to fig. 5, the apparatus may specifically include: an obtaining module 502, an extracting module 504, a querying module 506, a first determining module 508, and an optimizing module 510, wherein:
an obtaining module 502, configured to obtain a first icon to be identified;
an extraction module 504, configured to extract a first feature vector of the first icon based on a pre-trained icon recognition model;
a query module 506, configured to query whether a target feature vector exists in a pre-established database, where the target feature vector is matched with a first feature vector of the first icon;
a first determining module 508, configured to determine, if the first icon is not a first icon, first annotation information of the first icon, where the first annotation information is used to describe an icon type of the first icon;
an optimizing module 510, configured to optimize the icon recognition model based on the first icon and the icon type of the first icon, so as to identify a second icon based on the optimized icon recognition model.
For the embodiment corresponding to fig. 5, when there is no target feature vector in the database, the icon recognition model is optimized based on the first icon and the icon type of the first icon, so that the second icon is recognized based on the optimized icon recognition model, the update iteration of the icon recognition model can be fast, and the flexibility of icon recognition can be improved.
Optionally, the query module includes:
the query unit is used for determining the similarity between each feature vector in the database and the first feature vector;
and determining whether a target feature vector exists in the database or not according to the similarity.
Optionally, the query unit includes:
the query subunit is used for determining that a target feature vector exists in the database if the feature vector with the similarity greater than or equal to a preset similarity threshold exists; otherwise, determining that the target feature vector does not exist;
the target feature vector is a feature vector with the similarity greater than or equal to a preset similarity threshold.
Optionally, the apparatus further comprises:
and the second determining module is used for determining the icon type corresponding to the feature vector with the highest similarity in the target feature vectors, and taking the icon type as the icon type of the first icon.
Optionally, the apparatus further comprises:
and the storage module is used for storing the first characteristic vector and the icon type of the first icon into the database so as to update the database.
Optionally, the apparatus further comprises:
the replacing module is used for extracting a second characteristic vector of the first icon based on the optimized icon recognition model;
replacing the first feature vector in the database with the second feature vector.
Optionally, the apparatus further comprises:
a third determining module, configured to determine, when it is determined that a target feature vector does not exist in the database, second tagging information of the first icon, where the second tagging information is used to tag the first feature vector as a temporary feature vector;
replacing the module, comprising:
a replacement unit, configured to determine a first feature vector labeled as a temporary feature vector in the database, and replace the first feature vector with the second feature vector.
The device provided by the embodiment of the present invention can implement each process implemented by the device in the method embodiments of fig. 2 to fig. 4, and is not described herein again to avoid repetition. Further, it should be noted that, among the respective components of the apparatus of the present invention, the components thereof are logically divided according to the functions to be realized, but the present invention is not limited thereto, and the respective components may be newly divided or combined as necessary.
Figure 6 is a schematic diagram of a hardware structure of a terminal device implementing various embodiments of the present invention,
the terminal device 600 includes but is not limited to: a radio frequency unit 601, a network module 602, an audio output unit 603, an input unit 604, a sensor 605, a display unit 606, a user input unit 607, an interface unit 608, a memory 609, a processor 610, and a power supply 611. Those skilled in the art will appreciate that the terminal device configuration shown in fig. 6 does not constitute a limitation of the terminal device, and that the terminal device may include more or fewer components than shown, or combine certain components, or a different arrangement of components. In the embodiment of the present invention, the terminal device includes, but is not limited to, a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted terminal, a wearable device, a pedometer, and the like.
The radio frequency unit 601 is configured to obtain a first icon to be identified;
a processor 610 for extracting a first feature vector of the first icon based on a pre-trained icon recognition model;
querying whether a pre-established database has a target characteristic vector, wherein the target characteristic vector is matched with a first characteristic vector of the first icon;
if not, determining first labeling information of the first icon, wherein the first labeling information is used for describing the icon type of the first icon;
optimizing the icon recognition model based on the first icon and the icon type of the first icon, so that a second icon is recognized based on the optimized icon recognition model.
Obtaining a first icon to be identified; extracting a first feature vector of the first icon based on a pre-trained icon recognition model; querying whether a pre-established database has a target characteristic vector, wherein the target characteristic vector is matched with a first characteristic vector of the first icon; if not, determining first labeling information of the first icon, wherein the first labeling information is used for describing the icon type of the first icon; optimizing the icon recognition model based on the first icon and the icon type of the first icon, so as to recognize a second icon based on the optimized icon recognition model; the flexibility of icon recognition can be improved.
It should be understood that, in the embodiment of the present invention, the radio frequency unit 601 may be used for receiving and sending signals during a message sending and receiving process or a call process, and specifically, receives downlink data from a base station and then processes the received downlink data to the processor 610; in addition, the uplink data is transmitted to the base station. In general, radio frequency unit 601 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. Further, the radio frequency unit 601 may also communicate with a network and other devices through a wireless communication system.
The terminal device provides the user with wireless broadband internet access through the network module 602, such as helping the user send and receive e-mails, browse webpages, access streaming media, and the like.
The audio output unit 603 may convert audio data received by the radio frequency unit 601 or the network module 602 or stored in the memory 609 into an audio signal and output as sound. Also, the audio output unit 603 can also provide audio output related to a specific function performed by the terminal apparatus 600 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 603 includes a speaker, a buzzer, a receiver, and the like.
The input unit 604 is used to receive audio or video signals. The input Unit 604 may include a Graphics Processing Unit (GPU) 6041 and a microphone 6042, and the Graphics processor 6041 processes image data of a still picture or video obtained by an image capturing apparatus (such as a camera) in a video capture mode or an image capture mode. The processed image frames may be displayed on the display unit 606. The image frames processed by the graphic processor 6041 may be stored in the memory 609 (or other storage medium) or transmitted via the radio frequency unit 601 or the network module 602. The microphone 6042 can receive sound, and can process such sound into audio data. The processed audio data may be converted into a format output transmittable to a mobile communication base station via the radio frequency unit 601 in case of the phone call mode.
The terminal device 600 further comprises at least one sensor 605, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor that can adjust the luminance of the display panel 6061 according to the brightness of ambient light, and a proximity sensor that can turn off the display panel 6061 and/or the backlight when the terminal apparatus 600 is moved to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), detect the magnitude and direction of gravity when stationary, and can be used to identify the terminal device posture (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration identification related functions (such as pedometer, tapping), and the like; the sensors 605 may also include fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., which are not described in detail herein.
The display unit 606 is used to display information input by the user or information provided to the user. The Display unit 606 may include a Display panel 6061, and the Display panel 6061 may be configured by a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like.
The user input unit 607 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the terminal device. Specifically, the user input unit 607 includes a touch panel 6071 and other input devices 6072. Touch panel 6071, also referred to as a touch screen, may collect touch operations by a user on or near it (e.g., operations by a user on or near touch panel 6071 using a finger, stylus, or any suitable object or accessory). The touch panel 6071 may include two parts of a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 610, receives a command from the processor 610, and executes the command. In addition, the touch panel 6071 can be implemented by various types such as a resistive type, a capacitive type, an infrared ray, and a surface acoustic wave. The user input unit 607 may include other input devices 6072 in addition to the touch panel 6071. Specifically, the other input devices 6072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a track ball, a mouse, and a joystick, which are not described herein again.
Further, the touch panel 6071 can be overlaid on the display panel 6061, and when the touch panel 6071 detects a touch operation on or near the touch panel 6071, the touch operation is transmitted to the processor 610 to determine the type of the touch event, and then the processor 610 provides a corresponding visual output on the display panel 6061 according to the type of the touch event. Although in fig. 6, the touch panel 6071 and the display panel 6061 are two independent components to implement the input and output functions of the terminal device, in some embodiments, the touch panel 6071 and the display panel 6061 may be integrated to implement the input and output functions of the terminal device, and this is not limited here.
The interface unit 608 is an interface for connecting an external device to the terminal apparatus 600. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The interface unit 608 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the terminal apparatus 600 or may be used to transmit data between the terminal apparatus 600 and an external device.
The memory 609 may be used to store software programs as well as various data. The memory 609 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. Further, the memory 609 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.
The processor 610 is a control center of the terminal device, connects various parts of the entire terminal device by using various interfaces and lines, and performs various functions of the terminal device and processes data by running or executing software programs and/or modules stored in the memory 609 and calling data stored in the memory 609, thereby performing overall monitoring of the terminal device. Processor 610 may include one or more processing units; preferably, the processor 610 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 610.
The terminal device 600 may further include a power supply 611 (such as a battery) for supplying power to various components, and preferably, the power supply 611 may be logically connected to the processor 610 through a power management system, so as to implement functions of managing charging, discharging, and power consumption through the power management system.
In addition, the terminal device 600 includes some functional modules that are not shown, and are not described in detail herein.
Preferably, an embodiment of the present invention further provides a terminal device, which includes a processor 610, a memory 609, and a computer program stored in the memory 609 and capable of running on the processor 610, where the computer program, when executed by the processor 610, implements each process of the above icon identification method embodiment, and can achieve the same technical effect, and in order to avoid repetition, details are not described here again.
The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements each process of the embodiment of the icon identification method, and can achieve the same technical effect, and in order to avoid repetition, the detailed description is omitted here. The computer-readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.
While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (5)
1. An icon identification method is used in an application scene, and the application scene comprises the following steps: the test platform and the terminal equipment who bears the test object install application on the terminal equipment, show first icon and second icon on application's the user interface, wherein: in the automatic testing process, the testing platform acquires an interface image of the user interface, identifies the icon in the interface image by adopting an icon identification module, and executes a corresponding testing step on the first icon if the icon type of the first icon is identified; if the first icon is not identified, marking the icon type of the first icon in a man-made or machine marking mode, and executing a related test step on the first icon; it is characterized by comprising:
acquiring a first icon to be identified;
extracting a first feature vector of the first icon based on a pre-trained icon recognition model; the pre-trained icon recognition model is obtained by pre-collecting a preset number of icons and training corresponding icon types; the icon identification model comprises a plurality of layers of networks, and the first characteristic vector is a vector corresponding to the last layer of network of the icon identification model;
determining the similarity between each feature vector in a database and the first feature vector;
if the feature vector with the similarity larger than or equal to a preset similarity threshold exists, determining that a target feature vector exists in the database; otherwise, determining that the target feature vector does not exist; the target feature vector is a feature vector with the similarity greater than or equal to a preset similarity threshold;
if the target characteristic vector does not exist, determining first labeling information of the first icon, wherein the first labeling information is used for describing the icon type of the first icon;
when it is determined that the target feature vector does not exist in the database, determining second labeling information of the first icon, wherein the second labeling information is used for labeling the first feature vector as a temporary feature vector;
storing the first feature vector and the icon type of the first icon into the database for updating the database;
optimizing the icon recognition model based on the first icon and the icon type of the first icon, so as to recognize a second icon based on the optimized icon recognition model;
extracting a second feature vector of the first icon based on the optimized icon recognition model;
determining a first feature vector labeled as a temporary feature vector in the database, and replacing the first feature vector with the second feature vector;
the first characteristic vector is extracted from the icon recognition model when the first icon and the icon type of the first icon are not used for training the icon model; and the second characteristic vector is extracted from the optimized icon recognition model after the icon model is trained by using the first icon and the icon type of the first icon.
2. The method according to claim 1, wherein the database stores the corresponding relationship between the feature vector and the icon type; the method further comprises the following steps:
and determining the icon type corresponding to the feature vector with the highest similarity in the target feature vectors, and taking the icon type as the icon type of the first icon.
3. An icon recognition device is used in an application scene, wherein the application scene comprises: the test platform and the terminal equipment who bears the test object install application on the terminal equipment, show first icon and second icon on application's the user interface, wherein: in the automatic testing process, the testing platform acquires an interface image of the user interface, identifies the icon in the interface image by adopting an icon identification module, and executes a corresponding testing step on the first icon if the icon type of the first icon is identified; if the first icon is not identified, marking the icon type of the first icon in a man-made or machine marking mode, and executing a related test step on the first icon; it is characterized by comprising:
the acquisition module is used for acquiring a first icon to be identified;
the extraction module is used for extracting a first feature vector of the first icon based on a pre-trained icon recognition model; the pre-trained icon recognition model is obtained by pre-collecting a preset number of icons and training corresponding icon types; the icon identification model comprises a plurality of layers of networks, and the first characteristic vector is a vector corresponding to the last layer of network of the icon identification model;
the query module is used for querying whether a target characteristic vector exists in a pre-established database, and the target characteristic vector is matched with the first characteristic vector of the first icon;
a first determining module, configured to determine first annotation information of the first icon if the first icon is not the first icon, where the first annotation information is used to describe an icon type of the first icon;
the optimization module is used for optimizing the icon recognition model based on the first icon and the icon type of the first icon so as to recognize a second icon based on the optimized icon recognition model;
wherein, the inquiry module includes:
the query unit is used for determining the similarity between each feature vector in the database and the first feature vector; determining whether a target characteristic vector exists in the database or not according to the similarity;
the query subunit is used for determining that a target feature vector exists in the database if the feature vector with the similarity greater than or equal to a preset similarity threshold exists; otherwise, determining that the target feature vector does not exist;
the target feature vector is a feature vector with the similarity greater than or equal to a preset similarity threshold;
the device also includes:
the second determining module is used for determining the icon type corresponding to the feature vector with the highest similarity in the target feature vectors, and taking the icon type as the icon type of the first icon;
the storage module is used for storing the first characteristic vector and the icon type of the first icon into the database so as to update the database;
the replacing module is used for extracting a second characteristic vector of the first icon based on the optimized icon recognition model;
replacing the first feature vector in the database with the second feature vector;
the first characteristic vector is extracted from the icon recognition model when the first icon and the icon type of the first icon are not used for training the icon model; the second characteristic vector is extracted from the optimized icon recognition model after the icon model is trained by using the first icon and the icon type of the first icon;
a third determining module, configured to determine, when it is determined that a target feature vector does not exist in the database, second tagging information of the first icon, where the second tagging information is used to tag the first feature vector as a temporary feature vector;
a replacement module comprising:
a replacement unit, configured to determine a first feature vector labeled as a temporary feature vector in the database, and replace the first feature vector with the second feature vector.
4. A terminal device, comprising: memory, processor and computer program stored on the memory and executable on the processor, which computer program, when being executed by the processor, carries out the steps of the method as claimed in claim 1 or 2.
5. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, carries out the steps of the method as set forth in claim 1 or 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910105196.3A CN109886324B (en) | 2019-02-01 | 2019-02-01 | Icon identification method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910105196.3A CN109886324B (en) | 2019-02-01 | 2019-02-01 | Icon identification method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109886324A CN109886324A (en) | 2019-06-14 |
| CN109886324B true CN109886324B (en) | 2020-12-18 |
Family
ID=66927907
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910105196.3A Active CN109886324B (en) | 2019-02-01 | 2019-02-01 | Icon identification method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109886324B (en) |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103955713B (en) * | 2014-04-24 | 2017-08-18 | 海信集团有限公司 | A kind of icon-based programming method and apparatus |
| CN104159071A (en) * | 2014-07-11 | 2014-11-19 | 深圳瞭望通达科技有限公司 | Intelligent target identification device, system and method based on cloud service |
| CN107153844A (en) * | 2017-05-12 | 2017-09-12 | 上海斐讯数据通信技术有限公司 | The accessory system being improved to flowers identifying system and the method being improved |
| CN107967482A (en) * | 2017-10-24 | 2018-04-27 | 广东中科南海岸车联网技术有限公司 | Icon-based programming method and device |
-
2019
- 2019-02-01 CN CN201910105196.3A patent/CN109886324B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109886324A (en) | 2019-06-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108845853B (en) | Application program starting method and mobile terminal | |
| CN109947650B (en) | Script step processing method, device and system | |
| CN108427873B (en) | Biological feature identification method and mobile terminal | |
| CN108494665B (en) | Group message display method and mobile terminal | |
| CN112689201B (en) | Barrage information identification method, barrage information display method, server and electronic equipment | |
| CN108616448B (en) | Information sharing path recommendation method and mobile terminal | |
| CN107870674B (en) | Program starting method and mobile terminal | |
| CN108460817B (en) | Jigsaw puzzle method and mobile terminal | |
| CN109495616B (en) | Photographing method and terminal equipment | |
| CN110096203B (en) | Screenshot method and mobile terminal | |
| CN110913067A (en) | Information sending method and electronic equipment | |
| CN109951889B (en) | Internet of things network distribution method and mobile terminal | |
| CN108920040B (en) | Application icon sorting method and mobile terminal | |
| CN110990679A (en) | Information searching method and electronic equipment | |
| CN109885490B (en) | Picture comparison method and device | |
| CN110909776A (en) | Image identification method and electronic equipment | |
| CN108628534B (en) | Character display method and mobile terminal | |
| CN107765954B (en) | Application icon updating method, mobile terminal and server | |
| CN107809515B (en) | Display control method and mobile terminal | |
| CN110908751B (en) | Information display and collection method and device, electronic equipment and medium | |
| CN109684006B (en) | Terminal control method and device | |
| CN109714462B (en) | Method for marking telephone number and mobile terminal thereof | |
| CN108897467B (en) | Display control method and terminal equipment | |
| CN110888572A (en) | Message display method and terminal equipment | |
| CN107885887B (en) | file storage method and mobile terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20210716 Address after: 100016 no.4301, No.52 Jiuxianqiao hospital, Chaoyang District, Beijing Patentee after: BEIJING TESTIN INFORMATION TECHNOLOGY Co.,Ltd. Address before: Room 2016, building 2, No.8, Fenghuang Third Road, Zhongxin Guangzhou Knowledge City, Guangzhou 510260, Guangdong Province Patentee before: GUANGZHOU TESTIN INFORMATION TECHNOLOGY Co.,Ltd. |