CN112287738A - Text matching method and device for graphic control, medium and electronic equipment - Google Patents
Text matching method and device for graphic control, medium and electronic equipment Download PDFInfo
- Publication number
- CN112287738A CN112287738A CN202010313405.6A CN202010313405A CN112287738A CN 112287738 A CN112287738 A CN 112287738A CN 202010313405 A CN202010313405 A CN 202010313405A CN 112287738 A CN112287738 A CN 112287738A
- Authority
- CN
- China
- Prior art keywords
- text
- image
- graphic control
- information
- matching
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 238000004590 computer program Methods 0.000 claims description 2
- 230000001771 impaired effect Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 9
- 230000008569 process Effects 0.000 description 8
- 238000012545 processing Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000003993 interaction Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 238000012015 optical character recognition Methods 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/40—Document-oriented image-based pattern recognition
- G06V30/41—Analysis of document content
- G06V30/413—Classification of content, e.g. text, photographs or tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/338—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/40—Document-oriented image-based pattern recognition
- G06V30/41—Analysis of document content
- G06V30/418—Document matching, e.g. of document images
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/14—Session management
- H04L67/146—Markers for unambiguous identification of a particular session, e.g. session cookie or URL-encoding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/55—Push-based network services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The disclosure provides a text matching method of a graphic control, a text matching device of the graphic control, a computer readable storage medium and electronic equipment, and relates to the technical field of image information. A text matching method for a graphic control comprises the following steps: acquiring an image of a graphic control and an identifier of the graphic control; performing text recognition on the image of the graphic control; if the image of the graphic control does not contain the text information, determining the text information corresponding to the image of the graphic control from the information matching database; and matching the text information with the identification of the graphic control so as to broadcast the matched text information based on the identification of the graphic control. The method and the device can enable the graphic control to be accurately matched with the corresponding text information.
Description
Technical Field
The present disclosure relates to the field of image information technologies, and in particular, to a text matching method for a graphic control, a text matching device for a graphic control, a computer-readable storage medium, and an electronic device.
Background
With the development of information technology and the popularization of the internet, the life and work of people have changed greatly. Especially for visually impaired people, they can use the internet for learning, shopping, etc.
At present, in the process of using a mobile terminal by a visually impaired person, the mobile terminal can firstly acquire the text content of a graphical user interface of the mobile terminal, and then convert the text content into voice by using a screen reading program to be transmitted to the visually impaired person, so that the visually impaired person can know the text content of the graphical user interface. In addition to textual content, the graphical user interface of the mobile terminal may also expose various functions or information through various graphical controls.
However, the mobile terminal cannot acquire corresponding functions or information from various graphical controls, and cannot accurately match the various graphical controls with the corresponding functions or information, so that a visually impaired person can only know part of information of a graphical user interface of the mobile terminal.
It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present disclosure, and thus may include information that does not constitute prior art known to those of ordinary skill in the art.
Disclosure of Invention
The purpose of the present disclosure is to provide a text matching method for a graphic control, a text matching device for a graphic control, a computer-readable storage medium, and an electronic device, so as to overcome, at least to a certain extent, a problem that a graphic control cannot be accurately matched with corresponding text information due to limitations and defects of related technologies.
According to a first aspect of the present disclosure, there is provided a text matching method for a graphic control, including: acquiring an image of a graphic control and an identifier of the graphic control; performing text recognition on the image of the graphic control; if the image of the graphic control does not contain the text information, determining the text information corresponding to the image of the graphic control from the information matching database; and matching the text information with the identification of the graphic control so as to broadcast the matched text information based on the identification of the graphic control.
According to a second aspect of the present disclosure, there is provided a text matching apparatus for a graphic control, including: the information acquisition module is used for acquiring the image of the graphic control and the identification of the graphic control; the text recognition module is used for performing text recognition on the image of the graphic control; the text determining module is used for determining text information corresponding to the image of the graphic control from the information matching database if the image of the graphic control does not contain the text information; and the text matching module is used for matching the text information with the identification of the graphic control so as to broadcast the matched text information based on the identification of the graphic control.
Optionally, the text determination module may be configured to perform: judging whether the information storage database contains text information or not based on the identification of the graphic control; and if the information storage database does not contain the text information, determining the text information corresponding to the image of the graphic control from the information matching database.
Optionally, the text matching apparatus for a graphic control further includes: a text comparison module that may be configured to perform: and if the image of the graphic control contains the text information, matching the text information with the identifier of the graphic control so as to broadcast the matched text information based on the identifier of the graphic control.
Optionally, the text comparison module may be further configured to perform: if the image of the graphic control contains text information, determining the text information as first matching information; determining second matching information corresponding to the image of the graphic control from the information matching database; and matching the first matching information, the second matching information and the identification of the graphic control.
Optionally, the information obtaining module may be further configured to perform: the method comprises the steps of obtaining an image of a graphic control on the terminal device, and determining the identification of the graphic control based on the position information of the image of the graphic control on the terminal device.
Optionally, the text matching module may be further configured to perform: judging whether the text quantity of the text information is larger than a preset quantity or not; if the number of the texts is larger than the preset number, determining the text information as a candidate text; determining target text information corresponding to the position information from the candidate text; and matching the target text information with the identification of the graphic control.
Optionally, the text recognition module may be further configured to perform: performing text recognition on the image of the graphic control within preset time; and if the text recognition result corresponding to the image of the graphical control is not obtained within the preset time, sending reminding information to the terminal equipment so as to obtain the image of the graphical control again.
According to a third aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a text matching method for a graphical control as described above.
According to a fourth aspect of the present disclosure, there is provided an electronic device comprising: one or more processors; a storage device to store one or more programs that, when executed by one or more processors, cause the one or more processors to implement a text matching method for a graphical control as described above.
Exemplary embodiments of the present disclosure have the following advantageous effects:
in the technical solutions provided by some embodiments of the present disclosure, first, an image of a graphic control and an identifier of the graphic control are obtained; then, performing text recognition on the image of the graphic control; secondly, if the image of the graphic control does not contain text information, determining the text information corresponding to the image of the graphic control from the information matching database; and then, matching the text information with the identification of the graphical control so as to broadcast the matched text information based on the identification of the graphical control. On one hand, the method and the device can identify whether the image of the graphic control contains the text information, avoid determining the text information corresponding to the image of the graphic control only by using the image of the graphic control, and improve the integrity of the text information determination process. In another aspect. The method and the device can match the identification of the graphic control with the determined text information, so that the graphic control is accurately matched with the corresponding text information, the server can accurately acquire the text information corresponding to the image of the graphic control based on the identification of the graphic control, and further, the terminal device can play the voice corresponding to the matched text information based on the subsequent identification of the graphic control, so that the user can perceive the voice.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure. It is to be understood that the drawings in the following description are merely exemplary of the disclosure, and that other drawings may be derived from those drawings by one of ordinary skill in the art without the exercise of inventive faculty. In the drawings:
FIG. 1 schematically illustrates a flow chart of a method of text matching of a graphical control according to an exemplary embodiment of the present disclosure;
FIG. 2 schematically illustrates a flow diagram of a method of text matching of graphical controls according to an exemplary embodiment of the present disclosure;
FIG. 3 schematically illustrates a flow diagram of a method of text matching of a graphical control according to another exemplary embodiment of the present disclosure;
FIG. 4 schematically illustrates an interaction diagram of a text matching method for a graphical control according to an exemplary embodiment of the present disclosure;
FIG. 5 schematically illustrates a block diagram of an apparatus for text matching of graphical controls according to an exemplary embodiment of the present disclosure;
FIG. 6 schematically illustrates a block diagram of an apparatus for text matching of graphical controls according to another exemplary embodiment of the present disclosure;
fig. 7 schematically shows a block diagram of an electronic device in an exemplary embodiment according to the present disclosure.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the subject matter of the present disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and the like. In other instances, well-known technical solutions have not been shown or described in detail to avoid obscuring aspects of the present disclosure.
Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
In the present disclosure, the terms "comprises" and "comprising" are used in an open-ended fashion, and mean that there may be additional elements/components/etc. in addition to the listed elements/components/etc. In addition, the terms "first" and "second" used in the present disclosure are for the purpose of distinction only and should not be construed as a limitation of the present disclosure.
The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the steps. For example, some steps may be decomposed, and some steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
With the rapid development of the internet and mobile terminal devices, visually handicapped people can learn, shop, etc. on the mobile terminal devices by means of the internet technology.
In the case where a visually impaired person operates a mobile terminal device, the existing technical solution includes: the mobile terminal can determine the text content in the graphical user interface and output the text content in a voice mode by using the screen reading program, so that the visually impaired can know the content on the graphical user interface. However, the mobile terminal cannot acquire and output information in the graphical control of the graphical user interface through the screen reading program, and cannot match the graphical control with corresponding information, so that visually impaired people can know the content of the graphical control of the graphical user interface.
In order to solve the problem, the disclosure provides a text matching method for a graphic control.
It should be noted that, in the exemplary embodiment of the present disclosure, the text matching method of the graphic control described below may be generally implemented by a server, that is, the steps of the text matching method of the graphic control may be performed by the server, in which case, the text matching apparatus of the graphic control may be configured in the server.
In addition, the text matching method of the graphic control can also be implemented by a terminal device (e.g., a mobile phone, a tablet, a personal computer, etc.), that is, the steps of the text matching method of the graphic control can be executed by the terminal device, in which case, the text matching device of the graphic control can be configured in the terminal device.
Hereinafter, the steps of the text matching method for a graphic control in the present exemplary embodiment will be described in more detail with reference to the drawings and the examples.
Fig. 1 schematically illustrates a flowchart of a text matching method for a graphical control according to an exemplary embodiment of the present disclosure. In the following description, a server is used as an execution subject. Referring to fig. 1, the text matching method of the graphic control may include the steps of:
and S102, acquiring the image of the graphic control and the identification of the graphic control.
In an exemplary embodiment of the present disclosure, the graphical control may be presented in the form of an image, and may respond to a user's interaction instruction and provide information or functionality corresponding to the graphical control. The image of the graphical control may or may not contain textual information and an image. The image of the graphic control can be determined according to the clicking position of the visually impaired on the graphic user interface of the terminal equipment, and can also be determined by the terminal equipment according to the hierarchical structure and the position coordinates of the graphic control. Wherein the image can be obtained by the terminal device by intercepting the image of the graphical user interface.
The identification of the graphic control can be a storage number of the graphic control on the server or the terminal device, can be determined according to the position information of the graphic control on the terminal device, and can also be determined according to the position information of the image of the graphic control on the terminal device. The position information of the image of the graphical control can comprise position coordinates and a hierarchical structure of the image of the graphical control on the graphical user interface. The position information of the graphical control is the same as the position information of the image of the graphical control. For example, the image of the graphical control is a left-pointing arrow located in the upper left corner of the graphical user interface and is at the first level of the graphical user interface. It should be noted that, all the information that can be in one-to-one correspondence with the graphical controls can be used as the setting rule of the identifiers of the graphical controls.
According to the exemplary embodiment of the disclosure, the server may obtain an image of a graphical control on the terminal device, and determine the identifier of the graphical control based on the position information of the image of the graphical control on the terminal device. The identification of the graphic control can also be determined by the terminal equipment and sent to the server.
And S104, performing text recognition on the image of the graphic control.
After the image of the graphic control is obtained, the server may perform text Recognition on the image of the graphic control by using an Optical Character Recognition (OCR) technology to determine whether the image of the graphic control contains text information.
According to an exemplary embodiment of the present disclosure, the server may perform text recognition on an image of the graphic control within a preset time; and if the text recognition result corresponding to the image of the graphical control is not obtained within the preset time, sending reminding information to the terminal equipment so as to obtain the image of the graphical control again.
The preset time may refer to a time that may be required for the server to determine text recognition according to the size of the image of the graphic control. The text recognition result may include: the server may identify the result of the graphical control's image containing textual information or the server may not identify the result of the graphical control's image containing textual information. The text recognition result may include recognized text information. The reminding information can contain the image of the graphical control which needs to be redetermined by the terminal equipment and is uploaded to the server.
The method and the device avoid the situation that the determined text information is incomplete due to the fact that the obtained image of the graph control is not clear by sending the reminding information, and improve the integrity and the accuracy of the graph control matching process.
And S106, if the image of the graphic control does not contain the text information, determining the text information corresponding to the image of the graphic control from the information matching database.
In an exemplary embodiment of the present disclosure, the images and the text information contained in the information matching database may be matched one-to-one. Wherein the information matching database can be obtained from the internet. The server can inquire out text information corresponding to the image of the image control from the information matching database according to the image of the image control. Specifically, the server may also determine feature points or similar points of the image control, compare the feature points of the image control with the feature points of the image included in the information matching database, and then obtain text information corresponding to the image of the image control.
For example, image A of the graphical control is a left arrow, and the graphical control is identified as 1-31-02. Wherein 1-31-02 represents the second image of the image a of the graphical control located in the upper left corner of the first hierarchical structure in the graphical user interface of the terminal device. In addition, in the information matching database, text information corresponding to an image with a left arrow is returned.
The server firstly acquires an image A of the graphic control and an identifier of the graphic control, then performs text recognition on the image A of the graphic control to determine that the image of the graphic control does not contain text information, and then compares the image A of the graphic control with an image contained in an information matching database to determine that the text information corresponding to the image A of the graphic control is as follows: and returning.
According to an exemplary embodiment of the present disclosure, the server may also determine whether the information storage database contains text information based on the identifier of the graphical control; and if the information storage database does not contain the text information, determining the text information corresponding to the image of the graphic control from the information matching database.
The information storage database may include an identifier of the graphic control and text information corresponding to an image of the graphic control. It should be noted that the image of the graphic control, the identifier of the graphic control, and the text information corresponding to the image of the graphic control are in a one-to-one correspondence relationship. That is, if the information storage database includes the identifier of the graphical control and the text information corresponding to the image of the graphical control, the server may determine the text information corresponding to the image of the graphical control from the information storage database.
For example, image A of the graphical control is a left arrow, and the graphical control is identified as 1-31-02. Wherein 1-31-02 represents the second image of the image a of the graphical control located in the upper left corner of the first hierarchical structure in the graphical user interface of the terminal device. In addition, in the information matching database, text information corresponding to an image with a left arrow is returned. The text information "return" corresponding to the identification "1-31-02" of the graphical control and the identification "1-31-02" of the graphical control are not stored in the information storage database.
Firstly, a server acquires an image A of a graphic control and an identifier of the graphic control; then, performing text recognition on the image A of the graphic control to determine that the image A of the graphic control does not contain text information; then, the server compares the identifier 1-31-02 of the graphic control with the identifier of the graphic control contained in the information storage database to obtain that the information storage database does not contain text information; subsequently, the server compares the image A of the graphic control with the image contained in the information matching database, and determines that the text information corresponding to the image A of the graphic control is as follows: and returning.
And S108, matching the text information with the identification of the graphic control so as to broadcast the matched text information based on the identification of the graphic control.
After determining the text information corresponding to the image of the graphic control, the server may match the text information with the identifier of the graphic control and store the matched text information in the information storage database, so that the server may query the text information corresponding to the image of the graphic control from the information storage database.
In an exemplary embodiment of the disclosure, the server may feed back text information corresponding to the image of the graphic control to the terminal device, so that the terminal device broadcasts the text information. The server can also convert the text information corresponding to the image of the graphic control into voice, and send the converted voice to the terminal equipment, and the terminal equipment broadcasts the converted voice based on the identification of the graphic control, so that a visually impaired person can know the text information corresponding to the image of the graphic control in the graphic user interface.
According to the exemplary embodiment of the disclosure, if the image of the graphic control contains the text information, the text information is matched with the identifier of the graphic control, so that the matched text information is broadcasted based on the identifier of the graphic control.
For example, image A of the graphical control, identified as 1-31-02, is an arrow to the left and the textual information "back". The image B of the graphic control is a magnifier image, and the identification of the graphic control is 1-33-01. Wherein 1-31-02 represents the second image of the image a of the graphical control located in the upper left corner of the first hierarchical structure in the graphical user interface of the terminal device. 1-33-01 represents the first image of the graphic control whose image B is located in the upper right corner of the first hierarchy in the graphical user interface of the terminal device. In addition, in the information matching database, text information corresponding to an image with a left arrow is returned; the text information corresponding to the magnifier image is a search.
Referring to fig. 2, for the image a of the graphical control, in step S201, the server obtains the image a of the graphical control and the identifier "1-31-02" of the graphical control; in step S203, performing text recognition on the image a of the graphical control, determining that the image a of the graphical control contains text information "return", and executing step S207; in step S207, the identifier "1-31-02" of the graphic control is matched with the text information "return", stored in the information storage database, and sent to the terminal device, so that the terminal device broadcasts the text information "return" based on the identifier "1-31-02" of the graphic control.
For the image B of the graphic control, in step S201, the server first obtains the image B of the graphic control and the identifier "1-33-01" of the graphic control; in step S203, performing text recognition on the image B of the graphical control, determining that the image B of the graphical control does not contain text information, and executing step S205; in step S205, the server compares the image B of the image control with the image included in the information matching database, and determines that the text information corresponding to the image B of the image control is: searching; in step S207, the identifier "1-33-01" of the graphical control is matched with the text information "search", stored in the information storage database, and sent to the terminal device, so that the terminal device broadcasts the text information "search" based on the identifier "1-33-01" of the graphical control.
According to the method and the device, whether the image of the graphic control contains the text information or not is judged, different steps are executed, the identification of the graphic control is matched with the determined text information, the condition that the information is incomplete due to the fact that the text information is determined only according to the image of the graphic control is avoided, and the completeness and the accuracy of the text matching process of the graphic control are improved.
According to an exemplary embodiment of the present disclosure, if the image of the graphic control contains text information, the server may first determine that the text information is first matching information; then, second matching information corresponding to the image of the graphic control is determined from the information matching database; and then matching the first matching information, the second matching information and the identification of the graphical control.
For example, image C of the graphical control, labeled 1-11-02, contains a left arrow and the textual information "first page". Wherein 1-11-02 represents a second image of the image C of the graphical control located in the middle of the bottom of the first hierarchical structure in the graphical user interface of the terminal device. In addition, in the information matching database, text information corresponding to an image with a left arrow is returned.
The server acquires an image C of the graphic control and an identifier 1-11-02 of the graphic control; then, performing text recognition on the image C of the graphical control to determine that the image C of the graphical control contains text information 'first page'; secondly, the server determines the text information 'first page' as first matching information; thirdly, the server compares the image C of the graphic control with the image contained in the information matching database, and determines that the second matching information corresponding to the image C of the graphic control is as follows: returning; subsequently, the server can match the first matching information "first page", the second matching information "return" and the identifier "1-11-02" of the graphical control, store the first matching information "first page", the second matching information "return" and the identifier "1-11-02" of the graphical control to the information storage database, and send the information storage database to the terminal device, so that the terminal device can broadcast the information. That is, the text information corresponding to the image of the graphic control is: the first page is returned.
The text information contained in the image of the graphic control and the text information determined from the information matching database are matched with the identification of the graphic control, so that the text information corresponding to the image of the graphic control is more comprehensive and accurate, and the voice corresponding to the matched text information is broadcasted by the terminal equipment based on the identification of the graphic control, so that the user can perceive the voice.
According to an exemplary embodiment of the present disclosure, in a case where one image may exist in the information matching database corresponding to a plurality of text messages, the server may determine whether the number of texts of the text messages is greater than a preset number; if the number of the texts is larger than the preset number, determining the text information as a candidate text; determining target text information corresponding to the position information from the candidate text; and matching the target text information with the identification of the graphic control.
For example, image B of the graphical control is a magnifier image, and the graphical control is identified as 1-33-01. Wherein 1-33-01 represents the first image of the image B of the graphical control located in the upper right corner of the first hierarchical structure in the graphical user interface of the terminal device. In addition, in the information matching database, the text information corresponding to one magnifier image may be searched, enlarged, reduced, and the like.
The server acquires an image B of the graphic control and an identifier 1-33-01 of the graphic control; then, performing text recognition on the image B of the graphic control to determine that the image B of the graphic control does not contain text information; secondly, the server compares the image B of the graphic control with the images contained in the information matching database to obtain 3 pieces of text information, which are respectively: searching, amplifying and reducing; thirdly, if the server obtains that the 3 pieces of text information are more than the preset number of 1, determining the 3 pieces of text information as candidate texts, wherein the image B of the graphic control is positioned at the upper right corner of the first hierarchical structure in the graphic user interface of the terminal equipment, and the server determines that the text information is searched as target text information according to the position information of the image B of the graphic control; and then matching the target text information 'search' with the identifier '1-33-01' of the graphical control, and sending the target text information 'search' to the terminal equipment so as to broadcast the target text information by the terminal equipment.
In an exemplary embodiment of the present disclosure, after the server performs text recognition on the acquired image of the graphical control, and determines that the image of the graphical control does not contain text information, the server may first determine whether the information storage database contains text information based on an identifier of the graphical control; then, if the information storage database does not contain text information, determining text information corresponding to the image of the graphic control from the information matching database; and then matching the text information corresponding to the image of the graphic control with the identification of the graphic control, and storing the text information in an information storage database.
For example, image A of the graphical control is a left arrow, and the graphical control is identified as 1-31-02. Wherein 1-31-02 represents the second image of the image a of the graphical control located in the upper left corner of the first hierarchical structure in the graphical user interface of the terminal device. In addition, in the information matching database, text information corresponding to an image with a left arrow is returned. The text information "return" corresponding to the identification "1-31-02" of the graphical control and the identification "1-31-02" of the graphical control are not stored in the information storage database.
Taking fig. 3 as an example, in step S302, the server may obtain the image a of the graphical control and the identifier "1-31-02" of the graphical control; in step S304, performing text recognition on the image a of the graphical control, determining that the image a of the graphical control does not contain text information, and performing step S306; in step S306, the server compares the identifier "1-31-02" of the graphical control with the identifiers of the graphical controls contained in the information storage database to obtain text information corresponding to the image a that does not contain the graphical control in the information storage database, and executes step S308; in step S308, the server compares the image a of the graphic control with the image included in the information matching database, and determines that the text information corresponding to the image a of the graphic control is: returning; in step S310, the server may match the text information "return" corresponding to the image a of the graphical control with the identifier "1-31-02" of the graphical control, and store the result in the information storage database, so that the terminal device broadcasts the text information "return" based on the identifier "1-31-02" of the graphical control.
According to an exemplary embodiment of the present disclosure, the information storage database and the information matching database may also contain voice information. After the text recognition is carried out on the image of the graphic control, the server can also determine the voice information corresponding to the image of the graphic control from the information matching database according to the image of the graphic control, then match the voice information with the identification of the graphic control and store the voice information into the information storage database, and then send the voice information corresponding to the image of the graphic control to the terminal equipment so as to broadcast the voice information.
It should be noted that, in the case that the information storage database contains the voice information corresponding to the identifier of the graphic control, the terminal device may request the voice information corresponding to the identifier of the graphic control and broadcast the voice information.
Fig. 4 schematically illustrates an interaction diagram of a text matching method of a graphic control according to an exemplary embodiment of the present disclosure.
In step S401, the terminal device may determine an image of the graphical control in response to a click instruction of the user; in step S403, the server acquires the image of the graphical control and the identifier of the graphical control from the terminal device; in step S405, the server performs text recognition on the image of the graphical control, if the image of the graphical control does not contain text information, step S407 is executed, and if the image of the graphical control contains text information, step S409 is executed; in step S407, the server may determine text information corresponding to the image of the graphic control from the information matching database; in step S409, the server may match the text information corresponding to the image of the graphic control with the identifier of the graphic control; in step S411, the server transmits the text information to the terminal device; in step S413, the terminal device may broadcast the matched text information based on the identifier of the graphic control.
In an exemplary embodiment of the disclosure, before the terminal device determines the image of the graphical control, the terminal device may respond to a click instruction of a user to detect the type of the control, so that the terminal device determines the graphical control which needs to be matched. The terminal device can intercept the current page of the graphical user interface and determine the image of the graphical control according to the click command of the user. The terminal device may also only intercept the image of the graphical control and upload the identifier of the graphical control to the server.
In addition, if it is determined in step S405 that the image of the graphic control includes text information, in step S407, the server may determine that the text information included in the image of the graphic control is first matching information, and determine second matching information corresponding to the image of the graphic control from the information matching database. If it is determined in step S405 that the image of the graphical control does not contain the text information, in step S407, the server may determine whether the information storage database contains the text information based on the identifier of the graphical control; and if the information storage database does not contain the text information, determining the text information corresponding to the image of the graphic control from the information matching database.
For example, the terminal device determines that the image C of the graphic control is a leftward arrow and the text information "first page", and the identifier of the graphic control is 1-11-02; the image B of the graphic control is a magnifier image, and the identification of the graphic control is 1-33-01. Wherein 1-11-02 represents a second image of the image C of the graphical control located in the middle of the bottom of the first hierarchical structure in the graphical user interface of the terminal device. 1-33-01 represents the first image of the graphic control whose image B is located in the upper right corner of the first hierarchy in the graphical user interface of the terminal device. In addition, in the information matching database, text information corresponding to an image with a left arrow is returned; the text information corresponding to the magnifier image is a search. The textual information "search" corresponding to the identification of the graphical control "1-33-01" and the identification of the graphical control "1-33-01" is not stored in the information storage database.
For the image C of the graphic control, the server can firstly acquire the image C of the graphic control and the identifier '1-11-02' of the graphic control from the terminal equipment; then, performing text recognition on the image C of the graphical control to determine that the image C of the graphical control contains text information 'first page'; secondly, the server determines the text information 'first page' as first matching information; thirdly, the server compares the image C of the graphic control with the image contained in the information matching database, and determines that the second matching information corresponding to the image C of the graphic control is as follows: returning; subsequently, the server can match the first matching information "first page", the second matching information "return" and the identifier "1-11-02" of the graphical control, store the first matching information "first page", the second matching information "return" and the identifier "1-11-02" of the graphical control to the information storage database, and send the information storage database to the terminal device, so that the terminal device can broadcast the information.
For the image B of the graphic control, the server can firstly acquire the image B of the graphic control and the identifier 1-33-01 of the graphic control from the terminal equipment; then, performing text recognition on the image B of the graphic control to determine that the image B of the graphic control does not contain text information; secondly, the server compares the identifier 1-33-01 of the graphic control with the identifier of the graphic control contained in the information storage database to determine that the text information corresponding to the image B of the graphic control is not contained in the information storage database; thirdly, the server compares the image B of the graphic control with the image contained in the information matching database, and determines that the text information corresponding to the image B of the graphic control is as follows: searching; and then, matching the identifier 1-33-01 of the graphic control with the text information search, storing the matched identifier in an information storage database, and sending the stored identifier to the terminal equipment so as to broadcast the identifier by the terminal equipment.
It should be noted that although the various steps of the methods of the present disclosure are depicted in the drawings in a particular order, this does not require or imply that these steps must be performed in this particular order, or that all of the depicted steps must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions, etc.
Further, in an exemplary embodiment of the present disclosure, a text matching apparatus for a graphic control is also provided.
Fig. 5 schematically illustrates a block diagram of a text matching apparatus for a graphic control according to an exemplary embodiment of the present disclosure. Referring to fig. 5, a text matching apparatus 500 of a graphic control according to an exemplary embodiment of the present disclosure includes: an information acquisition module 502, a text recognition module 504, a text determination module 506, and a text matching module 508.
An information obtaining module 502, configured to obtain an image of a graphical control and an identifier of the graphical control; a text recognition module 504, configured to perform text recognition on the image of the graphical control; a text determining module 506, configured to determine, if the image of the graphical control does not contain text information, text information corresponding to the image of the graphical control from the information matching database; and the text matching module 508 is configured to match the text information with the identifier of the graphical control, so as to broadcast the matched text information based on the identifier of the graphical control.
According to another embodiment of the disclosure, the text determination module 506 may be configured to perform: judging whether the information storage database contains text information or not based on the identification of the graphic control; and if the information storage database does not contain the text information, determining the text information corresponding to the image of the graphic control from the information matching database.
According to another embodiment of the present disclosure, referring to fig. 6, compared to the text matching apparatus 500 of the graphic control, the text matching apparatus 600 of the graphic control may further include: text alignment module 601, which may be configured to perform: and if the image of the graphic control contains the text information, matching the text information with the identifier of the graphic control so as to broadcast the matched text information based on the identifier of the graphic control.
According to another embodiment of the present disclosure, the text alignment module 601 may be further configured to perform: if the image of the graphic control contains text information, determining the text information as first matching information; determining second matching information corresponding to the image of the graphic control from the information matching database; and matching the first matching information, the second matching information and the identification of the graphic control.
According to another embodiment of the present disclosure, the information obtaining module 502 may be further configured to perform: the method comprises the steps of obtaining an image of a graphic control on the terminal device, and determining the identification of the graphic control based on the position information of the image of the graphic control on the terminal device.
According to another embodiment of the disclosure, the text matching module 508 may be further configured to perform: judging whether the text quantity of the text information is larger than a preset quantity or not; if the number of the texts is larger than the preset number, determining the text information as a candidate text; determining target text information corresponding to the position information from the candidate text; and matching the target text information with the identification of the graphic control.
According to another embodiment of the disclosure, the text recognition module 504 may be further configured to perform: performing text recognition on the image of the graphic control within preset time; and if the text recognition result corresponding to the image of the graphical control is not obtained within the preset time, sending reminding information to the terminal equipment so as to obtain the image of the graphical control again.
The details of each module/unit in the above-mentioned apparatus have been described in detail in the embodiments of the method section, and thus are not described again.
In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium having stored thereon a program product capable of implementing the above-described method of the present specification. In some possible embodiments, aspects of the invention may also be implemented in the form of a program product comprising program code means for causing a terminal device to carry out the steps according to various exemplary embodiments of the invention described in the above-mentioned "exemplary methods" section of the present description, when the program product is run on the terminal device.
In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.
An electronic device 700 according to this embodiment of the invention is described below with reference to fig. 7. The electronic device 700 shown in fig. 7 is only an example and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 7, electronic device 700 is embodied in the form of a general purpose computing device. The components of the electronic device 700 may include, but are not limited to: the at least one processing unit 710, the at least one memory unit 720, a bus 730 connecting different system components (including the memory unit 720 and the processing unit 710), and a display unit 740.
Wherein the storage unit stores program code that is executable by the processing unit 710 such that the processing unit 710 performs the steps according to various exemplary embodiments of the present invention as described in the above section "exemplary method" of the present specification. For example, the processing unit 710 may perform steps S102 to S108 as shown in fig. 1.
The storage unit 720 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM)7201 and/or a cache memory unit 7202, and may further include a read only memory unit (ROM) 7203.
The storage unit 720 may also include a program/utility 7204 having a set (at least one) of program modules 7205, such program modules 7205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The electronic device 700 may also communicate with one or more external devices 800 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to communicate with the electronic device 700, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 700 to communicate with one or more other computing devices. Such communication may occur via an input/output (I/O) interface 750. Also, the electronic device 700 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the internet) via the network adapter 760. As shown, the network adapter 760 communicates with the other modules of the electronic device 700 via the bus 730. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.
Furthermore, the above-described figures are merely schematic illustrations of processes involved in methods according to exemplary embodiments of the invention, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.
It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is to be limited only by the terms of the appended claims.
Claims (10)
1. A text matching method for a graphic control is characterized by comprising the following steps:
acquiring an image of a graphic control and an identifier of the graphic control;
performing text recognition on the image of the graphic control;
if the image of the graphic control does not contain text information, determining the text information corresponding to the image of the graphic control from an information matching database;
and matching the text information with the identification of the graphic control so as to broadcast the matched text information based on the identification of the graphic control.
2. The method for matching text of a graphic control according to claim 1, wherein determining the text information corresponding to the image of the graphic control from an information matching database comprises:
judging whether an information storage database contains the text information or not based on the identification of the graphic control;
and if the information storage database does not contain the text information, determining the text information corresponding to the image of the graphic control from the information matching database.
3. The method for matching the text of the graphic control according to claim 1 or 2, wherein the method for matching the text of the graphic control further comprises:
and if the image of the graphic control contains the text information, matching the text information with the identifier of the graphic control so as to broadcast the matched text information based on the identifier of the graphic control.
4. The method of claim 3, wherein if the image of the graphical control contains the text information, matching the text information with the identifier of the graphical control comprises:
if the image of the graphic control contains the text information, determining that the text information is first matching information;
determining second matching information corresponding to the image of the graphic control from the information matching database;
and matching the first matching information, the second matching information and the identification of the graphic control.
5. The method for matching the text of the graphic control according to claim 1 or 2, wherein the obtaining of the image of the graphic control and the identification of the graphic control comprises:
the method comprises the steps of obtaining an image of a graphic control on terminal equipment, and determining the identification of the graphic control based on the position information of the image of the graphic control on the terminal equipment.
6. The method of claim 5, wherein matching the textual information with the identity of the graphical control comprises:
judging whether the text quantity of the text information is larger than a preset quantity or not;
if the text number is larger than the preset number, determining the text information as a candidate text; determining target text information corresponding to the position information from the candidate text;
and matching the target text information with the identification of the graphic control.
7. The method for text matching of a graphical control according to claim 1 or 2, wherein the text recognition of the image of the graphical control comprises:
performing text recognition on the image of the graphic control within preset time;
and if the text recognition result corresponding to the image of the graphical control is not obtained within the preset time, sending reminding information to terminal equipment so as to obtain the image of the graphical control again.
8. An apparatus for matching text of a graphic control, comprising:
the information acquisition module is used for acquiring an image of a graphic control and an identifier of the graphic control;
the text recognition module is used for performing text recognition on the image of the graphic control;
the text determining module is used for determining the text information corresponding to the image of the graphic control from an information matching database if the image of the graphic control does not contain the text information;
and the text matching module is used for matching the text information with the identification of the graphic control so as to broadcast the matched text information based on the identification of the graphic control.
9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements a text matching method for a graphical control according to any one of claims 1 to 7.
10. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the method of text matching for a graphical control as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010313405.6A CN112287738A (en) | 2020-04-20 | 2020-04-20 | Text matching method and device for graphic control, medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010313405.6A CN112287738A (en) | 2020-04-20 | 2020-04-20 | Text matching method and device for graphic control, medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112287738A true CN112287738A (en) | 2021-01-29 |
Family
ID=74420213
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010313405.6A Pending CN112287738A (en) | 2020-04-20 | 2020-04-20 | Text matching method and device for graphic control, medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112287738A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113343664A (en) * | 2021-06-29 | 2021-09-03 | 京东数科海益信息科技有限公司 | Method and device for determining matching degree between image texts |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080040693A1 (en) * | 2006-01-25 | 2008-02-14 | Microsoft Corporation | Computer interface for illiterate and near-illiterate users |
| CN103838464A (en) * | 2014-03-06 | 2014-06-04 | 北京保益互动科技发展有限公司 | Adaptive method of graphical controls for the blind to read mobile phone screens |
| CN107613352A (en) * | 2017-09-28 | 2018-01-19 | 深圳Tcl数字技术有限公司 | Sound control method, intelligent television and storage medium for intelligent television |
| CN109471678A (en) * | 2018-11-07 | 2019-03-15 | 苏州思必驰信息科技有限公司 | Voice midpoint controlling method and device based on image recognition |
| CN109803050A (en) * | 2019-01-14 | 2019-05-24 | 南京点明软件科技有限公司 | A kind of full frame guidance click method suitable for operation by blind mobile phone |
-
2020
- 2020-04-20 CN CN202010313405.6A patent/CN112287738A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20080040693A1 (en) * | 2006-01-25 | 2008-02-14 | Microsoft Corporation | Computer interface for illiterate and near-illiterate users |
| CN103838464A (en) * | 2014-03-06 | 2014-06-04 | 北京保益互动科技发展有限公司 | Adaptive method of graphical controls for the blind to read mobile phone screens |
| CN107613352A (en) * | 2017-09-28 | 2018-01-19 | 深圳Tcl数字技术有限公司 | Sound control method, intelligent television and storage medium for intelligent television |
| CN109471678A (en) * | 2018-11-07 | 2019-03-15 | 苏州思必驰信息科技有限公司 | Voice midpoint controlling method and device based on image recognition |
| CN109803050A (en) * | 2019-01-14 | 2019-05-24 | 南京点明软件科技有限公司 | A kind of full frame guidance click method suitable for operation by blind mobile phone |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113343664A (en) * | 2021-06-29 | 2021-09-03 | 京东数科海益信息科技有限公司 | Method and device for determining matching degree between image texts |
| CN113343664B (en) * | 2021-06-29 | 2023-08-08 | 京东科技信息技术有限公司 | Method and device for determining matching degree between image texts |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102106462B1 (en) | Method for filtering similar problem based on weight | |
| CN107656922B (en) | Translation method, translation device, translation terminal and storage medium | |
| CN109582880B (en) | Interest point information processing method, device, terminal and storage medium | |
| CN108829371B (en) | Interface control method and device, storage medium and electronic equipment | |
| CN110442697B (en) | Man-machine interaction method, system, computer equipment and storage medium | |
| CN107608618B (en) | Interaction method and device for wearable equipment and wearable equipment | |
| KR20210037637A (en) | Translation method, apparatus and electronic equipment | |
| CN109783589B (en) | Method, device and storage medium for resolving address of electronic map | |
| CN112925898B (en) | Question-answering method and device based on artificial intelligence, server and storage medium | |
| CN112559865A (en) | Information processing system, computer-readable storage medium, and electronic device | |
| US20210279411A1 (en) | Visual data mapping | |
| CN110134920B (en) | Pictogram compatible display method, device, terminal and computer readable storage medium | |
| CN107239209B (en) | Photographing search method, device, terminal and storage medium | |
| CN113626441A (en) | Text management method, device and equipment based on scanning equipment and storage medium | |
| CN112287738A (en) | Text matching method and device for graphic control, medium and electronic equipment | |
| CN113342954A (en) | Image information processing method and device applied to question-answering system and electronic equipment | |
| CN117312140A (en) | Method and device for generating test case, electronic equipment and storage medium | |
| CN111881900A (en) | Corpus generation, translation model training and translation method, apparatus, device and medium | |
| CN117033309A (en) | Data conversion method and device, electronic equipment and readable storage medium | |
| CN113050933B (en) | Brain graph data processing method, device, equipment and storage medium | |
| CN112966671A (en) | Contract detection method and device, electronic equipment and storage medium | |
| CN113487698B (en) | Form generation method and device based on two-channel neural network model | |
| CN113268193B (en) | Notebook page moving method, electronic equipment and computer storage medium | |
| CN108932326B (en) | Instance extension method, device, equipment and medium | |
| CN115544203A (en) | Machine learning-based inquiry method, device, medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |