WO2022111582A1

WO2022111582A1 - Text extraction method and apparatus

Info

Publication number: WO2022111582A1
Application number: PCT/CN2021/133172
Authority: WO
Inventors: 缪丹
Original assignee: 华为技术有限公司
Priority date: 2020-11-27
Filing date: 2021-11-25
Publication date: 2022-06-02
Also published as: CN114564141A

Abstract

The present disclosure relates to a text extraction method and apparatus, which are applied to the field of optical character recognition (OCR) in the field of artificial intelligence (AI). The method comprises: in response to a touch-control operation on a touch screen, acquiring a touch-control area; extracting first text information in the touch-control area by means of OCR technology; determining, from one or more textual controls displayed on the touch screen, a target control matching the touch-control area; acquiring second text information from the target control; and adjusting the first text information on the basis of the second text information, so as to obtain third text information. By means of the text extraction method and apparatus provided in the present disclosure, text information required by a user can be conveniently, quickly and accurately acquired.

Description

Text extraction method and device

This application claims the priority of the Chinese patent application with the application number 202011362776.X and the application name "Text Extraction Method and Apparatus" filed with the China Patent Office on November 27, 2020, the entire contents of which are incorporated into this application by reference .

technical field

The present disclosure relates to the field of Optical Character Recognition (OCR) in the field of terminal artificial intelligence (Artificial Intelligence, AI), and in particular, to a text extraction method and device.

Background technique

In life, words are everywhere, and words are an important means for people to perceive the world. Artificial intelligence technology can simulate, extend and expand human consciousness and thinking. Obtaining text information is an important part of artificial intelligence technology.

OCR technology and control word extraction technology are two common ways to obtain text information. OCR technology can read out the characters on pictures or paper and convert them into computer text. However, OCR technology cannot accurately recognize characters that are difficult for human eyes to distinguish, for example, OCR cannot accurately distinguish lowercase L (ie l) and uppercase i (ie I). OCR technology can not accurately identify the characters and password characters in the link. Although the text obtained by the control word extraction technology is completely consistent with the original text, the control word extraction technology obtains all the text in the entire control, which requires the user to find the desired part, which is cumbersome to operate.

SUMMARY OF THE INVENTION

In view of this, a text extraction method and device are proposed, which can conveniently, quickly and accurately obtain the text information required by the user.

In a first aspect, an embodiment of the present disclosure provides a text extraction method, including: a terminal device, in response to a touch operation on a touch screen, acquires a touch area, and extracts text information in the touch area through OCR technology, and records is the first text message. The terminal device determines a target control that matches the previously acquired touch area from one or more textual controls on the touch screen that can obtain text content, and obtains text information from the target control, which is recorded as second text information. The terminal device adjusts the first text information based on the second text information to obtain final third text information. In this way, by adjusting the first text information with accurate text position based on the second text information with correct text content, the third text information with accurate position and correct content can be obtained conveniently and quickly.

According to the first aspect, in the first implementation manner of the text extraction method, the terminal device may acquire the intersection ratio of each textual control displayed on the touch screen and the touch area; based on the intersection ratio , and determine the target control. In this way, the control from which the user wants to extract the text information can be more accurately determined, thereby improving the accuracy of the text content of the finally acquired text information.

According to the first aspect, or the first implementation manner of the first aspect, in a second implementation manner of the text extraction method, the first text information is adjusted based on the second text information to obtain The third text information may include: comparing the characters corresponding to the same position on the touch screen in the first text information and the second text information; comparing the first text information with the characters in the second text information The character corresponding to the same position and inconsistent content on the touch screen is determined as the target character; the target character in the first text information is replaced with the target character corresponding to the second text information on the touch screen at the same position characters in, the third text information is obtained. In this way, the text information extracted by the OCR technology can be made into text information with more accurate text content.

According to a second implementation manner of the first aspect, in a third implementation manner of the text extraction method, replacing the target character in the first text information with the target character corresponding to the same one on the touch screen The characters in the second text information of the position may include: determining a matching rate according to the number of the target characters and the number of characters in the first text information; in the case that the matching rate is greater than the first threshold, the first A target character in a piece of text information is replaced with a character in the second text information corresponding to the target character at the same position on the touch screen to obtain the third text information. In this way, the correctness can be improved by performing character replacement when the matching rate is high.

According to the first aspect, or the first implementation manner of the first aspect, in a fourth implementation manner of the text extraction method, the first text information is adjusted based on the second text information to obtain The third text information may include: detecting whether a character set satisfying a preset format exists in the second text information; if there is a character set satisfying the preset format in the second text information, from the A character set satisfying the preset format is extracted from the second text information; the first text information is replaced with the extracted character set to obtain the third text information. OCR technology is used to determine the control to which the link or password belongs, and then the link or password is automatically extracted from the control, which can not only ensure the integrity and location accuracy of the link or password, but also ensure the correctness of the link or password, and the operation is fast and convenient. .

According to a fourth implementation manner of the first aspect, in a fifth implementation manner of the text extraction method, the terminal device may provide a service corresponding to the character set in the preset format according to the third text information. In this way, service efficiency can be improved, and user satisfaction can be improved.

According to the first aspect, or any implementation manner of the above first aspect, in a sixth implementation manner of the text extraction method, the acquiring a touch area in response to a touch operation may include: in response to the touch operation, obtain the position information of the start touch point and the position information of the end touch point; determine the touch area according to the position information of the start touch point and the position information of the end touch point . In this way, the position where the user needs to acquire the text information can be determined effectively and accurately.

According to the first aspect, or according to any one of the second implementation manner to the fifth implementation manner of the first aspect, in the seventh implementation manner of the text extraction method, the text extraction method may also The method includes: loading an area selection marking layer in response to the touch operation; and determining the touch area based on a confirming operation of the area selection marking layer. In this way, the selection of the touch area can be made more accurate, thereby further improving the accuracy of the position of the text information.

According to a sixth implementation manner of the first aspect, in an eighth implementation manner of the text extraction method, the determining according to the position information of the start touch point and the position information of the end touch point The touch area may include: in the case that the start touch point and the end touch point correspond to the same text line, according to the first touch point between the start touch point and the end touch point. an area, determining the touch area. In this way, text information can be accurately obtained within the same text line.

According to a sixth implementation manner of the first aspect, in a ninth implementation manner of the text extraction method, the determining according to the position information of the start touch point and the position information of the end touch point The touch area may include: in the case that the start touch point and the end touch point correspond to adjacent text lines, according to the difference between the start touch point and the right border of the touch screen. The second area, and the third area between the end touch point and the left border of the touch screen, determine the touch area. In this way, text information can be accurately obtained in adjacent text lines.

According to a sixth implementation manner of the first aspect, in a tenth implementation manner of the text extraction method, the determining according to the position information of the start touch point and the position information of the end touch point The touch area includes: when the text lines corresponding to the start touch point and the end touch point are separated by one or more text lines, according to the start touch point and the touch screen. The fourth area between the right borders, the fifth area between the text line corresponding to the start touch point and the text line corresponding to the end touch point, and the end touch point and the left border of the touch screen The sixth area between them determines the touch area. In this way, text information can be accurately acquired in a large range.

According to any one of the eighth implementation manner to the tenth implementation manner of the first aspect, in an eleventh implementation manner of the text extraction method, the position information according to the starting touch point and the position information of the end touch point, determining the touch area may include: moving the start touch point to the positive y-axis and negative x-axis of the touch screen by a first distance, and obtaining the adjusted move the end touch point to the positive x-axis and negative y-axis of the touch screen by a second distance to obtain the adjusted end touch point; The position information of the point and the adjusted position information of the end touch point are used to determine the touch area. In this way, the touch area can be slightly enlarged, the influence of missing text selection caused by the inconsistency between the user's visual touch point and the actual touch point can be reduced, and the accuracy of the position marking can be improved.

In a second aspect, embodiments of the present disclosure provide a text extraction device, including: a first acquisition module, used for acquiring a touch area in response to a touch operation on a touch screen; and an extraction module, used for optical character recognition (OCR) The technology extracts the first text information in the touch area acquired by the first acquisition module; the determination module is used to determine the target matching the touch area from one or more textual controls displayed on the touch screen a control; a second acquisition module for acquiring second text information from the target control determined by the determination module; an adjustment module for extracting the text information extracted by the extraction module based on the second text information acquired by the second acquisition module The first text information is adjusted to obtain third text information.

According to the second aspect, in a first implementation manner of the text extraction apparatus, the determining module includes: a first acquiring unit, configured to acquire the relationship between each textual control displayed on the touch screen and the touch area cross-merger ratio; a first determining unit, configured to determine the target control based on the cross-merger ratio.

According to the second aspect, or the first implementation manner of the second aspect, in a second implementation manner of the text extraction apparatus, the adjustment module includes: a comparison unit, configured to compare the first text information with the all The characters in the second text information corresponding to the same position on the touch screen are compared; the second determination unit is used to compare the characters in the first text information and the second text information corresponding to the same characters on the touch screen. A character whose position and content are inconsistent is determined as a target character; a first replacement unit is used to replace the target character in the first text information with the target character in the second text information corresponding to the same position on the touch screen. character to obtain the third text information.

According to a second implementation manner of the second aspect, in a third implementation manner of the text extraction apparatus, the first replacement unit is further configured to: according to the number of the target characters and the amount of the first text information The number of characters to determine the matching rate; when the matching rate is greater than the first threshold, replace the target character in the first text information with the second text information corresponding to the target character at the same position on the touch screen characters in, the third text information is obtained.

According to the second aspect, or the first implementation manner of the second aspect, in a fourth implementation manner of the text extraction apparatus, the adjustment module further includes: a detection unit configured to detect the second text information in the Whether there is a character set that meets the preset format; an extraction unit, configured to extract a character set that meets the preset format from the second text information when there is a character set that meets the preset format in the second text information. A character set in a preset format; and a second replacement unit, configured to replace the first text information with the extracted character set to obtain the third text information.

According to a fourth implementation manner of the second aspect, in a fifth implementation manner of the text extraction apparatus, the apparatus further includes: a service module, configured to provide and the preset according to the third text information The character set of the format corresponds to the service.

According to the second aspect, or any implementation manner of the above second aspect, in a sixth implementation manner of the text extraction apparatus, the first acquiring module includes: a second acquiring unit, configured to respond to the For touch operation, the position information of the start touch point and the position information of the end touch point are obtained; the third determination unit is used for obtaining the position information of the start touch point and the end touch point according to the position information of the start touch point and the end touch point. to determine the touch area.

According to the second aspect, or according to any one of the second implementation manner to the fifth implementation manner of the second aspect, in a seventh implementation manner of the text extraction apparatus, the first acquiring module It also includes: a loading unit for loading the area selection marking layer in response to the touch operation; and a fourth determining unit for determining the touch area based on the confirmation operation of the area selection marking layer. .

According to a sixth implementation manner of the second aspect, in an eighth implementation manner of the text extraction apparatus, the third determining unit is further configured to: select between the start touch point and the end touch point In the case of corresponding to the same text line, the touch area is determined according to the first area between the start touch point and the end touch point.

According to a sixth implementation manner of the second aspect, in a ninth implementation manner of the text extraction apparatus, the third determining unit is further configured to: at the start touch point and the end touch point In the case of corresponding adjacent text lines, according to the second area between the start touch point and the right border of the touch screen, and the third area between the end touch point and the left border of the touch screen area to determine the touch area.

According to a sixth implementation manner of the second aspect, in a tenth implementation manner of the text extraction apparatus, the third determining unit is further configured to: at the start touch point and the end touch point When the corresponding text lines are separated by one or more text lines, according to the fourth area between the starting touch point and the right border of the touch screen, the text line corresponding to the starting touch point and the The end touch point corresponds to the fifth area between the text lines, and the sixth area between the end touch point and the left border of the touch screen, to determine the touch area.

According to any one of the eighth implementation manner to the tenth implementation manner of the second aspect, in the tenth implementation manner of the text extraction apparatus, the third determining unit is further configured to: Move the starting touch point to the positive y-axis and negative x-axis of the touch screen by a first distance to obtain the adjusted starting touch point; move the ending touch point to the positive and negative x-axis of the touch screen The y-axis is moved in the negative direction by a second distance to obtain the adjusted end touch point; the touch area is determined according to the adjusted position information of the start touch point and the adjusted position information of the end touch point.

In a third aspect, an embodiment of the present disclosure provides a terminal device, which can execute the first aspect or one or more of the text extraction methods in multiple possible implementations of the first aspect.

In a fourth aspect, embodiments of the present disclosure provide a computer program product, comprising computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are stored in an electronic When running in the device, the processor in the electronic device executes the first aspect or one or more of the text extraction methods in the multiple possible implementation manners of the first aspect.

In the embodiment of the present disclosure, the first text information with the correct text position is accurately extracted by the OCR technology, the second text information with the correct text content is obtained by the control word extraction technology, and the text is analyzed based on the second text information with the correct text content. By adjusting the first text information with the accurate position, the third text information with the accurate position and correct content can be obtained conveniently and quickly.

These and other aspects of the present disclosure will be more clearly understood in the following description of the embodiment(s).

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure, and together with the description, serve to explain the principles of the disclosure.

FIG. 1 shows a schematic diagram of an implementation environment of a text extraction method provided by an embodiment of the present disclosure;

Fig. 2a, Fig. 2b and Fig. 2c respectively show exemplary schematic diagrams of the application interface displayed by the touch screen;

Fig. 3a, Fig. 3b, Fig. 3c, Fig. 3d and Fig. 3e show exemplary schematic diagrams of the touch area, respectively;

FIG. 4 shows a schematic structural diagram of a terminal device 200 according to an embodiment of the present disclosure;

5 shows a flowchart of a text extraction method according to an embodiment of the present disclosure;

FIG. 6 shows a schematic structural diagram of a text extraction apparatus according to an embodiment of the present disclosure.

Detailed ways

Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numbers in the figures denote elements that have the same or similar functions. While various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.

In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following detailed description. It will be understood by those skilled in the art that the present disclosure may be practiced without certain specific details. In some instances, methods, means, components and circuits well known to those skilled in the art have not been described in detail so as not to obscure the subject matter of the present disclosure.

FIG. 1 shows a schematic diagram of an implementation environment of a text extraction method provided by an embodiment of the present disclosure. Referring to FIG. 1 , the implementation environment includes a touch medium 100 and a terminal device 200 . The touch medium may include a stylus 101, a user's finger 102, and the like. The terminal device 200 can be any terminal device with a touch screen, and the terminal device 200 includes but is not limited to mobile phones, tablet computers, notebook computers, televisions, laptop computers, desktop computers, mobile phones, multimedia players, e-readers, Smart vehicle devices, smart home appliances, artificial intelligence devices, wearable electronic devices (such as smart watches, etc.), IoT devices, virtual reality/augmented reality/mixed reality devices, etc.

The terminal device 200 can install various applications, such as instant messaging applications, e-commerce applications, game applications, social networking applications, community applications, news applications, audio playback applications, video playback applications, live broadcast applications, browser applications, travel applications, and financial applications. , sports application, shooting application, image processing application, audio and video processing reference, reading application, takeaway application, recipe application, navigation application, traffic ticketing application, information recording application, mailbox application, health care application, resource management application, etc. The application installed on the terminal device 200 may be an independent application or an embedded application, that is, a small program.

In some possible implementations, the touch screen of the terminal device 200 may display an application interface, and the application interface may include one or more textual controls. The textual control may be used to represent a control that can obtain text content, for example, a text display control, a text input control, and the like. For example, the textual control can be a chat box in an instant messaging application, an input box in an information recording application, an area for displaying e-books in a reading application, an area for displaying the text content of a recipe in a recipe application, and an area for displaying text in a news application. content, the area where the text content is displayed in the browser application, etc. The text information extracted from the textual controls includes but is not limited to user names, passwords, links, words, sentences, paragraphs, articles and other contents. The text information extracted from the textual controls includes but is not limited to the forms of symbols, numbers, Chinese, English, Japanese, Korean, Spanish, German, French, and the like. The embodiments of the present disclosure do not limit the applications displayed in the terminal device 200, the textual controls included in the applications, and the textual information extracted from the textual controls.

Fig. 2a, Fig. 2b and Fig. 2c respectively show exemplary schematic diagrams of application interfaces displayed on the touch screen. In one example, the touch screen of the terminal device 200 may display the interface of the instant messaging application shown in FIG. 2a. Referring to Fig. 2a, the textual controls in the interface of the instant messaging application may include a chat information display box and a chat information input box. For example, the text content obtained from the chat information display box can include "I bought a ticket for the drama on the 10th, let's watch it together", "Okay, okay", "Who are the actors?", "The above The link has a detailed introduction, please take a look", etc.; the text information obtained from the chat information input box can include "OK, let me take a look" and so on. In yet another example, the touch screen of the terminal device 200 may display the interface of the community application shown in FIG. 2b. Referring to Fig. 2b, the textual controls in the interface of the community application may include a text display area and a search box in the display box. For example, the text information obtained from the text display area can include "Help recommend a three-day tour guide in Hangzhou", "The first day...end", "The introduction of attractions can be referred to...", "Tickets for attractions can be clicked" ……"Wait. The text information obtained from the search box may include "Hangzhou Travel Notes" (not shown) and the like. In yet another example, the touch screen of the terminal device 200 may display the interface of the browser application shown in FIG. 2c. Referring to Fig. 2c, the textual controls in the interface of the browser application may include a text display area. For example, the text information obtained from the text display area may include "XL... adjustable" and "chip introduction".

In a possible implementation manner, the user may perform touch operations (for example, operations such as clicking, double-clicking, sliding, two-finger pressing, etc.) on the touch screen of the terminal device 200 through touch media such as the stylus 101 and the user's finger 102 . . The terminal device 200 may include acquiring the touch area in response to the above touch operation.

3a, 3b, 3c, 3d, and 3e show exemplary schematic diagrams of the touch area, respectively.

In a possible implementation manner, the touch operation performed by the user may be used to determine the start touch point and the end touch point, and the terminal device may determine the touch area based on the start touch point and the end touch point.

Referring to FIG. 3 a , based on the application interface shown in FIG. 2 c , the user can perform a two-finger pressing operation on the touch screen of the terminal device 200 with a finger. In response to the two-finger pressing operation, the terminal device 200 can acquire position information of two touch points (referred to as the start touch point and the end touch point). The terminal device 200 divides the interface into multiple text lines through the OCR technology. The terminal device 200 can determine the text line corresponding to the starting touch point (denoted as the first text line) and the text line corresponding to the ending touch point (denoted as the first text line) according to the position information of the start touch point and the position information of the end touch point. for the second line of text). As shown in FIG. 3a, the first text line and the second text line are the same text line, and the terminal device 200 may determine the touch area according to the area between the start touch point and the end touch point (referred to as the first area).

Referring to Fig. 3b, based on the application interface shown in Fig. 2c, the user can perform a sliding operation on the touch screen of the terminal device 200 with a finger. The terminal device 200 may determine the start touch point and the end touch point according to the start point and end point of the sliding operation, and determine the first text line and the second text line. As shown in FIG. 3b, the first text line and the second text line are the same text line, and the terminal device 200 may determine the touch area according to the first area between the start touch point and the end touch point.

Referring to FIG. 3c, based on the application interface shown in FIG. 2c, the user can perform two clicks (single-click or double-click, etc.) operations on the touch screen of the terminal device 200 with a finger. The terminal device 200 may determine the start touch point and the end touch point according to the detected two click points, and determine the first text line and the second text line. As shown in FIG. 3c , the first text line and the second text line are adjacent text lines, and the terminal device 200 can end the touch according to the area between the starting touch point and the right border of the touch screen (referred to as the second area) and ending the touch The area between the point and the left border of the touch screen (denoted as the third area) determines the touch area.

Referring to FIG. 3d, based on the application interface shown in FIG. 2c, the user can perform a sliding operation on the touch screen of the terminal device 200 with a finger. The terminal device 200 may determine the start touch point and the end touch point according to the start point and end point of the sliding operation, and determine the first text line and the second text line. As shown in FIG. 3d, two text lines are separated between the first text line and the second text line. Refer to the second area), the area between the first text line and the second text line (marked as the fifth area), and the area between the end touch point and the left border of the touch screen (marked as the sixth area, refer to The third area) determines the touch area.

In a possible implementation manner, the first area, the second area, the third area, the fourth area, the fifth area and the sixth area may be rectangular areas. Certainly, the first to sixth regions may also be regions of other shapes, such as elliptical regions, trapezoidal regions, hexagonal regions, and octagonal regions, which are not limited in the present disclosure.

It should be noted that, in the embodiment of the present disclosure, the start touch point and the end touch point are determined based on their relative positions on the touch screen, which are consistent with the order of the text content. For example, when the first text line and the second text line are the same text line, the start touch point is located to the left of the end touch point; when the first text line and the second text line are different text lines , the start touch point is above the end touch point.

In a possible implementation manner, the touch operation can also be used to trigger area selection. In response to the touch operation, the terminal device may load the area selection marker layer, and determine the touch area based on the confirmation operation of the area selection marker layer.

Referring to FIG. 3e, based on the application interface shown in FIG. 2c, the terminal device 200 displays the area selection marker layer in response to touch operations such as long-pressing the screen and triggering the marker control. The user can adjust the position of the region selection marker layer. After the adjustment is completed, the user can click the "√" control to perform the confirmation operation of selecting the mark layer in the confirmation area. In response to the confirmation operation of the area selection marker layer, the terminal device 300 may determine the touch area according to the position of the area selection marker layer. In this way, the selection of the touch area can be made more accurate, thereby further improving the accuracy of the position of the text information.

Referring to FIG. 2a and FIG. 2b, it can be known that the touch screen of the terminal device 200 may display one textual control, or may simultaneously display multiple textual controls. The terminal device 200 may determine all textual controls currently displayed on the touch screen when detecting a touch operation on the touch screen. Afterwards, the terminal device 200 may determine a target control matching the touch area from these textual controls. In one example, the terminal device 200 may acquire the intersection ratio of each textual control displayed on the touch screen and the touch area, and determine the target control based on the intersection ratio. For example, the textual control with the largest corresponding union ratio is determined as the target control. It can be understood that, the position of the textual control displayed on the touch screen of the terminal device 200 can be obtained from the application currently displayed on the touch screen.

After the terminal device 200 acquires the touch area, the first text information in the touch area can be extracted by using the OCR technology. After determining the target widget, the terminal device 200 can directly acquire the second text information from the target widget. Since the first text information has more accurate location information, and the second text information has more accurate text content, the terminal device 200 can adjust the first text information based on the second text information, and update the first text information that is not very accurate. accurate part, so as to obtain the third text information with accurate position and accurate content.

For example, based on Fig. 3a, the terminal device 200 extracts the first text information "Integrated MOSFET switching power of 100mQ", and obtains the second text information "XL 1509-3.3E1 5G base station power chip features 2A continuous output current 8- The 30V wide operating voltage input integrates a 100mΩ MOSFET switching power tube and the output is adjustable from 18-28V". The terminal device 200 uses the second text information to adjust the first text information, and obtains the third text information "Integrated MOSFET switching power of 100 mΩ".

FIG. 4 shows a schematic structural diagram of a terminal device 200 according to an embodiment of the present disclosure.

The terminal device 200 may include a processor 210, an external memory interface 220, an internal memory 221, a USB interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 251, and a wireless communication module 252 , audio module 270, speaker 270A, receiver 270B, microphone 270C, headphone jack 270D, sensor module 280, buttons 290, motor 291, indicator 292, camera 293, display screen 294, and SIM card interface 295 and so on. The sensor module 280 may include a touch sensor 280K, (of course, the terminal device 200 may also include other sensors, such as a gyroscope sensor, an acceleration sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a pressure sensor, a distance sensor, a magnetic sensor, an environmental sensor light sensor, air pressure sensor, bone conduction sensor, etc., not shown in the figure).

The processor 210 may include one or more processing units, for example, the processor 210 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or Neural-network Processing Unit (NPU) Wait. Wherein, different processing units may be independent devices, or may be integrated in one or more processors. The controller may be the nerve center and command center of the terminal device 200 . The controller can generate an operation control signal according to the instruction operation code and timing signal, and complete the control of fetching and executing instructions.

A memory may also be provided in the processor 210 for storing instructions and data. In some embodiments, the memory in processor 210 is cache memory. The memory may hold instructions or data that have just been used or recycled by the processor 210 . If the processor 210 needs to use the instruction or data again, it can be called directly from the memory. Repeated accesses are avoided, and the waiting time of the processor 210 is reduced, thereby improving the efficiency of the system.

The processor 210 may execute the text extraction method provided by the embodiment of the present disclosure, so as to conveniently, quickly and accurately extract the text information required by the user. The processor 210 may include different devices. For example, when a CPU and a GPU are integrated, the CPU and the GPU may cooperate to execute the text extraction method provided by the embodiments of the present disclosure. for faster processing efficiency.

Display screen 294 is used to display images, videos, and the like. Display screen 294 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light). emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diode (quantum dot light emitting diodes, QLED) and so on. In some embodiments, the terminal device 200 may include one or N display screens 294 , where N is a positive integer greater than one. The display screen 294 may be used to display information entered by or provided to the user as well as various graphical user interfaces (GUIs). For example, display 294 may display photos, videos, web pages, or documents, and the like. As another example, display 294 may display a graphical user interface. The GUI includes a status bar, a hideable navigation bar, a time and weather widget, and an application icon, such as a browser icon. The status bar includes operator name (eg China Mobile), mobile network (eg 4G), time and remaining battery. The navigation bar includes a back button icon, a home button icon, and a forward button icon. In addition, it can be understood that, in some embodiments, the status bar may further include a Bluetooth icon, a Wi-Fi icon, an external device icon, and the like. It can also be understood that, in other embodiments, the graphical user interface may further include a Dock bar, and the Dock bar may include commonly used application icons and the like. After the processor 210 detects a touch event of the user's finger (or stylus, etc.) on an application icon, in response to the touch event, the user interface of the application corresponding to the application icon is opened and displayed on the display 294 The user interface of the application.

In the embodiment of the present disclosure, the display screen 294 may be an integrated flexible display screen, or a spliced display screen composed of two rigid screens and a flexible screen located between the two rigid screens, etc. Do limit.

After the processor 210 executes the text extraction method provided by the embodiment of the present disclosure, the terminal device 200 can control the display screen 294 to display a corresponding graphical user interface, such as the application interface shown in FIG. 2a, FIG. 2b, and FIG. 3b, the touch area shown in FIG. 3c and FIG. 3d, and the area selection marking layer shown in FIG. 3e.

Camera 293 (front or rear camera, or a camera that can be both front and rear camera) is used to capture still images or video. Generally, the camera 293 may include a photosensitive element such as a lens group and an image sensor, wherein the lens group includes a plurality of lenses (convex or concave) for collecting the light signal reflected by the object to be photographed, and transmitting the collected light signal to the image sensor . The image sensor generates an original image of the object to be photographed according to the light signal.

Internal memory 221 may be used to store computer executable program code, which includes instructions. The processor 210 executes various functional applications and data processing of the terminal device 200 by executing the instructions stored in the internal memory 221 . The internal memory 221 may include a storage program area and a storage data area. The storage program area may store operating system, code of application programs (such as camera application, WeChat application, etc.), and the like. The storage data area may store data created during the use of the terminal device 200 (such as images and videos captured by the camera application) and the like.

The internal memory 221 may also store one or more computer programs 1310 corresponding to the text extraction method provided by the embodiment of the present disclosure. The one or more computer programs 1304 are stored in the aforementioned memory 221 and configured to be executed by the one or more processors 210, and the one or more computer programs 1310 include instructions that may be used to perform the execution of FIG. 5 For each step in the corresponding embodiment, the computer program 1310 may include a first acquisition module, an extraction module, a determination module, a second acquisition module and an adjustment module, wherein the first acquisition module is used to respond to touch operations on the touch screen , obtains the touch area; the extraction module is used to extract the first text information in the touch area obtained by the first acquisition module through the optical character recognition OCR technology; the determination module is used to extract the textual information displayed on the touch screen from the Among the controls, a target control that matches the touch area obtained by the first obtaining module is determined, and the textual control is used to represent a control that can obtain text content; the second obtaining module is used for obtaining from the determining module. The second text information is obtained from the determined target control; the adjustment module is configured to adjust the first text information extracted by the extraction module based on the second text information obtained by the second obtaining module to obtain third text information. When the code of the text extraction method stored in the internal memory 221 is executed by the processor 210, the processor 210 may control the display screen to display third text information.

In addition, the internal memory 221 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (UFS), and the like.

Certainly, the code of the text extraction method provided by the embodiment of the present disclosure may also be stored in an external memory. In this case, the processor 210 may execute the code of the text extraction method stored in the external memory through the external memory interface 220 .

The functions of the touch sensor 280K in the sensor module 280 are described below.

Touch sensor 280K, also called "touch panel". The touch sensor 280K may be disposed on the display screen 294, and the touch sensor 280K and the display screen 294 form a touch screen, also called a "touch screen". The touch sensor 280K is used to detect touch operations on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to touch operations may be provided via display screen 294 . In the embodiment of the present disclosure, the user can perform the touch operations shown in FIGS. 3 a , 3 b , 3 c and 3 d on the touch screen, and the processor can obtain the touch area according to these touch operations.

Exemplarily, the display screen 294 of the terminal device 200 displays a main interface, and the main interface includes icons of multiple applications (such as instant messaging applications, browser applications, etc.). The user clicks the icon of the instant messaging application in the main interface through the touch sensor 280K, and triggers the processor 210 to start the instant messaging application. The display screen 294 displays an interface of an instant communication application, such as a login interface or a chat interface.

The wireless communication function of the terminal device 200 may be implemented by the antenna 1, the antenna 2, the mobile communication module 251, the wireless communication module 252, the modulation and demodulation processor, the baseband processor, and the like.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 200 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, the antenna 1 can be multiplexed as a diversity antenna of the wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 251 may provide a wireless communication solution including 2G/3G/4G/5G, etc. applied on the terminal device 200 . The mobile communication module 251 may include at least one filter, switch, power amplifier, low noise amplifier (LNA) and the like. The mobile communication module 251 can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 251 can also amplify the signal modulated by the modulation and demodulation processor, and then turn it into an electromagnetic wave for radiation through the antenna 1 . In some embodiments, at least part of the functional modules of the mobile communication module 251 may be provided in the processor 210 . In some embodiments, at least part of the functional modules of the mobile communication module 251 may be provided in the same device as at least part of the modules of the processor 210 . In this embodiment of the present disclosure, the mobile communication module 251 may also be used for information interaction with other terminal devices.

The modem processor may include a modulator and a demodulator. Wherein, the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and passed to the application processor. The application processor outputs sound signals through audio devices (not limited to the speaker 270A, the receiver 270B, etc.), or displays images or videos through the display screen 294 . In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modulation and demodulation processor may be independent of the processor 210, and may be provided in the same device as the mobile communication module 251 or other functional modules.

The wireless communication module 252 can provide applications on the terminal device 200 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellites Wireless communication solutions such as global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared technology (IR). The wireless communication module 252 may be one or more devices integrating at least one communication processing module. The wireless communication module 252 receives electromagnetic waves via the antenna 2 , frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 210 . The wireless communication module 252 can also receive the signal to be sent from the processor 210 , perform frequency modulation on the signal, amplify the signal, and then convert it into an electromagnetic wave for radiation through the antenna 2 . In this embodiment of the present disclosure, the wireless communication module 252 is configured to transmit data with other terminal devices under the control of the processor 210. For example, when the processor 210 executes the text extraction method provided by the embodiment of the present disclosure, the processor can control the The wireless communication module 252 sends a service request to other terminal devices, and can also receive service results provided by other terminal devices based on the above-mentioned service request. For example, sending a web page access request to other terminal devices, and receiving web page content provided by other terminal devices.

In addition, the terminal device 200 may implement audio functions through an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, an earphone interface 270D, an application processor, and the like. Such as music playback, recording, etc.

It should be understood that, in practical applications, the terminal device 200 may include more or less components than those shown in FIG. 4 , which is not limited by the embodiment of the present disclosure. The illustrated terminal device 200 is only an example, and the terminal device 200 may have more or fewer components than those shown in the figures, may combine two or more components, or may have a different configuration of components. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

The workflow of the software and hardware of the terminal device 200 is exemplarily described below with reference to the application interface shown in FIG. 3a.

In the case where the terminal device 200 displays the application interface shown in FIG. 3a on the display screen 294, when the user touches both sides of the information "Integrated MOSFET switching power of 100mΩ", the touch sensor 280K receives the touch operation, and the corresponding Hardware terminals are sent to the kernel layer. The kernel layer processes touch operations into raw input events (including touch coordinates, timestamps of touch operations, etc.). Raw input events are stored at the kernel layer. The application framework layer obtains the original input event from the kernel layer, and identifies the application corresponding to the input event. Taking the touch operation as a two-finger pressing operation and the application corresponding to the two-finger pressing operation being a text extraction application as an example, the text extraction application invokes the interface of the application framework layer to start the text extraction application. The text extraction application acquires the touch area shown in FIG. 3a in response to the above-mentioned two-finger pressing operation; extracts the first text information in the touch area "integrated with 100mQ MOSFET switching power" through OCR technology; displays the display screen 294 In the textual control of the 5G base station, determine the target control that matches the touch area shown in Figure 3a; obtain the second text information from the target control "XL 1509-3.3E1 5G base station power chip features 2A continuous output current 8-30V wide The working voltage input integrates a 100mΩ MOSFET switching power tube output 18-28V adjustable"; based on the second text information, the first text information is adjusted to obtain the third text information "Integrated 100mΩ MOSFET switching power".

FIG. 5 shows a flowchart of a text extraction method according to an embodiment of the present disclosure. The method may be performed by a terminal device, such as the terminal device 200 shown in FIG. 4 . As shown in Figure 5, the method may include:

Step S601, in response to a touch operation on the touch screen, obtain a touch area.

Step S602, extracting the first text information in the touch area by using an optical character recognition (OCR) technology.

Step S603, determining a target control matching the touch area from one or more textual controls displayed on the touch screen.

Wherein, the textual control is used to represent a control that can obtain text content.

Step S604, acquiring second text information from the target control.

Step S605: Adjust the first text information based on the second text information to obtain third text information.

In step S601, the touch operation may include a click operation, a sliding operation, and a pressing operation. The click operation may include two click operations, two double click operations, one click operation and one double click operation, or one double click operation and one click operation, and the like. The sliding operation may include a single-finger sliding operation, a multi-finger sliding operation, and the like. The pressing operation may include a single-finger pressing operation, a multi-finger pressing operation (eg, a two-finger pressing operation), and the like. The touch operation may be performed by a stylus, a user's finger, or a user's knuckle, or the like. The terminal device can acquire the touch area in response to the touch operation on the touch screen. The touch area can accurately mark the position of the text information that the user needs to obtain. In one example, considering that the text has the feature of being divided into lines, the touch area may include one or more rectangular areas. The touch area can also be other areas that can accurately mark the position.

In a possible implementation manner, step S601 may include: in response to the touch operation, acquiring the position information of the start touch point and the position information of the end touch point; according to the position of the start touch point information and the position information of the end touch point to determine the touch area.

A touch operation may generate two or more touch points. In the embodiment of the present disclosure, the start touch point and the end touch point need to be determined from the two or more touch points. The start touch point may be used to mark the start point of the text information that the user wants to obtain, and the end touch point may be used to mark the end point of the text information that the user wants to obtain. The starting touch point and the ending touch point are determined according to the order of the text content in the text information, not the order in which the touch points are generated. For a sliding operation that slides from bottom left to top right, the starting touch point is the last touch point generated by the sliding operation, and the ending touch point is the first touch point generated by the sliding operation. For example, the terminal device determines the distance between the text line corresponding to each touch point generated by the touch operation and the upper boundary of the touch screen, and determines the touch point with the smallest distance as the first touch point. Similarly, the terminal device determines the distance between the text line corresponding to each touch point generated by the touch operation and the lower boundary of the touch screen, and determines the touch point with the smallest distance as the second touch point. In the case that there is one first touch point, the terminal device may determine the first touch point as the starting touch point; in the case that there are multiple first touch points, the terminal device may determine the first touch point with the left border of the touch screen The smallest first touch point is determined as the initial touch point. In the case that there is one second touch point, the terminal device can determine the second touch point as the end touch point; in the case that there are multiple second touch points, the terminal device can determine the distance from the right border of the touch screen The smallest second touch point is determined as the end touch point.

The position information of the start touch point and the position information of the end touch point are determined based on the touch screen. In one example, the lower left corner of the touch screen may be determined as the coordinate origin, the positive right side of the coordinate origin may be used as the positive direction of the x-axis, the positive left side of the coordinate origin may be used as the negative direction of the y-axis, and the positive side of the coordinate origin may be used as the positive direction of the y-axis. As the positive direction of the y-axis, the negative direction of the y-axis is directly below the origin of the coordinates. In this way, the position information of the starting touch point and the position information of the ending touch point can be represented by x and y.

In an example, in the case that the start touch point and the end touch point correspond to the same text line, the terminal device can an area, determining the touch area. Taking the first area as a rectangular area as an example, the terminal device can determine the left boundary of the first area according to the starting touch point, determine the right boundary of the first area according to the ending touch point, and determine the rectangle area according to the lines divided by the OCR technology. The upper and lower boundaries are determined, and the determined rectangular area between the upper, lower, left and right boundaries is determined as the touch area.

In an example, in the case that the starting touch point and the ending touch point correspond to adjacent text lines, the terminal device can The second area, and the third area between the end touch point and the left border of the touch screen, determine the touch area.

The manner of determining the second area and the third area may refer to the manner of determining the first area, which will not be repeated here. It can be understood that the text information in the second area should be arranged before the text information in the third area. In one example, the touch area can be obtained by splicing the third area after the second area.

In an example, when the text lines corresponding to the start touch point and the end touch point are separated by one or more text lines, the terminal device can The fourth area between the right border of the touch screen, the fifth area between the text line corresponding to the start touch point and the text line corresponding to the end touch point, and the end touch point and the left side of the touch screen The sixth area between the boundaries determines the touch area.

The manner of determining the fourth area, the fifth area and the sixth area may refer to the manner of determining the first area, which will not be repeated here. In an example, the fifth area may be divided into one or more sub-areas, and each sub-area corresponds to a text line. Then, the sub-regions of the fifth region are sequentially spliced after the fourth region, and the sixth region is spliced after the sub-regions corresponding to the last text line of the fifth region to obtain a touch-sensitive region.

In the embodiment of the present disclosure, the position of the text information that the user needs to acquire can be accurately marked by the above method.

In a possible implementation manner, before acquiring the position information of the start touch point and the position information of the end touch point, the terminal device may first adjust the positions of the start touch point and the end touch point. In one example, the terminal device may move the starting touch point to the positive y-axis and negative x-axis of the touch screen by a first distance to obtain the adjusted starting touch point; move the ending touch point by a first distance. The control point is moved a second distance in the positive direction of the x-axis and the negative direction of the y-axis of the touch screen to obtain the adjusted end touch point. After that, the terminal device may determine the touch area according to the adjusted position information of the start touch point and the adjusted position information of the end touch point.

The first distance may include a first x-axis distance and a first y-axis distance. The first x-axis distance and the first y-axis distance may be the same or different. The first x-axis distance and the first y-axis distance may be set as required, for example, may be determined according to one or more of the size of the touch screen, the height of the text line, and the size of the text, which is not limited in the present disclosure. For example, the first x-axis distance may be 0.5 cm, and the first y-axis distance may be 0.5 cm. The second distance can refer to the first distance, which will not be repeated here.

In the embodiment of the present disclosure, by adjusting the position of the starting touch point in the positive direction of the y-axis and the negative direction of the x-axis, and adjusting the position of the end touch point in the negative direction of the y-axis and the positive direction of the x-axis, the touch can be slightly enlarged. area, reduce the impact of missing text selection caused by the inconsistency between the user's visual touch point and the actual touch point, and improve the accuracy of the position mark.

In a possible implementation manner, after determining the text line corresponding to the start touch point and the end touch point, the terminal device may take an area of an additional text line up and down, respectively, and add it to the touch area . In this way, the touch area can be effectively expanded, so as to better reduce the influence of missing text selection caused by the inconsistency between the user's visual touch point and the actual touch point, and improve the accuracy of the position mark. In particular, the effect of touch operations performed by fingers or knuckles is more pronounced.

In step S602, the first text information may represent the text information in the touch area extracted by the OCR technology. In an example, the terminal device can perform binarization processing, noise removal, tilt correction, line branch processing, character segmentation, character recognition, and panel restoration on the touch area (even if the text content is recognized, it is still in accordance with the original touch area) The text content displayed in the text is arranged in the same way, keeping the paragraphs of the text unchanged, the position unchanged, and the order unchanged).

In step S603, the textual control may be used to indicate a space where text content can be obtained. In one example, textual controls may include text presentation controls and text entry controls. For example, the textual control can be a short message display box, an instant messaging message display box, a memo, a notepad, and the like. One or more textual controls can be displayed simultaneously on the touch screen. It should be noted that the textual controls displayed on the touch screen include textual controls that are not fully displayed on the touch screen.

In a possible implementation manner, the terminal device may acquire the intersection ratio of each textual control displayed on the touch screen and the touch area; and determine the target control based on the intersection ratio. In one example, the terminal device may determine the textual control with the largest intersection ratio with the touch area as the target control. In yet another example, the terminal device may determine a textual control whose intersection ratio with the touch area is the largest and whose intersection ratio is greater than a specified threshold as the target control. The specified threshold can be set as required, for example, can be set to 85%, 90%, etc., which is not limited in the present disclosure.

In step S604, since the target control is a control that can obtain text content. Therefore, the terminal device can directly acquire the second text information from the target control. The second text information may be stored in attribute information of the target control.

In step S605, the terminal device may adjust the first text information based on the second text information, so that the incorrect text content in the first text information becomes the correct text content. Because the text position of the first text information itself is accurate. Therefore, the position of the text in the third text information obtained after the adjustment of the first text information is accurate, and the text content is correct.

In a possible implementation manner, step S605 may include: comparing the characters in the first text information and the second text information corresponding to the same position on the touch screen; , and the characters in the second text information correspond to the characters in the same position on the touch screen and the contents are inconsistent, and are determined as target characters; replace the target characters in the first text information with the characters corresponding to the target characters on the touch screen. Characters in the second text information at the same position to obtain the third text information.

In the embodiment of the present disclosure, the first text information and the second text information are aligned first, and characters in the first text information and the second text information that correspond to the same position on the touch screen are found. In an example, the first character of the first text information may be aligned with the first character of the second text information, and then the subsequent characters are compared in sequence to determine the matching rate (for example, the number of the same characters accounts for the percentage of the different characters). Quantity ratio); align the first character of the first text information with the second character of the second text information, and determine the matching rate again. By analogy, until the first character of the first text information is aligned with the last character of the second text information, the last matching rate is determined. Find the alignment position with the largest matching rate as the final alignment position. In yet another example, after a matching rate greater than a certain threshold (which can be set as required, for example, can be set to 95%, 90%, etc.) may be determined, the alignment position corresponding to the matching rate may be used as the final alignment position , no further operations are to be performed. Then, the characters corresponding to the same position on the touch screen in the first text information and the second text information are compared. Based on the comparison results, replacement processing is performed for inconsistent situations. For example, based on Fig. 3a, the terminal device 200 extracts the first text information "Integrated MOSFET switching power of 100mQ", and obtains the second text information "XL 1509-3.3E1 5G base station power chip features 2A continuous output current 8- The 30V wide operating voltage input integrates a 100mΩ MOSFET switching power tube and the output is adjustable from 18 to 28V." After the alignment process, the terminal device can determine that the characters in the first text message "Integrated 100mQ MOSFET switching power" are in turn with the first text. 2. Each character in the text message "Integrated MOSFET switching power of 100mΩ" corresponds to the same position. The terminal device may compare the characters in the same position in the first text information and the second text information. After comparison, the terminal device finds that the character "Q" in the first text information corresponding to the same position is different from the character "Ω" in the second text information. At this time, the terminal device can determine the character "Q" as the target character, so that the target character "Q" in the first text message "Integrated MOSFET switching power of 100mQ" is replaced with the character in the same position in the second text message" Ω" to get the final third text message "Integrated MOSFET switching work of 100mΩ".

Compared with the OCR technology, the text extraction method provided by the embodiments of the present disclosure can improve the correctness of the extracted text content. Compared with the control word extraction technology, the text extraction method provided by the embodiment of the present disclosure can improve the accuracy of the extracted text position, and saves the process of the user searching for the required text in the extraction result. That is to say, the text extraction method provided by the embodiments of the present disclosure can conveniently, quickly and accurately obtain the text information required by the user.

In an example, before the terminal device replaces the target character in the first text information with the character in the second text information corresponding to the target character at the same position on the touch screen, and obtains the third text information, The matching rate may be determined according to the number of the target characters and the number of characters in the first text information. Then, when the matching rate is greater than the first threshold, the terminal device replaces the target character in the first text information with a character in the second text information corresponding to the target character at the same position on the touch screen, The third text information is obtained.

The matching rate may be a ratio of the number of characters other than the target character in the first text information to the number of characters in the first text information. The first threshold may be set as required, for example, the first threshold may be 92%, 95%, etc., which is not limited in this embodiment of the present disclosure. When the matching rate is greater than the first threshold, it indicates that there is a small amount of wrongly extracted text content in the first text information, and the correctness of the first text information can be improved after adjustment. Therefore, when the matching rate is greater than the first threshold, the terminal device then replaces the target character in the first text information with a character in the second text information corresponding to the target character at the same position on the touch screen, to obtain the third text information. In the case that the matching rate is less than or equal to the first threshold, it indicates that there may be a large number of wrongly extracted text content in the first text information, which may be caused by the misalignment or inaccurate alignment of the first text information and the second text information, It may also be caused by an extraction error of one or both of the first text information and the second text information. At this time, the terminal device may re-align the first text information and the second text information or re-acquire the first text information and the second text information.

In this way, the correctness can be improved by performing character replacement when the matching rate is high.

In a possible implementation manner, step S605 may include: detecting whether a character set satisfying a preset format exists in the second text information; and a character set satisfying the preset format exists in the second text information In the case of , extract a character set that satisfies the preset format from the second text information; replace the first text information with the extracted character set to obtain the third text information.

The character set of the preset format may include a password or a link, etc., and the embodiment of the present disclosure does not limit the preset format. In an example, the terminal device may detect and extract a character set in a preset format for the second text information by using a regular expression or a natural language processing (Natural Language Processing, NLP) technology.

When the link or password is obtained by the OCR technology, it is prone to errors, and the OCR technology will break the link or the password when it wraps, which makes it difficult to accurately extract the link or password by the OCR technology. When a link or a password is obtained through the control word extraction technology, the user needs to find a specific location from the obtained text, which is a cumbersome operation. The text extraction method provided by the embodiments of the present disclosure can determine the control to which a link or a password belongs by using OCR technology, and then automatically extract the link or password from the control, which can not only ensure the integrity and location accuracy of the link or password, but also can Ensure the correctness of the link or password, and operate quickly and easily.

In a possible implementation manner, the extracted character set is used to replace the first text information, and after obtaining the third text information, the terminal device may further provide the preset text information according to the third text information The character set of the format corresponds to the service. For example, the terminal device can jump to the webpage corresponding to the link, and can also open the application corresponding to the password and jump to the corresponding details page, or copy the password and automatically jump to the corresponding details page when the corresponding application is opened.

In this way, service efficiency can be improved, and user satisfaction can be improved.

FIG. 6 shows a schematic structural diagram of a text extraction apparatus according to an embodiment of the present disclosure. As shown in FIG. 6, the apparatus 80 may include:

a first acquiring module 81, configured to acquire a touch area in response to a touch operation on the touch screen;

The extraction module 82 is used for extracting the first text information in the touch area obtained by the first obtaining module 81 through the optical character recognition OCR technology;

a determining module 83, configured to determine a target control matching the touch area from one or more textual controls displayed on the touch screen;

The second obtaining module 84 is configured to obtain second text information from the target control determined by the determining module 83;

The adjustment module 85 is configured to adjust the first text information extracted by the extraction module 82 based on the second text information acquired by the second acquisition module 84 to obtain third text information.

In a possible implementation manner, the determining module includes: a first acquiring unit, configured to acquire the intersection ratio of each textual control displayed on the touch screen and the touch area; the first determining unit, using The target control is determined based on the cross-union ratio.

In a possible implementation manner, the adjustment module includes: a comparison unit, configured to compare the characters corresponding to the same position on the touch screen in the first text information and the second text information; the second a determining unit, configured to determine the characters in the first text information and the characters in the second text information that correspond to the same position on the touch screen and have inconsistent contents as target characters; the first replacement unit is configured to replace the first A target character in a piece of text information is replaced with a character in the second text information corresponding to the target character at the same position on the touch screen to obtain the third text information.

In a possible implementation manner, the first replacement unit is further configured to: determine a matching rate according to the number of the target characters and the number of characters in the first text information; when the matching rate is greater than the first In the case of the threshold value, the target character in the first text information is replaced with a character in the second text information corresponding to the target character at the same position on the touch screen to obtain the third text information.

In a possible implementation manner, the adjustment module further includes: a detection unit for detecting whether a character set satisfying a preset format exists in the second text information; an extraction unit for When there is a character set that satisfies the preset format in the information, extract the character set that satisfies the preset format from the second text information; the second replacement unit is used to replace the character set with the extracted character set The third text information is obtained from the first text information.

In a possible implementation manner, the apparatus further includes: a service module, configured to provide a service corresponding to the character set in the preset format according to the third text information. In a possible implementation manner, the first obtaining module includes: a second obtaining unit, configured to obtain the position information of the starting touch point and the position information of the ending touch point in response to the touch operation; A third determining unit, configured to determine the touch area according to the position information of the start touch point and the position information of the end touch point.

In a possible implementation manner, the first obtaining module further includes: a loading unit, configured to load a region selection marker layer in response to the touch operation; and a fourth determination unit, configured to select a marker based on the region The confirmation operation of the layer determines the touch area.

In a possible implementation manner, the third determining unit is further configured to: in the case that the start touch point and the end touch point correspond to the same text line, according to the start touch point and the first area between the end touch point to determine the touch area.

In a possible implementation manner, the third determining unit is further configured to: in the case that the start touch point and the end touch point correspond to adjacent text lines, according to the start touch point The touch area is determined by a second area between the point and the right border of the touch screen, and a third area between the end touch point and the left border of the touch screen.

In a possible implementation manner, the third determining unit is further configured to: in the case that the text lines corresponding to the start touch point and the end touch point are separated by one or more text lines, according to The fourth area between the start touch point and the right border of the touch screen, the fifth area between the text line corresponding to the start touch point and the text line corresponding to the end touch point, and the A sixth area between the end touch point and the left border of the touch screen is used to determine the touch area.

In a possible implementation manner, the third determining unit is further configured to: move the initial touch point to the positive y-axis and negative x-axis of the touch screen by a first distance to obtain the adjusted starting touch point; moving the ending touch point to the positive x-axis and negative y-axis of the touch screen by a second distance to obtain the adjusted ending touch point; according to the adjusted starting touch point and the adjusted position information of the end touch point to determine the touch area.

Embodiments of the present disclosure provide a text extraction apparatus, comprising: a processor and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions.

Embodiments of the present disclosure provide a non-volatile computer-readable storage medium having computer program instructions stored thereon, the computer program instructions implementing the above method when executed by a processor.

Embodiments of the present disclosure provide a computer program product, including computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are stored in a processor of an electronic device When running in the electronic device, the processor in the electronic device executes the above method.

A computer-readable storage medium may be a tangible device that can hold and store instructions for use by the instruction execution device. Computer-readable storage media include, but are not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing, for example. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read-only memory (Electrically Programmable Read-Only-Memory, EPROM or flash memory), static random access memory (Static Random-Access Memory, SRAM), portable compact disk read-only memory (Compact Disc Read-Only Memory, CD - ROM), Digital Video Disc (DVD), memory sticks, floppy disks, mechanically encoded devices, such as punch cards or raised structures in grooves on which instructions are stored, and any suitable combination of the foregoing .

Computer readable program instructions or code described herein may be downloaded to various computing/processing devices from a computer readable storage medium, or to an external computer or external storage device over a network such as the Internet, a local area network, a wide area network and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer-readable program instructions from a network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .

The computer program instructions for carrying out the operations of the present disclosure may be assembly instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or in one or more source or object code written in any combination of programming languages, including object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or, may be connected to an external computer (eg, use an internet service provider to connect via the internet). In some embodiments, electronic circuits, such as programmable logic circuits, Field-Programmable Gate Arrays (FPGA), or Programmable Logic Arrays (Programmable Logic Arrays), are personalized by utilizing state information of computer-readable program instructions. Logic Array, PLA), the electronic circuitry can execute computer-readable program instructions to implement various aspects of the present disclosure.

Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer or other programmable data processing apparatus to produce a machine that causes the instructions when executed by the processor of the computer or other programmable data processing apparatus , resulting in means for implementing the functions/acts specified in one or more blocks of the flowchart and/or block diagrams. These computer readable program instructions can also be stored in a computer readable storage medium, these instructions cause a computer, programmable data processing apparatus and/or other equipment to operate in a specific manner, so that the computer readable medium on which the instructions are stored includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.

Computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process , thereby causing instructions executing on a computer, other programmable data processing apparatus, or other device to implement the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more functions for implementing the specified logical function(s) executable instructions. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.

It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in hardware (eg, circuits or ASICs (Application) that perform the corresponding functions or actions. Specific Integrated Circuit, application-specific integrated circuit)), or can be implemented by a combination of hardware and software, such as firmware.

While the invention has been described herein in connection with various embodiments, those skilled in the art will understand and understand from a review of the drawings, the disclosure, and the appended claims in practicing the claimed invention. Other variations of the disclosed embodiments are implemented. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that these measures cannot be combined to advantage.

Various embodiments of the present disclosure have been described above, and the foregoing descriptions are exemplary, not exhaustive, and not limiting of the disclosed embodiments. Numerous modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the various embodiments, the practical application or improvement over the technology in the marketplace, or to enable others of ordinary skill in the art to understand the various embodiments disclosed herein.

Claims

A text extraction method, characterized in that the method comprises:

Acquire a touch area in response to a touch operation on the touch screen;

Extract the first text information in the touch area by using optical character recognition OCR technology;

determining a target control matching the touch area from one or more textual controls displayed on the touch screen;

Obtain second text information from the target control;

The first text information is adjusted based on the second text information to obtain third text information.
The method according to claim 1, wherein the determining a target control matching the touch area from one or more textual controls displayed on the touch screen comprises:

obtaining the intersection ratio of each textual control displayed on the touch screen and the touch area;

Based on the cross-union ratio, the target control is determined.
The method according to claim 1 or 2, wherein the adjusting the first text information based on the second text information to obtain the third text information comprises:

comparing the characters corresponding to the same position on the touch screen in the first text information and the second text information;

In the first text information, the characters in the second text information that correspond to the same position on the touch screen and have inconsistent contents are determined as the target characters;

The third text information is obtained by replacing the target character in the first text information with the character in the second text information corresponding to the target character at the same position on the touch screen.
The method according to claim 3, wherein by replacing the target character in the first text information with a character in the second text information corresponding to the target character at the same position on the touch screen, the obtained The third text information includes:

Determine the matching rate according to the number of the target characters and the number of characters in the first text information;

In the case where the matching rate is greater than the first threshold, replace the target character in the first text information with the character in the second text information corresponding to the target character at the same position on the touch screen, to obtain the first Three text messages.
The method according to claim 1 or 2, wherein the adjusting the first text information based on the second text information to obtain the third text information comprises:

Detecting whether there is a character set satisfying a preset format in the second text information;

Extracting a character set that satisfies the preset format from the second text information when there is a character set that satisfies the preset format in the second text information;

The first text information is replaced with the extracted character set to obtain the third text information.
The method according to claim 5, wherein the method further comprises:

According to the third text information, a service corresponding to the character set of the preset format is provided.
The method according to any one of claims 1 to 6, wherein the acquiring a touch area in response to a touch operation comprises:

In response to the touch operation, obtain the position information of the start touch point and the position information of the end touch point;

The touch area is determined according to the position information of the start touch point and the position information of the end touch point.
The method according to any one of claims 1 to 6, wherein the acquiring a touch area in response to a touch operation comprises:

In response to the touch operation, loading a region selection marker layer;

The touch area is determined based on the confirmation operation of the area selection marking layer.
The method according to claim 7, wherein the determining the touch area according to the position information of the start touch point and the position information of the end touch point comprises:

In the case that the start touch point and the end touch point correspond to the same text line, the touch control is determined according to the first area between the start touch point and the end touch point area.
The method according to claim 7, wherein the determining the touch area according to the position information of the start touch point and the position information of the end touch point comprises:

In the case that the start touch point and the end touch point correspond to adjacent text lines, according to the second area between the start touch point and the right border of the touch screen, and the end The third area between the touch point and the left border of the touch screen determines the touch area.
The method according to claim 7, wherein the determining the touch area according to the position information of the start touch point and the position information of the end touch point comprises:

In the case that the text lines corresponding to the start touch point and the end touch point are separated by one or more text lines, according to the fourth value between the start touch point and the right border of the touch screen area, the fifth area between the text line corresponding to the start touch point and the text line corresponding to the end touch point, and the sixth area between the end touch point and the left border of the touch screen, Determine the touch area.
The method according to any one of claims 9 to 11, wherein the determining the touch area according to the position information of the start touch point and the position information of the end touch point includes: :

moving the initial touch point to the positive y-axis and negative x-axis of the touch screen by a first distance to obtain the adjusted initial touch point;

moving the end touch point to the positive x-axis and negative y-axis of the touch screen by a second distance to obtain the adjusted end touch point;

The touch area is determined according to the adjusted position information of the starting touch point and the adjusted position information of the end touch point.
A text extraction device, characterized in that the device comprises:

a first acquisition module, configured to acquire a touch area in response to a touch operation on the touch screen;

an extraction module, configured to extract the first text information in the touch area acquired by the first acquisition module through optical character recognition (OCR) technology;

a determining module, configured to determine a target control matching the touch area from one or more textual controls displayed on the touch screen;

A second obtaining module, configured to obtain second text information from the target control determined by the determining module;

An adjustment module, configured to adjust the first text information extracted by the extraction module based on the second text information acquired by the second acquisition module to obtain third text information.
The apparatus according to claim 13, wherein the determining module comprises:

a first acquiring unit, configured to acquire the intersection ratio of each textual control displayed on the touch screen and the touch area;

a first determining unit, configured to determine the target control based on the cross-union ratio.
The device according to claim 13 or 14, wherein the adjustment module comprises:

a comparison unit, configured to compare the characters corresponding to the same position on the touch screen in the first text information and the second text information;

a second determining unit, configured to determine a character in the first text information and a character in the second text information corresponding to the same position on the touch screen and with inconsistent content as a target character;

The first replacement unit is configured to replace the target character in the first text information with the character in the second text information corresponding to the target character at the same position on the touch screen to obtain the third text information.
The device according to claim 15, wherein the first replacement unit is further used for:

Determine the matching rate according to the number of the target characters and the number of characters in the first text information;

In the case where the matching rate is greater than the first threshold, replace the target character in the first text information with the character in the second text information corresponding to the target character at the same position on the touch screen, to obtain the first Three text messages.
The device according to claim 13 or 14, wherein the adjustment module further comprises:

a detection unit, configured to detect whether there is a character set satisfying a preset format in the second text information;

an extraction unit, configured to extract, from the second text information, a character set that satisfies the preset format when there is a character set that satisfies the preset format in the second text information;

The second replacement unit is configured to replace the first text information with the extracted character set to obtain the third text information.
The apparatus of claim 17, wherein the apparatus further comprises:

A service module, configured to provide a service corresponding to the character set of the preset format according to the third text information.
The device according to any one of claims 13 to 18, wherein the first obtaining module comprises:

a second obtaining unit, configured to obtain the position information of the starting touch point and the position information of the ending touch point in response to the touch operation;

A third determining unit, configured to determine the touch area according to the position information of the start touch point and the position information of the end touch point.
The device according to any one of claims 13 to 18, wherein the first obtaining module further comprises:

a loading unit, configured to load the area selection mark layer in response to the touch operation;

The fourth determination unit is configured to determine the touch area based on the confirmation operation of the area selection marking layer.
The device according to claim 19, wherein the third determining unit is further configured to:

In the case that the start touch point and the end touch point correspond to the same text line, the touch control is determined according to the first area between the start touch point and the end touch point area.
The device according to claim 19, wherein the third determining unit is further configured to:

In the case that the start touch point and the end touch point correspond to adjacent text lines, according to the second area between the start touch point and the right border of the touch screen, and the end The third area between the touch point and the left border of the touch screen determines the touch area.
The device according to claim 19, wherein the third determining unit is further configured to:

In the case that the text lines corresponding to the start touch point and the end touch point are separated by one or more text lines, according to the fourth value between the start touch point and the right border of the touch screen area, the fifth area between the text line corresponding to the start touch point and the text line corresponding to the end touch point, and the sixth area between the end touch point and the left border of the touch screen, Determine the touch area.
The device according to any one of claims 21 to 23, wherein the third determining unit is further configured to:

moving the initial touch point to the positive y-axis and negative x-axis of the touch screen by a first distance to obtain the adjusted initial touch point;

moving the end touch point to the positive x-axis and negative y-axis of the touch screen by a second distance to obtain the adjusted end touch point;

The touch area is determined according to the adjusted position information of the starting touch point and the adjusted position information of the end touch point.
A text extraction device, characterized in that it includes a processor, a memory for storing executable instructions of the processor, and a touch screen for receiving touch operations, and when the processor invokes the executable instructions, the terminal A method as claimed in any of claims 1-12 is implemented.
A non-volatile computer-readable storage medium on which computer program instructions are stored, characterized in that, when the computer program instructions are executed by a processor, the method described in any one of claims 1-12 is implemented.