TW202338664A - Word recognition system and word recognition method - Google Patents

Abstract

A word recognition system and method are provided, which comprises inputting image information into a collection module, and making the collection module generate a text screenshot; and by the recognition module receiving the text screenshot, obtaining word string data without being affected by interference of background of the image information. Therefore, it can reduce the probability of affecting the accuracy of identification.

Description

Text recognition system and method

The present invention provides a text recognition technology, particularly a text recognition system and method suitable for online image content.

With the rise of online platforms, ordinary people can easily obtain the audio and video content they need through online platforms. For example, in recent years, the live broadcast system has gradually matured. In addition to broadcasting signals through traditional channels, TV stations have also integrated online social platforms such as Youtube and Facebook to play audio and video content signals through the live broadcast tools of the online social platforms. Even some websites Road Media is also trying to introduce live broadcast rooms.

Due to the overdevelopment of online information, related industries need to rely more on this online information. For example, if industry demands such as brand media or public relations companies need to grasp social public opinion in real time, they need to capture the audio and video content of these online social platforms for analysis.

However, these audio and video contents include image data and text data, which makes the file size of these audio and video contents too large, making access difficult.

Furthermore, although the demand side of the industry can use plug-in software to capture only the text data in the audio and video content, plug-in software is known to cause background interference to the image data in the video and audio content. (noise) is very sensitive and is often affected by background interference, which affects the accuracy of recognition. On the other hand, when conventional plug-in software recognizes text, it needs to operate under extremely high contrast conditions (such as black text on a white background or white text on a black background), otherwise the recognition accuracy is extremely low. For example, if the background of the title in the image data shows gradient colors or other patterns, the recognition accuracy will drop sharply.

Therefore, how to overcome the various problems of the above-mentioned conventional technologies has become an urgent issue to be solved.

The present invention provides an online text recognition system, which includes: a collection module that receives image information to generate text screenshots; and a recognition module that communicates with the collection module to receive the text screenshots, and performs recognition operations to obtain information containing The target information of the string data.

In the aforementioned text recognition system, the collection module adopts a web crawler mode to automatically search and collect the image information published on the Internet.

In the aforementioned text recognition system, the collection module performs a screenshot action on the image information to obtain an initial screenshot, and generates the text screenshot from the initial screenshot.

In the aforementioned character recognition system, the recognition module is an artificial intelligence module in the form of optical character recognition.

The aforementioned text recognition system also includes an assembly module that is connected to the recognition module to receive the target information, and organizes the target information into advanced information, where the advanced information includes the string data and its location. Corresponding reference materials.

The present invention further provides a text recognition method, which includes: performing a collection operation through a collection module to input image information into the collection module, so that the collection module generates text snippets. Figure; and perform recognition operations through the recognition module, so that the recognition module receives the text screenshot to obtain target information including string data.

In the aforementioned text recognition method, the collection operation adopts a web crawler method to automatically search and collect the image information published on the Internet.

In the aforementioned text recognition method, the collection operation is to perform a screenshot action on the image information to obtain an initial screenshot, and generate the text screenshot from the initial screenshot.

In the aforementioned text recognition method, the recognition module is an artificial intelligence module in the form of optical text recognition.

The aforementioned text recognition method further includes receiving the target information through an assembly module to organize the target information into advanced information, where the advanced information includes the string data and its corresponding reference materials.

It can be seen from the above that the character recognition system and method of the present invention mainly obtain the required character string data through the cooperation of the collection module and the recognition module for subsequent demand-side applications. Therefore, compared with conventional plug-ins, Software, the present invention can effectively acquire character string data without being affected by the background interference of the image information, thereby reducing the chance of affecting the accuracy of recognition, and the collection module of the present invention can generate screenshots of the text, and then use the recognition The module directly recognizes the string data from the text screenshot, thus improving the recognition accuracy.

1: Text recognition system

1a:Host

10:Collect modules

101: Retrieve model

102: Processing models

103: Temporary storage area

11:Identification module

111:Training model

112:Analytical model

12: Collection module

12a:Database

2: Electronic devices

A1,A2,A3: text block

P:Initial screenshot

T1, T2, T3: Text screenshots

S30~S33: steps

Figure 1 is a schematic structural diagram of the text recognition system of the present invention.

Figure 2 is a schematic configuration diagram of the character recognition system of the present invention.

Figure 3 is a flow chart of the text recognition method of the present invention.

4A to 4C are schematic diagrams of an embodiment of the collection module of the character recognition system of the present invention during operation.

It should be noted that the structures, proportions, sizes, etc. shown in the drawings attached to this specification are only used to coordinate with the content disclosed in the specification for the understanding and reading of those familiar with the art, and are not used to limit the implementation of the present invention. Therefore, it has no technical substantive significance. Any structural modifications, changes in proportions, or adjustments in size shall still fall within the scope of this invention without affecting the effects that can be produced and the purposes that can be achieved. The technical content disclosed by the invention must be within the scope that can be covered. At the same time, terms such as "a", "first", "second", "upper" and "lower" cited in this specification are only for convenience of description and are not used to limit the scope of the present invention. Changes or adjustments in the scope of implementation and relative relationships shall be regarded as the scope within which the present invention can be implemented without substantially changing the technical content.

Figure 1 is a schematic structural diagram of the character recognition system 1 of the present invention. As shown in FIG. 1 , the character recognition system 1 includes: a collection module 10 , a recognition module 11 and a collection module 12 .

In this embodiment, as shown in Figure 2, the character recognition system 1 is installed in a host 1a, such as a server, a cloud, a computer device or a mobile device with various processors, etc.

The collection module 10 is used to collect image information, where the image information includes image data and reference materials (such as image source, screenshot time, original screen or other items, etc.).

In this embodiment, the image information is suitable for network transmission format. For example, the image information is disclosed on an online platform, and the online platform can be an online social platform, media, brand main website and other public websites, such as Facebook fan page information, Instagram business account information, LINE official account information, Youtube channel information, Google Maps business information, MOD digital audio and video information, etc.

Furthermore, the collection module 10 can adopt a video crawler mode to automatically search and extract image information published on the Internet. For example, the collection module 10 is suitable for live broadcast images.

In addition, the collection module 10 includes a capture model 101 for capturing images and a processing model 102 for preprocessing images.

The capture model 101 is used to capture the screen and input initial parameters such as the video URL of Youtube or the live broadcast URL of Facebook (you can even enter the names of China Television, Public Television, Sanli, TVBS or other news networks). Initial parameters), the capture model will output an initial screenshot of a live news broadcast. For example, the retrieval model 101 is programmed using python programming language, such as Selenium, cv2, numpy, collections, PIL or other related program suites. Furthermore, the capture model 101 is constructed using Selenium, which uses a headless browser (headless) and is set as a hidden window to save system resources. After the text recognition system 1 is started, the capture model 101 will pause for one minute. And wait for the advertisement to end, and then take a screenshot for a predetermined number of seconds.

The processing model 102 is used to extract the text region of interest (Extracting Text Region of Interest), so that when an initial screenshot of a live news broadcast is input, the processing model 102 will output a title in the form of black text on a white background. Or subtitle image file for text screenshot. For example, the processing model 102 locks the corresponding title in the live news screen. and/or the absolute position of the subtitle, take a screenshot of the text block at the absolute position, and after adjusting the size of the text block, convert the text block into white background and black text specifications for use as The text screenshot is stored in the temporary storage area 103 of the cloud database. It should be understood that most news title blocks often change the background color due to the news theme. Therefore, the processing model 102 will depend on the color richness used in the background of the text block to accurately convert the text block to white on a black background. Text screenshot of font size.

The recognition module 11 communicates with the collection module 10 to receive the text screenshot, and performs recognition operations to obtain target information.

In this embodiment, the recognition module 11 is an artificial intelligence (AI) module in the form of Optical Character Recognition (OCR), which includes at least one training model 111 and an analysis model 112 .

The construction of the training model 111 is divided into a preparation stage and a training stage, and the preparation stage prepares the feature files used in the training stage.

For example, in this preparation stage, all Chinese character files to be trained (for example, the file name is .tiff) are input to output the feature file. In this embodiment, the first open source tool (such as jTessBoxEditorFX) is used to extract each character block in the zigzag file of word software and merge it into a composite picture, and the second open source tool (such as Tesseract ) generates a temporary box file, and then uses a third open source tool (such as jTessBoxEditorFX) to mark the training feature range of the synthetic image, and then saves it as a feature file for artificial intelligence (AI) training. It should be understood that since manually adjusting the frame lines of the composite image takes a lot of time, the tiff file is generated here using the Python programming language to generate a single text tiff file of various fonts/sizes, and then set the position of the text. Save a lot of time.

Furthermore, the training method of this training stage is to input the feature file to output the training model. In this embodiment, the feature file for AI training generated in this preparation stage is used, and then the fourth open source tool for AI text recognition machine learning (such as the Tesseract suite) is used. The core algorithm is the long-term and short-term algorithm in deep learning. The Long Short-Term Memory (LSTM) neural algorithm outputs various training models according to different fonts and font sizes.

It should be understood that there are many types of open source tools that can be selected according to needs, and are not limited to the above, and the same open source tools can be used according to needs, such as the first and third open source tools, or the second and fourth open source tools, and No special restrictions.

The analysis model 112 inputs the training model 111 and any text screenshot with black font on a white background, and then outputs target information including string data (such as a text file, not an image file). For example, the fifth open source tool (such as the Tesseract suite) is used to sharpen the text screenshot, and then transmitted to the designated training model 111 for recognition, so that the fifth open source tool (such as the Tesseract suite) outputs the corresponding text screenshot. The string data includes all the text in the screenshot corresponding to the text.

It should be understood that there are many types of artificial intelligence (AI) modules and their construction methods, which are not limited to the above.

The compilation module 12 is communicatively connected to the identification module 11 to organize the target information into advanced information.

In this embodiment, the collection module 12 combines the string data obtained by the recognition module 11 with the corresponding image source, video snapshot screen (i.e., screenshot), and screenshot time (i.e., the collection module 10 captures the screen time) and/or other project references (from the temporary Area 103) is organized into the advanced information, such as a structured data format, to be stored in a database 12a (such as Google BigQuery, MySQL, ElasticSearch or other commonly used types) for subsequent related applications (for example, when an analyst When reviewing the content of the screen, since the content has been recognized as ordinary text, the news screen that mentions specific words at a certain moment can be immediately obtained through the search system, and the purpose of image knowledge preservation is achieved at the same time).

Figure 3 is a flow chart of the text recognition method of the present invention. In this embodiment, the character recognition method is executed by the character recognition system 1 .

In step S31, the user activates the text recognition system 1 through the electronic device 2 (smartphone, computer or others as shown in Figure 2) to perform collection operations through the collection module 10, so that at least one image The information is input to the collection module 10, causing the collection module 10 to generate text screenshots.

In this embodiment, the collection operation adopts a video crawler method, so that the collection module 10 automatically collects image information on the network platform (the data source of step S30 shown in Figure 3). The collection operation can perform a screenshot action on the image information to obtain an initial screenshot, and generate the text screenshot from the initial screenshot. For example, the image information is a live news video, which contains a plurality of continuous frames, so multiple initial screenshots P (one of which is shown in Figure 4A) are captured through the capture model 101, and through the processing model 102 Extract text blocks A1, A2, and A3 in at least one initial screenshot (as shown in Figure 4B) to obtain at least one text screenshot T1, T2, and T3 (as shown in Figure 4C), and then store them in the temporary storage area 103.

In step S32, the recognition module 11 performs a recognition operation so that the recognition module 11 receives at least one text screenshot in the temporary storage area 103 to obtain target information including string data.

In this embodiment, the recognition module 11 simultaneously inputs the text screenshot and the training model 111 into the analysis model 112, so that the analysis model 112 provides string data corresponding to the text screenshot, as shown in the following table :

Furthermore, the training model 111 of the recognition module 11 can simultaneously receive text screenshots in the temporary storage area 103 to perform machine learning operations, as shown in step S32a in Figure 3 . For example, the machine learning operation includes a preparation stage, step S32b shown in Figure 3, to input the required zigzag file (for example, the file name is .tiff).

In step S33, the compilation module 12 performs a compilation operation to store the target information.

In this embodiment, the collection module 12 stores the string data and the corresponding reference materials in a database 12a for subsequent application. For example, the subtitle content, subtitle occurrence time and original pictures of the live broadcast content are recognized and captured through AI method, and then stored in the database 12a, so that the demand side (such as brand public relations department, media public relations department, public relations company, analyst or other etc.) It is easier to quickly extract the TV public opinion content within a specific time range through the search engine (such as Google) in step S34, and the content can be obtained immediately without the help of additional typing personnel.

In another application, the demand side can develop a news push function to integrate commonly used push tools such as Line and E-mail. Therefore, when the live video content mentions specific words, it will be displayed in the E-mail inbox or Line The chat group can immediately receive the corresponding push message (including screen images and corresponding source descriptions, etc.). The demand side only needs to confirm the correctness of the push message and can make subsequent applications based on the importance of the push message. .

Or, the target information is transmitted to the training model 111 of the recognition module 11 to perform machine learning operations.

Therefore, multiple sets of target information can be stored in the database 12a for the demand side (such as brand media or public relations companies) to statistically analyze hot topics in each period, as shown in the following table:

To sum up, the text recognition system and method of the present invention mainly obtain the required words from Youtube live broadcast, Facebook live broadcast, digital TV signal source or other video data through the cooperation of the collection module and the recognition module. The string data is provided for subsequent demand-side applications. Therefore, compared with conventional plug-in software, the present invention can effectively obtain string data without being affected by background interference of the image information, thus reducing the chance of affecting the accuracy of recognition, and The collection module of the present invention can generate text screenshots, and then use the recognition module to directly identify string data from the text screenshots, thus greatly improving the recognition accuracy.

Therefore, if the demand side (such as brand public relations department, media public relations department, public relations company, etc.) needs to grasp public opinion in real time, the target information provided by the text recognition system 1 can quickly grasp multiple TV stations (or social networks) Platform) public opinion.

Furthermore, the subtitle content (string data) is obtained from the live content through the recognition module of AI technology, and the string data and the subtitle occurrence time (screenshot time) are stored in the database together with the original screen (or picture). It makes it easier for demand-side analysts to quickly extract public opinions and related screenshots within a specific time range through search engines (such as Google).

In addition, the recognition results of the text recognition system of the present invention can be used by users to pre-register to track specific words or specific public opinion news. When the content captured by the text recognition system contains specific words or specific public opinion news, the text recognition of the present invention The identification results of the system can be combined with the push system (such as Line, E-mail or other communication methods, etc.) to inform the user, so that the user (or company) can grasp the public opinion of multiple news media more efficiently, effectively shortening the response time. Down to minutes.

In addition, when the advanced information (or string data) is accumulated for a period of time, the demand side can integrate classification technologies such as natural language processing for statistical analysis. For example, analyze the number of seconds and occurrence of each topic in the live broadcast of various news media in the past. frequency. Furthermore, users can monitor multiple News channels from other sources such as YouTube or other sources can be viewed without the need to hire multiple additional monitors, which facilitates the provision of 24-hour uninterrupted monitoring services, making it easier for the demand side (such as the public relations team) to grasp in real time The public opinion needed.

The above embodiments are only illustrative to illustrate the principles, characteristics and effects of the present invention, and are not intended to limit the scope of the present invention. Anyone skilled in the art can make the above-mentioned modifications without violating the spirit and scope of the present invention. Modify and change the implementation form. Any equivalent changes and modifications made by applying the contents disclosed in the present invention shall still be covered by the patent application. Therefore, the protection scope of the present invention should be as listed in the patent application scope.

1: Text recognition system

1a:Host

10:Collect modules

101: Retrieve model

102: Processing models

103: Temporary storage area

11:Identification module

111:Training model

112:Analytical model

12: Collection module

12a:Database

Claims (10)

A text recognition system including: The collection module receives image information to generate text screenshots; and The recognition module communicates with the collection module to receive the text screenshot, and performs recognition operations on the text screenshot to obtain target information including string data. The text recognition system as described in claim 1, wherein the collection module adopts a web crawler mode to automatically search and collect the image information disclosed on the Internet. The text recognition system as described in claim 1, wherein the collection module performs a screenshot action on the image information to obtain an initial screenshot, and generates the text screenshot from the initial screenshot. The character recognition system as described in claim 1, wherein the recognition module is an artificial intelligence module in the form of optical character recognition. The character recognition system described in claim 1 further includes a compilation module that is connected to the recognition module to receive the target information, and organizes the target information into advanced information, wherein the advanced information includes the character String information and its corresponding reference materials. A text recognition method includes: Perform collection operations through a collection module, input image information into the collection module, and cause the collection module to generate text screenshots; and The text screenshot is received by the recognition module to perform recognition operations in order to obtain target information including string data. The text recognition method described in claim 6, wherein the collection operation adopts a web crawler method to automatically search and collect the image information published on the Internet. The text recognition method as described in claim 6, wherein the collection operation is to perform a screenshot action on the image information to obtain an initial screenshot, and generate the text screenshot from the initial screenshot. The character recognition method as described in claim 6, wherein the recognition module is an artificial intelligence module in the form of optical character recognition. The text recognition method described in claim 6 further includes receiving the target information through an assembly module to organize the target information into advanced information, wherein the advanced information includes the string data and its corresponding References.

Images