TWM631021U - Word recognition equipment - Google Patents

Abstract

A word recognition equipment is provided, which comprises inputting image information into a collection module, and making the collection module generate a text screenshot; and by the recognition module receiving the text screenshot, obtaining word string data without being affected by interference of background of the image information. Therefore, it can reduce the probability of affecting the accuracy of identification.

Description

text recognition device

This creation provides a text recognition technology, especially a text recognition device suitable for online video content.

With the rise of Internet platforms, ordinary people can easily obtain the audio and video content they need through Internet platforms. For example, in recent years, the live broadcast system has gradually matured. In addition to broadcasting signals through traditional channels, TV stations have also integrated online social platforms such as Youtube and Facebook to broadcast audio and video content signals through the live broadcast tools of the online social platform, and even some online social platforms. Road Media also tried to import live studios.

Due to the development of Internet information, related industries need to rely on the Internet information. For example, if an industry demand side such as a brand media or a public relations company needs to grasp social public opinion in real time, it is necessary to capture the video and audio content of these online social platforms for analysis.

However, these audio-visual contents include image data and text data, so that the file size of these audio-visual contents is too large, which makes it difficult to access.

Furthermore, although the industry demand side can only capture the text data in the audio and video content through the plug-in software, the conventional plug-in software is very sensitive to the background noise of the video data in the audio and video content, and is often affected by the background noise. Background interference affects the recognition accuracy. On the other hand, Xi When the plug-in software recognizes text, it needs to be performed under the conditions of extremely high contrast (such as black characters on a white background or white characters on a black background), otherwise the recognition accuracy rate is extremely low. For example, if the title background in the image data presents gradient colors or other patterns, the recognition accuracy will drop sharply.

Therefore, how to overcome the above-mentioned various problems of the conventional technology has become an urgent problem to be solved at present.

This creation provides an online text recognition device, which includes: a host; a collection module, which is mounted on the host and receives image information to generate text screenshots; and a recognition module, which is mounted on the host and communicates with the host The collection module receives the text screenshot, and performs an identification operation to obtain target information including string data.

In the aforementioned text recognition device, the collection module adopts a web crawler type to automatically search for and collect the image information published on the Internet.

In the aforementioned text recognition device, the collection module performs a screenshot operation on the image information to obtain an initial screenshot, and generates the text screenshot from the initial screenshot.

In the aforementioned text recognition device, the recognition module is an artificial intelligence module in the form of optical text recognition.

The aforementioned text recognition device further includes an aggregation module that is mounted on the host and is communicatively connected to the recognition module to receive the target information, which organizes the target information into advanced information, wherein the advanced information is Contains the string data and its corresponding reference data.

As can be seen from the above, the text recognition device of this creation mainly relies on the cooperation of the collection module and the recognition module to obtain the required character string data for subsequent application on the demand side. Therefore, compared with the conventional plug-in software, the This creation can effectively obtain string data regardless of the context of the image information The influence of interference, thus reducing the probability of affecting the recognition accuracy, and the collection module of this creation can generate the text screenshot, and then use the recognition module to directly identify the string data from the text screenshot, thus improving recognition. Correct rate.

1: Text recognition equipment

1a: host

10: Collect Mods

101: Capture the model

102: Processing Models

103: Temporary storage area

11: Identification module

111: Train the model

112: Analytical Models

12: Assemble module

12a:Database

2: Electronic device

A1,A2,A3: Text block

P: initial screenshot

T1, T2, T3: Text screenshots

S30~S33: Steps

FIG. 1 is a schematic diagram of the structure of the text recognition device of the present creation.

FIG. 2 is a schematic diagram of the configuration of the text recognition device of the present creation.

FIG. 3 is a flowchart of the recognition method of the text recognition device of the present creation.

4A to FIG. 4C are schematic views of an embodiment of the collection module of the character recognition device of the present creation during operation.

It should be noted that the structures, proportions, sizes, etc. shown in the drawings in this specification are only used to cooperate with the contents disclosed in the specification for the understanding and reading of those who are familiar with the art, and are not intended to limit the implementation of this creation. Therefore, it has no technical significance. Any modification of the structure, change of the proportional relationship or adjustment of the size, shall still fall within the scope of this work without affecting the effect and purpose of this creation. The technical content disclosed by the creation must be within the scope of coverage. At the same time, the terms such as "one", "first", "second", "upper" and "lower" quoted in this specification are only for the convenience of description and are not used to limit the possibilities of this creation. The scope of implementation and the change or adjustment of its relative relationship shall be regarded as the scope of implementation of this creation, provided there is no substantial change in the technical content.

FIG. 1 is a schematic diagram of the structure of the text recognition device 1 of the present creation. As shown in FIG. 1 , the character recognition device 1 includes: a host 1 a , a collection module 10 , a recognition module 11 and a collection module 12 .

In this embodiment, as shown in FIG. 2 , the collection module 10 , the identification module 11 and the aggregation module 12 are mounted in the host 1 a , and the host 1 a is a server, a cloud, and has various processes. computer device or mobile device, etc.

The collection module 10 is used for collecting image information, wherein the image information includes image data and reference data (such as image source, screenshot time, original image or other items, etc.).

In this embodiment, the image information is suitable for a network transmission format. For example, the image information is published on an online platform, and the online platform may be a public website such as an online community platform, media, or brand main website, such as Facebook fan page information, Instagram business account information, LINE official account information, Youtube channel information, Google map business information, MOD digital audio and video information, etc.

Furthermore, the collection module 10 can use a video crawler type to automatically search and extract image information published on the Internet. For example, the collection module 10 is suitable for live broadcast images.

Furthermore, the collection module 10 includes a capture model 101 for capturing images and a processing model 102 for preprocessing images.

The capture model 101 is used to capture images, so as to input initial parameters such as the video URL of Youtube or the live URL of Facebook (or even input names such as China Television, Public Television, Sanli, TVBS or other news network names). initial parameters), the extraction model will output as The initial screenshot of the live news screen. For example, the extraction model 101 is programmed using a python programming language, such as Selenium, cv2, numpy, collections, PIL, or other related program packages. Further, the capture model 101 is constructed by using Selenium, which uses a headless browser (headless) as a hidden window to save system resources, and after the text recognition device 1 is activated, the capture model 101 will pause for one minute And wait for the ad to end, then take another screenshot in a predetermined number of seconds.

The processing model 102 is used for extracting text regions of interest, so that when the input is an initial screenshot of a live news screen, the processing model 102 will output a title in the form of black characters on a white background. Or a subtitle image file for screenshots of text. For example, after locking the absolute position of the corresponding title and/or subtitle in the live news screen, the processing model 102 performs a screenshot operation of the text block at the absolute position, and after adjusting the size of the text block, The text block is converted into a specification of black characters on a white background, which is used as a screenshot of the text to be stored in the temporary storage area 103 such as a cloud database. It should be understood that most news headline blocks often change the background color due to the news theme, so the processing model 102 will accurately convert the text block to black on a white background according to the color richness of the background of the text block. A screenshot of the character specification.

The identification module 11 is communicatively connected to the collection module 10 to receive the text screenshot and perform identification operations to obtain target information.

In this embodiment, the recognition module 11 is an artificial intelligence (AI) module in the form of Optical Character Recognition (OCR), which includes at least a training model 111 and an analysis model 112 . .

The construction of the training model 111 is divided into a preparation stage and a training stage, and the preparation stage is to prepare a feature file for the training stage.

For example, in the preparation stage, input all Chinese character files to be trained (for example, the file name is .tiff) to output the feature file. In this embodiment, each character block in a zigzag file such as word software is extracted by a first open source tool (such as jTessBoxEditorFX), and combined into a composite image, and a second open source tool (such as Tesseract ) to generate a temporary box file, and then use a third open source tool (such as jTessBoxEditorFX) to mark the training feature range of the synthesized image, and then save it as a feature file for artificial intelligence (AI) training. It should be understood that since it takes a lot of time to manually adjust the frame line of the composite image, the generation of the tiff file here is to use the python programming language to generate a single text tiff file of various fonts/sizes, and then set the position of the text. Save a lot of time.

Furthermore, the training method in the training stage is to input the feature file to output the training model. In this embodiment, the feature file for AI training generated in the preparation stage is used, and then the fourth open source tool (such as the Tesseract suite) for AI text recognition machine learning is used, and its core algorithm is the long-term and short-term in deep learning. The memory (Long Short-Term Memory, referred to as LSTM) neural algorithm can output various training models according to different fonts and font sizes.

It should be understood that there are many types of open source tools, which can be selected according to needs, not limited to the above, and the same open source tools can be used according to needs, such as the first and third open source tools, or the second and fourth open source tools, and There are no particular restrictions.

The analysis model 112 inputs the training model 111 and a screenshot of any text in black on a white background, and then outputs target information including string data (eg, a text file, not an image file). For example, use a fifth open source tool (such as the Tesseract suite) to sharpen the text screenshot After processing, it is then transmitted to the designated training model 111 for identification, so that the fifth open source tool (such as the Tesseract suite) outputs the string data corresponding to the text screenshot, wherein the string data includes all the corresponding text screenshots. Word.

It should be understood that there are various types of the artificial intelligence (AI) module and its construction methods, which are not limited to the above.

The aggregating module 12 is communicatively connected to the identifying module 11 to organize the target information into advanced information.

In this embodiment, the compiling module 12 is the string data obtained by the identifying module 11 and its corresponding image source, video snapshot screen (ie, screenshot), and screenshot time (ie, the screen captured by the collection module 10 ). time) and/or other items and other reference data (from the temporary storage area 103) are organized into the advanced information, such as a structured data format, to be stored in a database 12a (such as Google BigQuery, MySQL, ElasticSearch or other Common types) for subsequent related applications (for example, when analysts review the content of the screen, because the content has been recognized as general text, the search system can immediately obtain the news screen that mentions a specific word at a certain moment, while reaching the image knowledge preservation purposes).

FIG. 3 is a flowchart of the recognition method of the text recognition device 1 of the present creation.

In step S31, the user activates the text recognition device 1 through the electronic device 2 (such as a smart phone, a computer or others as shown in FIG. 2), so as to perform a collection operation through the collection module 10 to make at least one image The information is input to the collection module 10, so that the collection module 10 generates a text screenshot.

In this embodiment, the collection operation adopts a video crawler method, so that the collection module 10 automatically collects the information of the network platform (step S30 shown in FIG. 3 ). image information on the data source), so that the collection operation can perform a screenshot action on the image information to obtain an initial screenshot, and the text screenshot is generated from the initial screenshot. For example, the image information is a live news video, which includes a plurality of consecutive frames, so the capture model 101 captures a plurality of initial screenshots P (one of which is shown in FIG. 4A ), and uses the processing model 102 Extract the text blocks A1, A2, A3 in at least one initial screenshot (as shown in FIG. 4B ) to obtain at least one text screenshot T1, T2, T3 (as shown in FIG. 4C ), and then store them in the temporary storage area 103.

In step S32 , the identification module 11 performs an identification operation, so that the identification module 11 receives at least one text screenshot in the temporary storage area 103 to acquire target information including string data.

In this embodiment, the text screenshot and the training model 111 are input into the analysis model 112 by the recognition module 11 at the same time, so that the analysis model 112 provides the string data corresponding to the text screenshot, as shown in the following table :

Furthermore, the training model 111 of the recognition module 11 can simultaneously receive the text screenshots in the temporary storage area 103 to perform machine learning operations, as shown in step S32a in FIG. 3 . For example, the machine learning operation system includes a preparatory stage, such as step S32b shown in FIG. 3 , for inputting a required zigzag file (eg, the file name is .tiff).

In step S33, the compilation operation is performed by the compilation module 12 to store the target information.

In this embodiment, the aggregating module 12 stores the string data and its corresponding reference data in a database 12a for subsequent application. For example, the subtitle content, the subtitle occurrence time and the original picture are identified and captured from the live broadcast content through AI, and then stored in the database 12a, so that the demand side (such as brand public relations department, media public relations department, public relations company, analysts or other etc.) it is easier to quickly extract the TV public opinion content within a specific time range through the search engine (eg, Google) in step S34, and the content can be obtained in real time without the help of additional typing staff.

In another application, the demand side can develop a push notification function to integrate the commonly used push tools such as Line and E-mail. Therefore, when a specific word is mentioned in the live video content, it will be displayed in the E-mail inbox or Line. The chat group can receive the corresponding push messages (such as screen images and corresponding source descriptions, etc.) in real time. The demand side only needs to confirm the correctness of the push messages, and then follow-up applications can be made according to the importance of the push messages. .

Alternatively, the target information is transmitted to the training model 111 of the identification module 11 to perform machine learning operations.

Therefore, the database 12a can store multiple sets of target information for the demand side (such as brand media or public relations companies) to statistically analyze the hot topics in each period, as shown in the following table:

To sum up, the text recognition device and method of this creation mainly rely on the cooperation of the collection module and the recognition module to obtain the required characters from Youtube live broadcast, Facebook live broadcast, digital TV signal source or other video data The string data is used for subsequent demand-side applications. Therefore, compared with the conventional plug-in software, this creation can effectively obtain the string data without being affected by the background interference of the image information, thereby reducing the probability of affecting the recognition accuracy. The collection module of this creation can generate text screenshots, and then use the recognition module to directly identify the string data from the text screenshots, thereby greatly improving the recognition accuracy.

Therefore, if the demand side (such as brand public relations department, media public relations department, public relations company, etc.) needs to grasp public opinion in real time, the target information provided by the text recognition device 1 can quickly grasp multiple TV sets (or social networks) platform) public opinion.

Furthermore, the subtitle content (string data) of the live broadcast content is obtained through the recognition module of AI technology, and the string data and the subtitle generation time (screenshot time) are stored in the database together with the original picture (or picture), It makes it easier for demand-side analysts to quickly extract public opinion and related screenshots within a specific time frame through search engines (such as Google).

In addition, the recognition results of the text recognition device of this creation can be used for users to pre-register to track specific words or specific public opinion alerts. When the content captured by the text recognition device contains specific words or specific public opinion alerts, the text recognition of this creation The identification results of the device can be combined with the push broadcast system (such as Line, E-mail or other communication methods) to inform the user, so that the user (or enterprise) can more efficiently grasp the public opinion of multiple news media and effectively shorten the response time. Reduced to minutes.

In addition, after the advanced information (or string data) has been accumulated for a period of time, the demand side can integrate classification techniques such as natural language processing for statistical analysis, for example, to analyze the seconds and occurrences of the past live broadcasts of various topics in various news media frequency. Further, users can simultaneously monitor multiple news channels such as Youtube videos or other sources without hiring additional monitoring personnel to watch, so as to facilitate the provision of 24-hour uninterrupted monitoring services, enabling demand-side (such as public relations) team) can more easily and instantly grasp the required public opinion.

The above embodiments are only illustrative of the principles, features and effects of the present creation, and are not intended to limit the scope of implementation of the present creation. Modifications and changes are made to the implementation form. any application Equivalent changes and modifications made by creating the disclosed content shall still be covered by the scope of the patent application. Therefore, the scope of protection of the rights of this creation should be listed in the scope of the patent application.

1: Text recognition equipment

1a: host

10: Collect Mods

101: Capture the model

102: Processing Models

103: Temporary storage area

11: Identification module

111: Train the model

112: Analytical Models

12: Assemble module

12a:Database

Claims (5)

A character recognition device, comprising: host; A collection module that is deployed in the host and receives image information to generate text screenshots; and The identification module is installed in the host and communicated with the collection module to receive the text screenshot, and perform identification operation on the text screenshot, so as to obtain target information including string data. The character recognition device according to claim 1, wherein the collection module adopts a web crawler type to automatically search for and collect the image information published on the Internet. The character recognition device according to claim 1, wherein the collection module performs a screenshot operation on the image information to obtain an initial screenshot, and generates the text screenshot from the initial screenshot. The character recognition device according to claim 1, wherein the recognition module is an artificial intelligence module in the form of optical character recognition. The character recognition device as claimed in claim 1 further comprises an aggregation module that is mounted on the host and is communicatively connected to the recognition module to receive the target information, and organizes the target information into advanced information, wherein the Advanced information includes the string data and its corresponding reference data.

Images