WO2023116137A1 - 一种文本识别方法及其相关设备 - Google Patents

一种文本识别方法及其相关设备 Download PDF

Info

Publication number
WO2023116137A1
WO2023116137A1 PCT/CN2022/124761 CN2022124761W WO2023116137A1 WO 2023116137 A1 WO2023116137 A1 WO 2023116137A1 CN 2022124761 W CN2022124761 W CN 2022124761W WO 2023116137 A1 WO2023116137 A1 WO 2023116137A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
feature extraction
noise suppression
text recognition
recognized
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/124761
Other languages
English (en)
French (fr)
Inventor
张家鑫
黄灿
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Youzhuju Network Technology Co Ltd
Original Assignee
Beijing Youzhuju Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Youzhuju Network Technology Co Ltd filed Critical Beijing Youzhuju Network Technology Co Ltd
Publication of WO2023116137A1 publication Critical patent/WO2023116137A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Definitions

  • the present application relates to the technical field of image processing, in particular to a text recognition method and related equipment.
  • Text recognition technology is a typical image processing technology; and the text recognition technology has a wide range of applications. Among them, the text recognition technology is used to perform recognition processing on character information in an image data.
  • the present application provides a text recognition method and related equipment, which can improve the effect of text recognition.
  • An embodiment of the present application provides a text recognition method, the method comprising:
  • the image to be recognized is input into a pre-built text recognition model to obtain the text recognition result of the image to be recognized output by the text recognition model;
  • the text recognition model includes a feature extraction network, a recurrent neural network RNN and a transcription Layer CTC: the feature extraction network is used to extract image features of the image to be recognized, and the feature extraction network is also used to suppress noise information carried by the image to be recognized.
  • the feature extraction network includes an image feature extraction subnetwork and an image noise suppression subnetwork; the image feature extraction subnetwork is used to extract image features of the image to be recognized; the image noise The suppression sub-network is used to suppress noise information carried by the image to be recognized.
  • the image noise suppression sub-network includes a spatial noise suppression module and/or a channel noise suppression module; wherein the spatial noise suppression module is used to suppress background noise carried by the image to be recognized;
  • the channel noise suppression module is used to implement channel attention mechanism.
  • the image noise suppression sub-network includes a spatial noise suppression module
  • the determination process of the text recognition result includes:
  • a text recognition result of the image to be recognized is determined according to the feature of the image to be processed, the RNN and the CTC.
  • the spatial noise suppression module includes a spatial weight determination submodule and a feature weight fusion submodule;
  • the image feature extraction subnetwork includes N feature extraction modules;
  • the image noise suppression subnetwork includes N channel noise suppression modules;
  • the input data of the nth channel noise suppression module includes the output data of the nth feature extraction module; wherein, n is a positive integer, n ⁇ N, and N is a positive integer;
  • the input data of the first feature extraction module includes the image to be identified, and the input data of the i+1 feature extraction module includes the output data of the i channel noise suppression module; wherein, i is a positive integer, i+1 ⁇ N, N is a positive integer.
  • the image feature extraction subnetwork includes N feature extraction modules;
  • the image noise suppression subnetwork includes N channel noise suppression modules and spatial noise suppression modules;
  • the input data of the nth channel noise suppression module includes the output data of the nth feature extraction module; wherein, n is a positive integer, n ⁇ N, and N is a positive integer;
  • the input data of the first feature extraction module includes the image to be identified, and the input data of the i+1 feature extraction module includes the output data of the i channel noise suppression module; wherein, i is a positive integer, i+1 ⁇ N, N is a positive integer;
  • the input data of the spatial noise suppression module includes the output data of the Nth channel noise suppression module.
  • the embodiment of the present application also provides a text recognition device, including:
  • an acquisition unit configured to acquire an image to be identified
  • a recognition unit configured to input the image to be recognized into a pre-built text recognition model, and obtain the text recognition result of the image to be recognized output by the text recognition model;
  • the text recognition model includes a feature extraction network, a loop A neural network RNN and a transcription layer CTC; the feature extraction network is used to extract image features of the image to be recognized, and the feature extraction network is also used to suppress noise information carried by the image to be recognized.
  • the embodiment of the present application also provides a device, the device includes a processor and a memory:
  • the memory is used to store computer programs
  • the processor is configured to execute any implementation manner of the text recognition method provided in the embodiments of the present application according to the computer program.
  • the embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium is used to store a computer program, and the computer program is used to execute any implementation manner of the text recognition method provided in the embodiment of the present application.
  • the embodiment of the present application also provides a computer program product, which, when running on a terminal device, enables the terminal device to execute any implementation manner of the text recognition method provided in the embodiment of the present application.
  • the embodiment of the present application has at least the following advantages:
  • the image to be recognized carrying noise information
  • the image to be recognized is input into a pre-built text recognition model, and the text recognition of the image to be recognized output by the text recognition model is obtained result.
  • the text recognition model includes feature extraction network, recurrent neural network and transcription layer.
  • the feature extraction network is used to extract the image features of the image to be recognized and suppress the noise information carried by the image to be recognized.
  • the feature extraction network has the function of image noise suppression, the image features extracted from the image to be recognized by the feature extraction network carry less noise information (even, no noise information), so that the image feature to be recognized can be effectively reduced. Recognizing the adverse effect caused by the noise information carried by the image on the text recognition process can effectively improve the accuracy of the text recognition result of the image to be recognized, and further help to improve the text recognition effect.
  • FIG. 1 is a schematic diagram of image data carrying noise information provided by an embodiment of the present application
  • FIG. 2 is a flowchart of a text recognition method provided in an embodiment of the present application
  • FIG. 3 is a schematic diagram of a background noise suppression result provided in an embodiment of the present application.
  • FIG. 4 is a schematic structural diagram of a text recognition model provided in an embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of another text recognition model provided by the embodiment of the present application.
  • FIG. 6 is a schematic structural diagram of another text recognition model provided by the embodiment of the present application.
  • Fig. 7 is a schematic diagram of the working principle of another text recognition model provided by the embodiment of the present application.
  • FIG. 8 is a schematic structural diagram of a text recognition device provided by an embodiment of the present application.
  • CRNN Convolutional Recurrent Neural Network
  • CNN convolutional Neural Networks
  • RNN recurrent neural network
  • CTC Connectionist Temporal Classification
  • an embodiment of the present application provides a text recognition method, the method includes: after acquiring the image to be recognized that carries noise information, inputting the image to be recognized A pre-built text recognition model is used to obtain the text recognition result of the image to be recognized output by the text recognition model.
  • the text recognition model includes feature extraction network, recurrent neural network and transcription layer. The feature extraction network is used to extract the image features of the image to be recognized and suppress the noise information carried by the image to be recognized.
  • the feature extraction network has the function of image noise suppression, the image features extracted from the image to be recognized by the feature extraction network carry less noise information (even, no noise information), so that the image feature to be recognized can be effectively reduced. Recognizing the adverse effect caused by the noise information carried by the image on the text recognition process can effectively improve the accuracy of the text recognition result of the image to be recognized, and further help to improve the text recognition effect.
  • the embodiment of the present application does not limit the subject of execution of the text recognition method.
  • the text recognition method provided in the embodiment of the present application can be applied to data processing devices such as terminal devices or servers.
  • the terminal device may be a smart phone, a computer, a personal digital assistant (Personal Digital Assistant, PDA), or a tablet computer.
  • the server can be an independent server, a cluster server or a cloud server.
  • this figure is a flow chart of a text recognition method provided by an embodiment of the present application.
  • the text recognition method provided in the embodiment of this application includes S1-S2:
  • image to be recognized refers to image data that needs text recognition processing (image data as shown in FIG. 1); and the “image to be recognized” carries noise information.
  • noise information refers to image information that can adversely affect the text recognition process.
  • the embodiment of the present application does not limit the above-mentioned "image to be recognized", for example, it may be any image data.
  • the "image to be recognized” can be an image data with a height of 32 and an unlimited width (that is, an image of 32 ⁇ W data).
  • W represents the width of one image data.
  • the embodiment of the present application does not limit the acquisition process of the above-mentioned "image to be recognized".
  • it may specifically include: after the image acquisition data is acquired, the image can be The collected data is adjusted for image size to obtain the image to be recognized, so that the height of the image to be recognized reaches the fixed height of the preset model, and the aspect ratio of the image to be recognized is consistent with the aspect ratio of the image collection data.
  • image acquisition data refers to image data acquired by an image acquisition device (eg, camera, video camera, etc.).
  • the image size can be adjusted for the image acquisition data to obtain the image to be recognized, so that the height of the image to be recognized can reach the above " Preset model fixed height", and make the aspect ratio of the image to be recognized consistent with the aspect ratio of the image acquisition data, so that the text recognition model can be used to perform text recognition processing on the image to be recognized in order to achieve
  • image acquisition data is used for the purpose of text recognition processing.
  • S2 Input the image to be recognized into the pre-built text recognition model, and obtain the text recognition result of the image to be recognized output by the text recognition model.
  • the above "text recognition result of the image to be recognized” is used to represent the character information carried by the image to be recognized.
  • the "text recognition result of the image to be recognized” may be "STARBUCKS”.
  • the above “text recognition model” is used to perform text recognition processing on the input data of the text recognition model; and the “text recognition model” is a machine learning model.
  • the above “text recognition model” is an improved CRNN; and the “text recognition model” includes feature extraction network, RNN and CTC.
  • the input data of CTC includes the output data of RNN; the output data of RNN includes the output data of feature extraction network.
  • feature extraction network is not only used for feature extraction for the input data of the feature extraction network; and the “feature extraction network” is also used for noise suppression processing for the input data of the feature extraction network. It can be seen that after the above “image to be recognized” is input into the feature extraction network, the feature extraction network can extract the image features of the "image to be recognized", and can also suppress the noise information carried by the "image to be recognized”.
  • the embodiment of the present application does not limit the implementation manner of the above-mentioned "feature extraction network", for example, it may specifically include: an image feature extraction subnetwork and an image noise suppression subnetwork.
  • image feature extraction sub-network is used to perform image feature extraction processing on the input data of the image feature extraction sub-network, so that the "image feature extraction sub-network” has an image feature extraction function. It can be seen that after the above "image to be recognized” is input into the image feature extraction sub-network, the image feature extraction sub-network can extract the image features of the "image to be recognized”.
  • the embodiment of the present application does not limit the implementation of the above "image feature extraction sub-network".
  • any existing or future image feature extraction network eg, CNN
  • the embodiment of the present application also provides a possible implementation of the above-mentioned "image feature extraction sub-network", for example, it may specifically include: N feature extraction modules.
  • feature extraction module has an image feature extraction function; and the embodiment of the present application does not limit the “feature extraction module”, for example, because the residual block has a relatively good effect on feature extraction and preventing gradient disappearance, so in order to improve the image Feature extraction effect, the above “feature extraction module” can be a residual block (Residual Block). It should be noted that a residual block can include two CNNs.
  • the "image feature extraction sub-network” may include 8 residual blocks, so that the “image feature extraction sub-network” has Better image feature extraction effect, so that the subsequent "image feature extraction sub-network” can be used to extract image features of an image data (for example, the above "image to be recognized”).
  • image noise suppression sub-network is used to perform noise suppression processing on the input data of the image noise suppression sub-network, so that the "image noise suppression sub-network” has an image noise suppression function, so that the "image noise suppression sub-network” " can be used to suppress the noise information carried by an image data (for example, the "image to be recognized” above).
  • the embodiment of the present application does not limit the implementation manner of the above-mentioned "image noise suppression sub-network”.
  • image noise suppression sub-network For ease of understanding, the following description will be made in conjunction with three situations.
  • a global attention mechanism (Global-Attention) in space can be used for implementation.
  • the embodiment of the present application provides a possible implementation manner of the above “image noise suppression sub-network”, which may specifically include: a spatial noise suppression module.
  • the “spatial noise suppression module” is used to suppress the background noise carried by an image data (for example, the above "image to be recognized”).
  • the feature extraction network includes an image feature extraction subnetwork and an image noise suppression subnetwork
  • the image noise suppression subnetwork includes a spatial noise suppression module
  • Step 11 Input the image to be recognized into the image feature extraction sub-network, and obtain the image features to be used output by the image feature extraction sub-network.
  • image feature to be used refers to the image feature of the "image to be recognized” above, so that the “image feature to be used” is used to represent the image information (eg, character information and noise information) carried by the image to be recognized.
  • image information eg, character information and noise information
  • Step 12 Input the image features to be used into the spatial noise suppression module, and obtain the image features to be processed output by the spatial noise suppression module.
  • spatial noise suppression module is used to perform background noise suppression processing on the input data of the spatial noise suppression module.
  • the embodiment of the present application does not limit the implementation manner of the above-mentioned "spatial noise suppression module”, for example, it may be implemented based on a spatial global attention mechanism.
  • the embodiment of the present application provides a possible implementation of the above-mentioned "spatial noise suppression module", which may specifically include: a spatial weight determination submodule and a feature weighted fusion submodule; and the input of the feature weighted fusion submodule
  • the data includes the output data of the spatial weight determination sub-module and the output data of the "image feature extraction sub-network" above.
  • the determination process of "image features to be processed” may specifically include steps 21-22:
  • Step 21 Input the image features to be used into the spatial weight determination sub-module, and obtain the spatial weights to be used output by the spatial weight determination sub-module.
  • spatial weight determination sub-module is used to perform spatial weight determination processing (that is, background noise affects weight determination processing) for the input data of the spatial weight determination sub-module.
  • the embodiment of the present application does not limit the implementation of the above "spatial weight determination sub-module", for example, it may be implemented by using a layer of convolutional neural network.
  • the aforementioned "spatial weight to be used” is used to represent the weighted weight of each feature element in the aforementioned "image feature to be used".
  • the "space weight to be used” can be an R ⁇ E matrix (as shown in formula (2) shown), and the sum of elements in each column in the "to-be-used space weight” is 1 (as shown in formula (3)).
  • F use represents the above-mentioned "image feature to be used”
  • D space represents the above-mentioned “space weight to be used”
  • R represents the feature map height of the above-mentioned “image feature to be used”, and R is a positive integer
  • E represents the above-mentioned "to-be-used space weight”.
  • Step 22 Input the image features to be used and the spatial weights to be used into the feature weighted fusion sub-module, and obtain the image features to be processed output by the feature weighted fusion sub-module.
  • feature weighted fusion sub-module is used to perform weighted sum processing on the image features to be used according to the spatial weights to be used (as shown in formulas (4)-(5)).
  • F discern represents the image feature to be processed.
  • the above formulas (1)-(5) can be used to suppress the background noise of the image features to be used, so as to enlarge the
  • the gap between the feature elements representing character information and the feature elements used to represent background noise is used to obtain the image features to be processed, so that the image features to be processed can achieve the effect of highlighting character information and dim background noise (as shown in Figure 3 shown), so that the image features to be processed can clearly distinguish the character information and background noise carried by the above-mentioned "image to be recognized", so that it can effectively avoid directly compressing the feature map of the above-mentioned image features to be used to a height of 1
  • the character information is integrated with the background noise, it can effectively improve the effect of image feature extraction.
  • the spatial noise suppression module can be used to perform background noise suppression processing on the image features to be used, and the image features to be processed are obtained and output, so that the image features to be processed Among the image features, there is a large difference between the feature elements used to represent character information and the feature elements used to represent background noise, so that the image features to be processed can clearly distinguish the character information carried by the above-mentioned "image to be recognized" And background noise, which is beneficial to improve the effect of image feature extraction.
  • Step 13 According to the features of the image to be processed, RNN and CTC, determine the text recognition result of the image to be recognized.
  • step 13 can be implemented by using the implementation process of "determining the text recognition result according to the output data of CNN, RNN and CTC" used in CRNN .
  • the input data of the spatial noise suppression module may include the output data of the image feature extraction sub-network
  • the input data of the RNN may include the output data of the spatial noise suppression module
  • channel noise suppression can be performed using channel attention mechanism (Squeeze-and-Excitation Networks, SE-Net).
  • SE-Net channel attention mechanism
  • the embodiment of the present application provides another possible implementation manner of the aforementioned "image noise suppression subnetwork", which may specifically include: a channel noise suppression module.
  • the “channel noise suppression module” is used to implement the channel attention mechanism.
  • the image feature extraction subnetwork may include N features An extraction module
  • the image noise suppression sub-network may include N channel noise suppression modules.
  • the input data of the nth channel noise suppression module includes the output data of the nth feature extraction module; wherein, n is a positive integer, n ⁇ N, and N is a positive integer.
  • the input data of the first feature extraction module includes the image to be recognized
  • the input data of the i+1th feature extraction module includes the output data of the i-th channel noise suppression module; where i is a positive integer, i+1 ⁇ N, N is a positive integer.
  • the first feature extraction module performs image feature extraction processing on an image data (for example, the above "image to be recognized") to obtain the output data of the first feature extraction module; then the first channel noise suppression module Perform channel noise suppression processing on the output data of the first feature extraction module to obtain the output data of the first channel noise suppression module; then use the second feature extraction module for the output data of the first channel noise suppression module Perform image feature extraction processing to obtain the output data of the second feature extraction module; then perform channel noise suppression processing on the output data of the second feature extraction module by the second channel noise suppression module to obtain the second channel The output data of the noise suppression module; ...
  • the Nth feature extraction module carries out image feature extraction processing for the output data of the N-1 channel noise suppression module, and obtains the Nth feature extraction The output data of the module; then the Nth channel noise suppression module performs channel noise suppression processing on the output data of the Nth feature extraction module to obtain the output data of the Nth channel noise suppression module; finally, according to the Nth The output data of the channel noise suppression module, RNN and CTC, determine the text recognition result of the image data.
  • each channel noise suppression module mentioned above can adopt a channel attenuation strategy to reduce network costs, so that the channel noise suppression module has almost no additional time overhead after engineering, which is conducive to improving text recognition efficiency.
  • Case 3 in order to overcome the noise shown in the above two cases, the present application provides another possible implementation of the above "image noise suppression sub-network", which may specifically include: a spatial noise suppression module and a channel noise suppression module.
  • image noise suppression sub-network may specifically include: a spatial noise suppression module and a channel noise suppression module.
  • the image feature extraction subnetwork may include N features An extraction module
  • the image noise suppression sub-network may include N channel noise suppression modules and a spatial noise suppression module.
  • the input data of the nth channel noise suppression module includes the output data of the nth feature extraction module; wherein, n is a positive integer, n ⁇ N, and N is a positive integer.
  • the input data of the first feature extraction module includes the image to be recognized
  • the input data of the i+1th feature extraction module includes the output data of the i-th channel noise suppression module; where i is a positive integer, i+1 ⁇ N, N is a positive integer.
  • the input data of the spatial noise suppression module includes the output data of the Nth channel noise suppression module.
  • the first feature extraction module performs image feature extraction processing on an image data (for example, the above "image to be recognized") to obtain the output data of the first feature extraction module; then the first channel noise suppression module Perform channel noise suppression processing on the output data of the first feature extraction module to obtain the output data of the first channel noise suppression module; then use the second feature extraction module for the output data of the first channel noise suppression module Perform image feature extraction processing to obtain the output data of the second feature extraction module; then perform channel noise suppression processing on the output data of the second feature extraction module by the second channel noise suppression module to obtain the second channel The output data of the noise suppression module; ...
  • the Nth feature extraction module carries out image feature extraction processing for the output data of the N-1 channel noise suppression module, and obtains the Nth feature extraction the output data of the module; then the Nth channel noise suppression module performs channel noise suppression processing on the output data of the Nth feature extraction module to obtain the output data of the Nth channel noise suppression module; then, the spatial noise suppression The module performs background noise suppression processing on the output data of the N channel noise suppression module to obtain the output data of the spatial noise suppression module; finally, according to the output data, RNN and CTC of the spatial noise suppression module, determine the image data Text recognition results.
  • BLSTM Bi-directional Long Short Term Memory Network
  • the text recognition model includes feature extraction network, recurrent neural network and transcription layer.
  • the feature extraction network is used to extract the image features of the image to be recognized and suppress the noise information carried by the image to be recognized.
  • the feature extraction network has the function of image noise suppression, the image features extracted from the image to be recognized by the feature extraction network carry less noise information (even, no noise information), so that the image feature to be recognized can be effectively reduced. Recognizing the adverse effect caused by the noise information carried by the image on the text recognition process can effectively improve the accuracy of the text recognition result of the image to be recognized, and further help to improve the text recognition effect.
  • the embodiment of the present application also provides a text recognition device, which will be explained and described below with reference to the accompanying drawings.
  • FIG. 8 this figure is a schematic structural diagram of a text recognition device provided by an embodiment of the present application.
  • the text recognition device 800 provided in the embodiment of the present application includes:
  • the recognition unit 802 is configured to input the image to be recognized into a pre-built text recognition model, and obtain the text recognition result of the image to be recognized output by the text recognition model; wherein, the text recognition model includes a feature extraction network, A cyclic neural network RNN and a transcription layer CTC; the feature extraction network is used to extract image features of the image to be recognized, and the feature extraction network is also used to suppress noise information carried by the image to be recognized.
  • the text recognition model includes a feature extraction network, A cyclic neural network RNN and a transcription layer CTC; the feature extraction network is used to extract image features of the image to be recognized, and the feature extraction network is also used to suppress noise information carried by the image to be recognized.
  • the feature extraction network includes an image feature extraction subnetwork and an image noise suppression subnetwork; the image feature extraction subnetwork is used to extract image features of the image to be recognized; the image noise The suppression sub-network is used to suppress noise information carried by the image to be recognized.
  • the image noise suppression sub-network includes a spatial noise suppression module and/or a channel noise suppression module; wherein the spatial noise suppression module is used to suppress background noise carried by the image to be recognized;
  • the channel noise suppression module is used to implement channel attention mechanism.
  • the image noise suppression sub-network includes a spatial noise suppression module
  • the identification unit 802 is specifically configured to: input the image to be identified into the image feature extraction sub-network to obtain the image features to be used output by the image feature extraction sub-network; input the image features to be used to the The spatial noise suppression module obtains the image features to be processed output by the spatial noise suppression module; and determines the text recognition result of the image to be recognized according to the image features to be processed, the RNN and the CTC.
  • the spatial noise suppression module includes a spatial weight determination submodule and a feature weight fusion submodule;
  • the process of determining the image features to be processed includes: inputting the image features to be used into the spatial weight determination sub-module to obtain the spatial weights to be used output by the spatial weight determination sub-module; inputting the image features to be used and the to-be-used spatial weights are input into the feature weighted fusion sub-module to obtain the image features to be processed output by the feature weighted fusion sub-module.
  • the image feature extraction subnetwork includes N feature extraction modules;
  • the image noise suppression subnetwork includes N channel noise suppression modules;
  • the input data of the nth channel noise suppression module includes the output data of the nth feature extraction module; wherein, n is a positive integer, n ⁇ N, and N is a positive integer;
  • the input data of the first feature extraction module includes the image to be identified, and the input data of the i+1 feature extraction module includes the output data of the i channel noise suppression module; wherein, i is a positive integer, i+1 ⁇ N, N is a positive integer.
  • the image feature extraction subnetwork includes N feature extraction modules;
  • the image noise suppression subnetwork includes N channel noise suppression modules and spatial noise suppression modules;
  • the input data of the nth channel noise suppression module includes the output data of the nth feature extraction module; wherein, n is a positive integer, n ⁇ N, and N is a positive integer;
  • the input data of the first feature extraction module includes the image to be identified, and the input data of the i+1 feature extraction module includes the output data of the i channel noise suppression module; wherein, i is a positive integer, i+1 ⁇ N, N is a positive integer;
  • the input data of the spatial noise suppression module includes the output data of the Nth channel noise suppression module.
  • the text recognition device 800 Based on the relevant content of the above-mentioned text recognition device 800, it can be seen that for the text recognition device 800 provided in the embodiment of the present application, after acquiring the image to be recognized that carries noise information, input the image to be recognized into the pre-built text recognition model , to obtain the text recognition result of the image to be recognized output by the text recognition model.
  • the text recognition model includes feature extraction network, recurrent neural network and transcription layer.
  • the feature extraction network is used to extract the image features of the image to be recognized and suppress the noise information carried by the image to be recognized.
  • the feature extraction network has the function of image noise suppression, the image features extracted from the image to be recognized by the feature extraction network carry less noise information (even, no noise information), so that the image feature to be recognized can be effectively reduced. Recognizing the adverse effect caused by the noise information carried by the image on the text recognition process can effectively improve the accuracy of the text recognition result of the image to be recognized, and further help to improve the text recognition effect.
  • the embodiment of the present application also provides a device, the device includes a processor and a memory:
  • the memory is used to store computer programs
  • the processor is configured to execute any implementation manner of the text recognition method provided in the embodiments of the present application according to the computer program.
  • the embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium is used to store a computer program, and the computer program is used to execute any of the text recognition methods provided in the embodiment of the present application. implementation.
  • the embodiment of the present application also provides a computer program product, which, when running on the terminal device, enables the terminal device to execute any implementation manner of the text recognition method provided in the embodiment of the present application.
  • At least one (item) means one or more, and “multiple” means two or more.
  • “And/or” is used to describe the association relationship of associated objects, indicating that there can be three types of relationships, for example, “A and/or B” can mean: only A exists, only B exists, and A and B exist at the same time , where A and B can be singular or plural.
  • the character “/” generally indicates that the contextual objects are an “or” relationship.
  • At least one of the following” or similar expressions refer to any combination of these items, including any combination of single or plural items.
  • At least one item (piece) of a, b or c can mean: a, b, c, "a and b", “a and c", “b and c", or "a and b and c ", where a, b, c can be single or multiple.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)

Abstract

一种文本识别方法及其相关设备，涉及图像处理技术领域。该方法包括：在获取到携带有噪声信息的待识别图像之后，将该待识别图像输入预先构建的文本识别模型，得到该文本识别模型输出的该待识别图像的文本识别结果。其中，文本识别模型包括特征提取网络、循环神经网络和转录层。特征提取网络用于提取待识别图像的图像特征、以及抑制该待识别图像携带的噪声信息。可见，特征提取网络具有图像噪声抑制功能，使得利用特征提取网络针对该待识别图像提取所得的图像特征几乎不携带噪声信息，如此能够有效地降低该待识别图像携带的噪声信息对文本识别过程所造成的不良影响，从而能够有效地提高该待识别图像的文本识别结果的准确性，进而有利于提高文本识别效果。

Description

一种文本识别方法及其相关设备
本申请要求于2021年12月21日提交中国专利局、申请号为202111575226.0、申请名称为“一种文本识别方法及其相关设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理技术领域，尤其涉及一种文本识别方法及其相关设备。
背景技术
文本识别技术是一种典型的图像处理技术；而且该文本识别技术的应用范围十分广泛。其中，文本识别技术用于针对一个图像数据中字符信息进行识别处理。
然而，因一些文本识别技术存在缺陷，导致这些文本识别技术针对一些图像数据(例如，携带有噪声信息的图像数据等)的文本识别效果较差。
发明内容
为了解决上述技术问题，本申请提供了一种文本识别方法及其相关设备，能够提高文本识别效果。
为了实现上述目的，本申请实施例提供的技术方案如下：
本申请实施例提供一种文本识别方法，所述方法包括：
获取待识别图像；
将所述待识别图像输入预先构建的文本识别模型，得到所述文本识别模型输出的所述待识别图像的文本识别结果；其中，所述文本识别模型包括特征提取网络、循环神经网络RNN和转录层CTC；所述特征提取网络用于提取所述待识别图像的图像特征，且所述特征提取网络还用于抑制所述待识别图像携带的噪声信息。
在一种可能的实施方式中，所述特征提取网络包括图像特征提取子网络和图像噪声抑制子网络；所述图像特征提取子网络用于提取所述待识别图像的图像特征；所述图像噪声抑制子网络用于抑制所述待识别图像携带的噪声信息。
在一种可能的实施方式中，所述图像噪声抑制子网络包括空间噪声抑制模块和/或通道噪声抑制模块；其中，所述空间噪声抑制模块用于抑制所述待识别图像携带的背景噪声；所述通道噪声抑制模块用于实现通道注意力机制。
在一种可能的实施方式中，所述图像噪声抑制子网络包括空间噪声抑制模块；
所述文本识别结果的确定过程，包括：
将所述待识别图像输入所述图像特征提取子网络，得到所述图像特征提取子网络输出的待使用图像特征；
将所述待使用图像特征输入所述空间噪声抑制模块，得到所述空间噪声抑制模块输出的待处理图像特征；
根据所述待处理图像特征、所述RNN和所述CTC，确定所述待识别图像的文本识别结果。
在一种可能的实施方式中，所述空间噪声抑制模块包括空间权重确定子模块和特征加权融合子模块；
所述待处理图像特征的确定过程，包括：
将所述待使用图像特征输入所述空间权重确定子模块，得到所述空间权重确定子模块输出的待使用空间权重；
将所述待使用图像特征和所述待使用空间权重输入所述特征加权融合子模块，得到所述特征加权融合子模块输出的待处理图像特征。
在一种可能的实施方式中，所述图像特征提取子网络包括N个特征提取模块；所述图像噪声抑制子网络包括N个通道噪声抑制模块；
第n个通道噪声抑制模块的输入数据包括第n个特征提取模块的输出数据；其中，n为正整数，n≤N，N为正整数；
第1个特征提取模块的输入数据包括所述待识别图像，且第i+1个特征提取模块的输入数据包括第i个通道噪声抑制模块的输出数据；其中，i为正整数，i+1≤N，N为正整数。
在一种可能的实施方式中，所述图像特征提取子网络包括N个特征提取模块；所述图像噪声抑制子网络包括N个通道噪声抑制模块和空间噪声抑制模块；
第n个通道噪声抑制模块的输入数据包括第n个特征提取模块的输出数据；其中，n为正整数，n≤N，N为正整数；
第1个特征提取模块的输入数据包括所述待识别图像，且第i+1个特征提取模块的输入数据包括第i个通道噪声抑制模块的输出数据；其中，i为正整数，i+1≤N，N为正整数；
所述空间噪声抑制模块的输入数据包括第N个通道噪声抑制模块的输出数据。
本申请实施例还提供了一种文本识别装置，包括：
获取单元，用于获取待识别图像；
识别单元，用于将所述待识别图像输入预先构建的文本识别模型，得到所述文本识别模型输出的所述待识别图像的文本识别结果；其中，所述文本识别模型包括特征提取网络、循环神经网络RNN和转录层CTC；所述特征提取网络用于提取所述待识别图像的图像特征，且所述特征提取网络还用于抑制所述待识别图像携带的噪声信息。
本申请实施例还提供了一种设备，所述设备包括处理器以及存储器：
所述存储器用于存储计算机程序；
所述处理器用于根据所述计算机程序执行本申请实施例提供的文本识别方法的任一实施方式。
本申请实施例还提供了一种计算机可读存储介质，所述计算机可读存储介质用于存储计算机程序，所述计算机程序用于执行本申请实施例提供的文本识别方法的任一实施方式。
本申请实施例还提供了一种计算机程序产品，所述计算机程序产品在终端设备上运行时，使得所述终端设备执行本申请实施例提供的文本识别方法的任一实施方式。
与现有技术相比，本申请实施例至少具有以下优点：
本申请实施例提供的技术方案中，在获取到携带有噪声信息的待识别图像之后，将该待识别图像输入预先构建的文本识别模型，得到该文本识别模型输出的该待识别图像的文本识别结果。其中，文本识别模型包括特征提取网络、循环神经网络和转录层。特征提取网络用于提取待识别图像的图像特征、以及抑制该待识别图像携带的噪声信息。
可见，因特征提取网络具有图像噪声抑制功能，使得利用特征提取网络，针对该待识 别图像提取所得的图像特征携带有较少噪声信息(甚至，不携带噪声信息)，如此能够有效地降低该待识别图像携带的噪声信息对文本识别过程所造成的不良影响，从而能够有效地提高该待识别图像的文本识别结果的准确性，进而有利于提高文本识别效果。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请中记载的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其它的附图。
图1为本申请实施例提供的一种携带有噪声信息的图像数据的示意图；
图2为本申请实施例提供的一种文本识别方法的流程图；
图3为本申请实施例提供的一种背景噪声抑制结果示意图；
图4为本申请实施例提供的一种文本识别模型的结构示意图；
图5为本申请实施例提供的另一种文本识别模型的结构示意图；
图6为本申请实施例提供的又一种文本识别模型的结构示意图；
图7为本申请实施例提供的又一种文本识别模型的工作原理示意图；
图8为本申请实施例提供的一种文本识别装置的结构示意图。
具体实施方式
发明人在针对文本识别技术的研究中发现：
①因卷积循环神经网络(Convolutional RecurrentNeural Network，CRNN)具备推理速度快、模型熟练快、以及泛化性能强等优点，故可以采用CRNN实现文本识别技术。其中，CRNN包括卷积神经网络(Convolutional Neural Networks，CNN)、循环神经网络(Recurrent Neural Network，RNN)和转录层(Connectionist Temporal Classification，CTC)。
②对于一个携带有噪声信息的图像数据(如图1所示的图像数据)来说，因CRNN中CNN不具备噪声抑制功能，使得利用该CNN针对该图像数据提取所得的图像特征携带有大量噪声信息，从而使得这些噪声信息能够严重影响该CRNN中后续处理过程(也就是，RNN的编码过程以及CTC的翻译过程)，如此易导致针对该图像数据的文本识别结果效果很差。
③因CRNN中CTC不具备足够的纠错能力来调整输出结果，使得该CTC无法克服图像特征所携带的噪声信息所导致的不良影响，从而使得针对该图像数据的文本识别结果效果很差。
基于上述发现，为了解决背景技术部分所示的技术问题，本申请实施例提供了一种文本识别方法，该方法包括：在获取到携带有噪声信息的待识别图像之后，将该待识别图像输入预先构建的文本识别模型，得到该文本识别模型输出的该待识别图像的文本识别结果。其中，文本识别模型包括特征提取网络、循环神经网络和转录层。特征提取网络用于提取待识别图像的图像特征、以及抑制该待识别图像携带的噪声信息。
可见，因特征提取网络具有图像噪声抑制功能，使得利用特征提取网络，针对该待识别图像提取所得的图像特征携带有较少噪声信息(甚至，不携带噪声信息)，如此能够有效 地降低该待识别图像携带的噪声信息对文本识别过程所造成的不良影响，从而能够有效地提高该待识别图像的文本识别结果的准确性，进而有利于提高文本识别效果。
另外，本申请实施例不限定文本识别方法的执行主体，例如，本申请实施例提供的文本识别方法可以应用于终端设备或服务器等数据处理设备。其中，终端设备可以为智能手机、计算机、个人数字助理(Personal Digital Assitant，PDA)或平板电脑等。服务器可以为独立服务器、集群服务器或云服务器。
为了使本技术领域的人员更好地理解本申请方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
方法实施例
参见图2，该图为本申请实施例提供的一种文本识别方法的流程图。
本申请实施例提供的文本识别方法，包括S1-S2：
S1：获取待识别图像。
上述“待识别图像”是指需要进行文本识别处理的图像数据(如图1所示的图像数据)；而且该“待识别图像”携带有噪声信息。其中，“噪声信息”是指能够对文本识别处理过程产生不良影响的图像信息。
另外，本申请实施例不限定上述“待识别图像”，例如，其可以是任意一个图像数据。又如，当下文“文本识别模型”的输入数据的高度被预先固定为32时，该“待识别图像”可以是一个高度为32且宽度不限定的图像数据(也就是，32×W的图像数据)。其中，W表示一个图像数据的宽度。
此外，本申请实施例也不限定上述“待识别图像”的获取过程，例如，其具体可以包括：在获取到图像采集数据之后，可以按照预设模型固定高度(例如，32)，对该图像采集数据进行图像尺寸调整，得到待识别图像，以使该待识别图像的高度达到该预设模型固定高度，并使得该待识别图像的宽高比与该图像采集数据的宽高比保持一致。
上述“图像采集数据”是指由图像采集设备(例如，照相机、摄像机等)采集到的图像数据。
上述“预设模型固定高度”是根据下文“文本识别模型”的输入数据的高度需求进行设定的。
基于上述S1的相关内容可知，在获取到一个需要进行文本识别处理的图像采集数据之后，可以针对该图像采集数据进行图像尺寸调整，得到待识别图像，以使该待识别图像的高度达到上述“预设模型固定高度”，并使得该待识别图像的宽高比与该图像采集数据的宽高比保持一致，以便后续能够利用文本识别模型，针对该待识别图像进行文本识别处理，以实现针对上述“图像采集数据”进行文本识别处理的目的。
S2：将待识别图像输入预先构建的文本识别模型，得到该文本识别模型输出的待识别图像的文本识别结果。
上述“待识别图像的文本识别结果”用于表示该待识别图像携带的字符信息。例如，当上述“待识别图像”为图1所示的图像数据时，“待识别图像的文本识别结果”可以是“STARBUCKS”。
上述“文本识别模型”用于针对该文本识别模型的输入数据进行文本识别处理；而且该“文本识别模型”是一种机器学习模型。
另外，上述“文本识别模型”是一种改进的CRNN；而且该“文本识别模型”包括特征提取网络、RNN和CTC。其中，CTC的输入数据包括RNN的输出数据；RNN的输出数据包括特征提取网络的输出数据。
需要说明的是，因上述RNN以及CTC分别采用CRNN中RNN以及CTC进行实施，故为了简要起见，本申请对此不做介绍。另外，上述“特征提取网络”未采用CRNN中CNN进行实施，故为了便于理解，下文详细介绍该“特征提取网络”的相关内容。
上述“特征提取网络”不仅用于针对该特征提取网络的输入数据进行特征提取；而且该“特征提取网络”还用于针对该特征提取网络的输入数据进行噪声抑制处理。可见，在将上文“待识别图像”输入特征提取网络之后，该特征提取网络能够提取该“待识别图像”的图像特征，且还能够抑制该“待识别图像”携带的噪声信息。
另外，本申请实施例不限定上述“特征提取网络”的实施方式，例如，其具体可以包括：图像特征提取子网络和图像噪声抑制子网络。
上述“图像特征提取子网络”用于针对该图像特征提取子网络的输入数据进行图像特征提取处理，以使该“图像特征提取子网络”具有图像特征提取功能。可见，在将上文“待识别图像”输入图像特征提取子网络之后，该图像特征提取子网络能够提取该“待识别图像”的图像特征。
另外，本申请实施例不限定上述“图像特征提取子网络”的实施方式，例如，可以采用现有的或者未来出现的任一种图像特征提取网络(例如，CNN)进行实施。又如，为了兼顾文本识别效率以及文本识别准确性，本申请实施例还提供了上述“图像特征提取子网络”的一种可能的实施方式，例如，其具体可以包括：N个特征提取模块。
上述“特征提取模块”具有图像特征提取功能；而且本申请实施例不限定该“特征提取模块”，例如，因残差块在特征提取和防止梯度消失上具有比较好的效果，故为了提高图像特征提取效果，上述“特征提取模块”可以是残差块(Residual Block)。需要说明的是，一个残差块可以包括两个CNN。
上述“N”表示上述“图像特征提取子网络”中特征提取模块的个数；而且本申请实施例不限定该“N”，例如，为了兼顾文本识别效率以及文本识别准确性，可以预先设定N＝8。
基于上述“图像特征提取子网络”的相关内容可知，在一种可能的实施方式中，该“图像特征提取子网络”可以包括8个残差块，以使该“图像特征提取子网络”具有较好的图像特征提取效果，以便后续该“图像特征提取子网络”能够被用于提取一个图像数据(例如，上文“待识别图像”)的图像特征。
上述“图像噪声抑制子网络”用于针对该图像噪声抑制子网络的输入数据进行噪声抑制处理，以使该“图像噪声抑制子网络”具有图像噪声抑制功能，从而使得该“图像噪声抑制子网络”能够被用于抑制一个图像数据(例如，上文“待识别图像”)携带的噪声信息。
另外，本申请实施例不限定上述“图像噪声抑制子网络”的实施方式，为了便于理解，下面结合三种情况进行说明。
情况1，为了抑制图像数据所携带的背景噪声，可以采用空间上的全局注意力机制(Global-Attention)进行实施。基于此，本申请实施例提供了上述“图像噪声抑制子网络”的一种可能的实施方式，其具体可以包括：空间噪声抑制模块。其中，“空间噪声抑制模块”用于抑制一个图像数据(例如，上文“待识别图像”)携带的背景噪声。
为了便于理解上述“空间噪声抑制模块”的工作原理，下面以上文“待识别图像的文本识别结果”的确定过程为例进行说明。
作为示例，当上述“文本识别模型”包括特征提取网络、RNN和CTC，该特征提取网络包括图像特征提取子网络和图像噪声抑制子网络，且该图像噪声抑制子网络包括空间噪声抑制模块时，“待识别图像的文本识别结果”的确定过程，具体可以包括步骤11-步骤13：
步骤11：将待识别图像输入图像特征提取子网络，得到该图像特征提取子网络输出的待使用图像特征。
其中，“待使用图像特征”是指上文“待识别图像”的图像特征，以使该“待使用图像特征”用于表示待识别图像携带的图像信息(例如，字符信息以及噪声信息)。
步骤12：将待使用图像特征输入空间噪声抑制模块，得到该空间噪声抑制模块输出的待处理图像特征。
上述“空间噪声抑制模块”用于针对该空间噪声抑制模块的输入数据进行背景噪声抑制处理。
另外，本申请实施例不限定上述“空间噪声抑制模块”的实施方式，例如，其可以基于空间上的全局注意力机制进行实施。基于此，本申请实施例提供了上述“空间噪声抑制模块”的一种可能的实施方式，其具体可以包括：空间权重确定子模块和特征加权融合子模块；而且该特征加权融合子模块的输入数据包括该空间权重确定子模块的输出数据以及上文“图像特征提取子网络”的输出数据。
为了便于理解上述“空间噪声抑制模块”的工作原理，下面以上述“待处理图像特征”的确定过程为例进行说明。
作为示例，当上述“空间噪声抑制模块”包括空间权重确定子模块和特征加权融合子模块时，“待处理图像特征”的确定过程具体可以包括步骤21-步骤22：
步骤21：将待使用图像特征输入空间权重确定子模块，得到该空间权重确定子模块输出的待使用空间权重。
上述“空间权重确定子模块”用于针对该空间权重确定子模块的输入数据进行空间权重确定处理(也就是，背景噪声影响权重确定处理)。
另外，本申请实施例不限定上述“空间权重确定子模块”的实施方式，例如，其可以采用一层卷积神经网络进行实施。
上述“待使用空间权重”用于表示上述“待使用图像特征”中各个特征元素的加权权重。例如，当上述“待使用图像特征”为R×E的特征图(如公式(1)所示)时，该“待使用空间权重”可以是一个R×E的矩阵(如公式(2)所示)，且该“待使用空间权重”中每一列元素的加和为1(如公式(3)所示)。
Figure PCTCN2022124761-appb-000001
Figure PCTCN2022124761-appb-000002
Figure PCTCN2022124761-appb-000003
式中，F use表示上述“待使用图像特征”；D space表示上述“待使用空间权重”；R表示上述“待使用图像特征”的特征图高度，且R为正整数；E表示上述“待使用图像特征”的特征图宽度，且E为正整数。
步骤22：将待使用图像特征和待使用空间权重输入特征加权融合子模块，得到该特征加权融合子模块输出的待处理图像特征。
上述“特征加权融合子模块”用于按照待使用空间权重，对待使用图像特征进行加权求和处理(如公式(4)-(5)所示)。
F discern＝[b 1 … b E]          (4)
Figure PCTCN2022124761-appb-000004
式中，F discern表示待处理图像特征。
基于上述步骤21至步骤22的相关内容可知，在获取到待使用图像特征之后，可以利用上述公式(1)-(5)，对该待使用图像特征进行背景噪声抑制处理，以拉大用于表示字符信息的特征元素与用于表示背景噪声的特征元素之间的差距，得到待处理图像特征，以使该待处理图像特征能够达到高亮字符信息且暗淡背景噪声的效果(如图3所示的效果)，从而使得该待处理图像特征能够明显地区分开上述“待识别图像”携带的字符信息以及背景噪声，如此能够有效地避免在直接将上述待使用图像特征的特征图高度压缩为1时，所导致的字符信息与背景噪声融为一体的不良影响，从而能够有效地提高图像特征提取效果。
基于上述步骤12的相关内容可知，在获取到待使用图像特征之后，可以利用空间噪声抑制模块，对该待使用图像特征进行背景噪声抑制处理，得到并输出待处理图像特征，以使该待处理图像特征中用于表示字符信息的特征元素与用于表示背景噪声的特征元素之间呈现较大的差异性，从而使得该待处理图像特征能够明显地区分开上述“待识别图像”携带的字符信息以及背景噪声，如此有利于提高图像特征提取效果。
步骤13：根据待处理图像特征、RNN和CTC，确定待识别图像的文本识别结果。
需要说明的是，本申请实施例不限定步骤13的实施方式，例如，该步骤13可以采用在CRNN中所使用的“根据CNN的输出数据、RNN和CTC确定文本识别结果”的实现过程进行实施。
基于上述步骤11至步骤13的相关内容可知，如图4所示，当上文“文本识别模型”包括特征提取网络、RNN和CTC，且该特征提取网络包括图像特征提取子网络和空间噪声抑制模块时，该空间噪声抑制模块的输入数据可以包括该图像特征提取子网络的输出数据，且RNN的输入数据可以包括该该空间噪声抑制模块的输出数据。
基于上述情况1的相关内容可知，对于上文“文本识别模型”来说，其可以借助上述“空间噪声抑制模块”，实现针对一个图像数据(例如，上文“待识别图像”)携带的背景噪声进行抑制处理，以使该图像数据携带的背景噪声几乎不会影响该图像数据携带的字符信息的特征表达，如此能够有效地降低该图像数据携带的背景噪声对RNN以及CTC所造成的不良影响，从而有利于提高文本识别效果。
情况2，可以采用通道注意力机制(Squeeze-and-Excitation Networks，SE-Net)进行通道噪声抑制。基于此，本申请实施例提供了上述“图像噪声抑制子网络”的另一种可能的实施方式，其具体可以包括：通道噪声抑制模块。其中，“通道噪声抑制模块”用于实现通道注意力机制。
为了便于理解，下面结合示例进行说明。
作为示例，当上文“文本识别模型”包括特征提取网络、RNN和CTC，且该特征提取网络包括图像特征提取子网络和图像噪声抑制子网络时，该图像特征提取子网络可以包括N个特征提取模块，且该图像噪声抑制子网络可以包括N个通道噪声抑制模块。其中，第n个通道噪声抑制模块的输入数据包括第n个特征提取模块的输出数据；其中，n为正整数，n≤N，N为正整数。另外，第1个特征提取模块的输入数据包括待识别图像，且第i+1个特征提取模块的输入数据包括第i个通道噪声抑制模块的输出数据；其中，i为正整数，i+1≤N，N为正整数。
可见，当上文“文本识别模型”为图5所示的文本识别模型时，该“文本识别模型”的工作原理如下：
先由第1个特征提取模块针对一个图像数据(例如，上文“待识别图像”)进行图像特征提取处理，得到该第1个特征提取模块的输出数据；再由第1个通道噪声抑制模块针对该第1个特征提取模块的输出数据进行通道噪声抑制处理，得到该第1个通道噪声抑制模块的输出数据；再由第2个特征提取模块针对该第1个通道噪声抑制模块的输出数据进行图像特征提取处理，得到该第2个特征提取模块的输出数据；再由第2个通道噪声抑制模块针对该第2个特征提取模块的输出数据进行通道噪声抑制处理，得到该第2个通道噪声抑制模块的输出数据；……(以此类推)；再由第N个特征提取模块针对该第N-1个通道噪声抑制模块的输出数据进行图像特征提取处理，得到该第N个特征提取模块的输出数据；再由第N个通道噪声抑制模块针对该第N个特征提取模块的输出数据进行通道噪声抑制处理，得到该第N个通道噪声抑制模块的输出数据；最后，根据该第N个通道噪声抑制模块的输出数据、RNN和CTC，确定该图像数据的文本识别结果。
另外，上述各个通道噪声抑制模块可以采用通道衰减的策略来降低网络成本，以实现在工程化之后该通道噪声抑制模块几乎没有时间上的额外开销，如此有利于提高文本识别效率。
基于上述情况2的相关内容可知，对于上文“文本识别模型”来说，其可以借助上述“通道噪声抑制模块”，实现针对通道噪声抑制处理，以使通道噪声几乎不会影响一个图像数据携带的字符信息的特征表达，如此能够有效地降低通道噪声对RNN以及CTC所造成的不良影响，从而有利于提高文本识别效果。
情况3，为了克服上述两种情况所示的噪声，本申请提供了上述“图像噪声抑制子网络”的又一种可能的实施方式，其具体可以包括：空间噪声抑制模块和通道噪声抑制模块。为了便于理解，下面结合示例进行说明。
作为示例，当上文“文本识别模型”包括特征提取网络、RNN和CTC，且该特征提取网络包括图像特征提取子网络和图像噪声抑制子网络时，该图像特征提取子网络可以包括N个特征提取模块，且该图像噪声抑制子网络可以包括N个通道噪声抑制模块和空间噪声抑制模块。其中，第n个通道噪声抑制模块的输入数据包括第n个特征提取模块的输出数据；其中，n为正整数，n≤N，N为正整数。另外，第1个特征提取模块的输入数据包括待识别图像，且第i+1个特征提取模块的输入数据包括第i个通道噪声抑制模块的输出数据；其中，i为正整数，i+1≤N，N为正整数。空间噪声抑制模块的输入数据包括第N个通道噪声抑制模块的输出数据。
可见，当上文“文本识别模型”为图6所示的文本识别模型时，该“文本识别模型”的工作原理(如图7所示)如下：
先由第1个特征提取模块针对一个图像数据(例如，上文“待识别图像”)进行图像特征提取处理，得到该第1个特征提取模块的输出数据；再由第1个通道噪声抑制模块针对该第1个特征提取模块的输出数据进行通道噪声抑制处理，得到该第1个通道噪声抑制模块的输出数据；再由第2个特征提取模块针对该第1个通道噪声抑制模块的输出数据进行图像特征提取处理，得到该第2个特征提取模块的输出数据；再由第2个通道噪声抑制模块针对该第2个特征提取模块的输出数据进行通道噪声抑制处理，得到该第2个通道噪声抑制模块的输出数据；……(以此类推)；再由第N个特征提取模块针对该第N-1个通道噪声抑制模块的输出数据进行图像特征提取处理，得到该第N个特征提取模块的输出数据；再由第N个通道噪声抑制模块针对该第N个特征提取模块的输出数据进行通道噪声抑制处理，得到该第N个通道噪声抑制模块的输出数据；然后，由空间噪声抑制模块针对该第N个通道噪声抑制模块的输出数据进行背景噪声抑制处理，得到该空间噪声抑制模块的输出数据；最后，根据该空间噪声抑制模块的输出数据、RNN和CTC，确定该图像数据的文本识别结果。
需要说明的是，图7中“BLSTM”是指双向长短时记忆网络(Bi-directional Long Short Term Memory Network，BLSTM)；而且BLSTM是一种RNN。
基于上述“文本识别模型”的相关内容可知，对于本申请实施例提供的“文本识别模型”来说，因该“文本识别模型”中图像噪声抑制子网络具有噪声抑制功能，使得该“文本识别模型”中特征提取网络也具有噪声抑制功能，从而使得该“文本识别模型”也具有噪声抑制功能，进而使得利用该“文本识别模型”针对一个图像数据所得到的文本识别结果更准确，如此有利于提高文本识别效果。
需要说明的是，本申请实施例不限定上述“文本识别模型”的构建过程，可以采用现有的或者未来出现的任一种机器学习模型构建方法进行实施即可。
基于上述S1至S2的相关内容可知，对于本申请实施例提供的文本识别方法来说，在获取到携带有噪声信息的待识别图像之后，将该待识别图像输入预先构建的文本识别模型，得到该文本识别模型输出的该待识别图像的文本识别结果。其中，文本识别模型包括特征 提取网络、循环神经网络和转录层。特征提取网络用于提取待识别图像的图像特征、以及抑制该待识别图像携带的噪声信息。
可见，因特征提取网络具有图像噪声抑制功能，使得利用特征提取网络，针对该待识别图像提取所得的图像特征携带有较少噪声信息(甚至，不携带噪声信息)，如此能够有效地降低该待识别图像携带的噪声信息对文本识别过程所造成的不良影响，从而能够有效地提高该待识别图像的文本识别结果的准确性，进而有利于提高文本识别效果。
基于上述方法实施例提供的文本识别方法，本申请实施例还提供了一种文本识别装置，下面结合附图进行解释和说明。
装置实施例
装置实施例提供的文本识别装置的技术详情，请参照上述方法实施例。
参见图8，该图为本申请实施例提供的一种文本识别装置的结构示意图。
本申请实施例提供的文本识别装置800，包括：
获取单元801，用于获取待识别图像；
识别单元802，用于将所述待识别图像输入预先构建的文本识别模型，得到所述文本识别模型输出的所述待识别图像的文本识别结果；其中，所述文本识别模型包括特征提取网络、循环神经网络RNN和转录层CTC；所述特征提取网络用于提取所述待识别图像的图像特征，且所述特征提取网络还用于抑制所述待识别图像携带的噪声信息。
在一种可能的实施方式中，所述特征提取网络包括图像特征提取子网络和图像噪声抑制子网络；所述图像特征提取子网络用于提取所述待识别图像的图像特征；所述图像噪声抑制子网络用于抑制所述待识别图像携带的噪声信息。
在一种可能的实施方式中，所述图像噪声抑制子网络包括空间噪声抑制模块和/或通道噪声抑制模块；其中，所述空间噪声抑制模块用于抑制所述待识别图像携带的背景噪声；所述通道噪声抑制模块用于实现通道注意力机制。
在一种可能的实施方式中，所述图像噪声抑制子网络包括空间噪声抑制模块；
所述识别单元802，具体用于：将所述待识别图像输入所述图像特征提取子网络，得到所述图像特征提取子网络输出的待使用图像特征；将所述待使用图像特征输入所述空间噪声抑制模块，得到所述空间噪声抑制模块输出的待处理图像特征；根据所述待处理图像特征、所述RNN和所述CTC，确定所述待识别图像的文本识别结果。
在一种可能的实施方式中，所述空间噪声抑制模块包括空间权重确定子模块和特征加权融合子模块；
所述待处理图像特征的确定过程，包括：将所述待使用图像特征输入所述空间权重确定子模块，得到所述空间权重确定子模块输出的待使用空间权重；将所述待使用图像特征和所述待使用空间权重输入所述特征加权融合子模块，得到所述特征加权融合子模块输出的待处理图像特征。
在一种可能的实施方式中，所述图像特征提取子网络包括N个特征提取模块；所述图像噪声抑制子网络包括N个通道噪声抑制模块；
第n个通道噪声抑制模块的输入数据包括第n个特征提取模块的输出数据；其中，n为正整数，n≤N，N为正整数；
第1个特征提取模块的输入数据包括所述待识别图像，且第i+1个特征提取模块的输入数据包括第i个通道噪声抑制模块的输出数据；其中，i为正整数，i+1≤N，N为正整数。
在一种可能的实施方式中，所述图像特征提取子网络包括N个特征提取模块；所述图像噪声抑制子网络包括N个通道噪声抑制模块和空间噪声抑制模块；
第n个通道噪声抑制模块的输入数据包括第n个特征提取模块的输出数据；其中，n为正整数，n≤N，N为正整数；
第1个特征提取模块的输入数据包括所述待识别图像，且第i+1个特征提取模块的输入数据包括第i个通道噪声抑制模块的输出数据；其中，i为正整数，i+1≤N，N为正整数；
所述空间噪声抑制模块的输入数据包括第N个通道噪声抑制模块的输出数据。
基于上述文本识别装置800的相关内容可知，对于本申请实施例提供的文本识别装置800来说，在获取到携带有噪声信息的待识别图像之后，将该待识别图像输入预先构建的文本识别模型，得到该文本识别模型输出的该待识别图像的文本识别结果。其中，文本识别模型包括特征提取网络、循环神经网络和转录层。特征提取网络用于提取待识别图像的图像特征、以及抑制该待识别图像携带的噪声信息。
可见，因特征提取网络具有图像噪声抑制功能，使得利用特征提取网络，针对该待识别图像提取所得的图像特征携带有较少噪声信息(甚至，不携带噪声信息)，如此能够有效地降低该待识别图像携带的噪声信息对文本识别过程所造成的不良影响，从而能够有效地提高该待识别图像的文本识别结果的准确性，进而有利于提高文本识别效果。
进一步地，本申请实施例还提供了一种设备，所述设备包括处理器以及存储器：
所述存储器用于存储计算机程序；
所述处理器用于根据所述计算机程序执行本申请实施例提供的文本识别方法的任一实施方式。
进一步地，本申请实施例还提供了一种计算机可读存储介质，所述计算机可读存储介质用于存储计算机程序，所述计算机程序用于执行本申请实施例提供的文本识别方法的任一实施方式。
进一步地，本申请实施例还提供了一种计算机程序产品，所述计算机程序产品在终端设备上运行时，使得所述终端设备执行本申请实施例提供的文本识别方法的任一实施方式。
应当理解，在本申请中，“至少一个(项)”是指一个或者多个，“多个”是指两个或两个以上。“和/或”，用于描述关联对象的关联关系，表示可以存在三种关系，例如，“A和/或B”可以表示：只存在A，只存在B以及同时存在A和B三种情况，其中A，B可以是单数或者复数。字符“/”一般表示前后关联对象是一种“或”的关系。“以下至少一项(个)”或其类似表达，是指这些项中的任意组合，包括单项(个)或复数项(个)的任意组合。例如，a，b或c中的至少一项(个)，可以表示：a，b，c，“a和b”，“a和c”，“b和c”，或“a和b和c”，其中a，b，c可以是单个，也可以是多个。
以上所述，仅是本发明的较佳实施例而已，并非对本发明作任何形式上的限制。虽然本发明已以较佳实施例揭露如上，然而并非用以限定本发明。任何熟悉本领域的技术人员，在不脱离本发明技术方案范围情况下，都可利用上述揭示的方法和技术内容对本发明技术方案做出许多可能的变动和修饰，或修改为等同变化的等效实施例。因此，凡是未脱离本 发明技术方案的内容，依据本发明的技术实质对以上实施例所做的任何简单修改、等同变化及修饰，均仍属于本发明技术方案保护的范围内。

Claims (11)

  1. 一种文本识别方法，所述方法包括：
    获取待识别图像；
    将所述待识别图像输入预先构建的文本识别模型，得到所述文本识别模型输出的所述待识别图像的文本识别结果；其中，所述文本识别模型包括特征提取网络、循环神经网络RNN和转录层CTC；所述特征提取网络用于提取所述待识别图像的图像特征，且所述特征提取网络还用于抑制所述待识别图像携带的噪声信息。
  2. 根据权利要求1所述的方法，其中，所述特征提取网络包括图像特征提取子网络和图像噪声抑制子网络；所述图像特征提取子网络用于提取所述待识别图像的图像特征；所述图像噪声抑制子网络用于抑制所述待识别图像携带的噪声信息。
  3. 根据权利要求2所述的方法，其中，所述图像噪声抑制子网络包括空间噪声抑制模块和/或通道噪声抑制模块；其中，所述空间噪声抑制模块用于抑制所述待识别图像携带的背景噪声；所述通道噪声抑制模块用于实现通道注意力机制。
  4. 根据权利要求2所述的方法，其中，所述图像噪声抑制子网络包括空间噪声抑制模块；
    所述文本识别结果的确定过程，包括：
    将所述待识别图像输入所述图像特征提取子网络，得到所述图像特征提取子网络输出的待使用图像特征；
    将所述待使用图像特征输入所述空间噪声抑制模块，得到所述空间噪声抑制模块输出的待处理图像特征；
    根据所述待处理图像特征、所述RNN和所述CTC，确定所述待识别图像的文本识别结果。
  5. 根据权利要求4所述的方法，其中，所述空间噪声抑制模块包括空间权重确定子模块和特征加权融合子模块；
    所述待处理图像特征的确定过程，包括：
    将所述待使用图像特征输入所述空间权重确定子模块，得到所述空间权重确定子模块输出的待使用空间权重；
    将所述待使用图像特征和所述待使用空间权重输入所述特征加权融合子模块，得到所述特征加权融合子模块输出的待处理图像特征。
  6. 根据权利要求2所述的方法，其中，所述图像特征提取子网络包括N个特征提取模块；所述图像噪声抑制子网络包括N个通道噪声抑制模块；
    第n个通道噪声抑制模块的输入数据包括第n个特征提取模块的输出数据；其中，n为正整数，n≤N，N为正整数；
    第1个特征提取模块的输入数据包括所述待识别图像，且第i+1个特征提取模块的输入数据包括第i个通道噪声抑制模块的输出数据；其中，i为正整数，i+1≤N，N为正整数。
  7. 根据权利要求2所述的方法，其中，所述图像特征提取子网络包括N个特征提取模块；所述图像噪声抑制子网络包括N个通道噪声抑制模块和空间噪声抑制模块；
    第n个通道噪声抑制模块的输入数据包括第n个特征提取模块的输出数据；其中，n为正整数，n≤N，N为正整数；
    第1个特征提取模块的输入数据包括所述待识别图像，且第i+1个特征提取模块的输入数据包括第i个通道噪声抑制模块的输出数据；其中，i为正整数，i+1≤N，N为正整数；
    所述空间噪声抑制模块的输入数据包括第N个通道噪声抑制模块的输出数据。
  8. 一种文本识别装置，包括：
    获取单元，用于获取待识别图像；
    识别单元，用于将所述待识别图像输入预先构建的文本识别模型，得到所述文本识别模型输出的所述待识别图像的文本识别结果；其中，所述文本识别模型包括特征提取网络、循环神经网络RNN和转录层CTC；所述特征提取网络用于提取所述待识别图像的图像特征，且所述特征提取网络还用于抑制所述待识别图像携带的噪声信息。
  9. 一种设备，所述设备包括处理器以及存储器：
    所述存储器用于存储计算机程序；
    所述处理器用于根据所述计算机程序执行权利要求1-7中任一项所述的方法。
  10. 一种计算机可读存储介质，所述计算机可读存储介质用于存储计算机程序，所述计算机程序用于执行权利要求1-7中任一项所述的方法。
  11. 一种计算机程序产品，所述计算机程序产品在终端设备上运行时，使得所述终端设备执行权利要求1-7中任一项所述的方法。
PCT/CN2022/124761 2021-12-21 2022-10-12 一种文本识别方法及其相关设备 Ceased WO2023116137A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111575226.0 2021-12-21
CN202111575226.0A CN114241467A (zh) 2021-12-21 2021-12-21 一种文本识别方法及其相关设备

Publications (1)

Publication Number Publication Date
WO2023116137A1 true WO2023116137A1 (zh) 2023-06-29

Family

ID=80760787

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/124761 Ceased WO2023116137A1 (zh) 2021-12-21 2022-10-12 一种文本识别方法及其相关设备

Country Status (2)

Country Link
CN (1) CN114241467A (zh)
WO (1) WO2023116137A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114241467A (zh) * 2021-12-21 2022-03-25 北京有竹居网络技术有限公司 一种文本识别方法及其相关设备

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130003149A1 (en) * 2011-07-01 2013-01-03 Yoshinori Murakami Image processing apparatus, image forming apparatus, image reading apparatus, image processing method, and recording medium
CN111028159A (zh) * 2019-10-22 2020-04-17 西安电子科技大学 一种图像条带噪声抑制方法及系统
CN112668580A (zh) * 2020-12-28 2021-04-16 南京航天数智科技有限公司 一种文本识别方法、文本识别装置及终端设备
CN114241467A (zh) * 2021-12-21 2022-03-25 北京有竹居网络技术有限公司 一种文本识别方法及其相关设备

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111461105B (zh) * 2019-01-18 2023-11-28 顺丰科技有限公司 一种文本识别方法和装置
CN111783756B (zh) * 2019-04-03 2024-04-16 北京市商汤科技开发有限公司 文本识别方法及装置、电子设备和存储介质
CN111126410B (zh) * 2019-12-31 2022-11-18 讯飞智元信息科技有限公司 字符识别方法、装置、设备及可读存储介质
CN111428593A (zh) * 2020-03-12 2020-07-17 北京三快在线科技有限公司 一种文字识别方法、装置、电子设备及存储介质
CN112541466A (zh) * 2020-12-21 2021-03-23 山东师范大学 基于上下文注意力机制网络的人群计数方法及系统
CN113807340B (zh) * 2021-09-07 2024-03-15 南京信息工程大学 一种基于注意力机制的不规则自然场景文本识别方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130003149A1 (en) * 2011-07-01 2013-01-03 Yoshinori Murakami Image processing apparatus, image forming apparatus, image reading apparatus, image processing method, and recording medium
CN111028159A (zh) * 2019-10-22 2020-04-17 西安电子科技大学 一种图像条带噪声抑制方法及系统
CN112668580A (zh) * 2020-12-28 2021-04-16 南京航天数智科技有限公司 一种文本识别方法、文本识别装置及终端设备
CN114241467A (zh) * 2021-12-21 2022-03-25 北京有竹居网络技术有限公司 一种文本识别方法及其相关设备

Also Published As

Publication number Publication date
CN114241467A (zh) 2022-03-25

Similar Documents

Publication Publication Date Title
KR102683700B1 (ko) 비디오 처리 방법, 장치, 전자 기기, 저장 매체 및 컴퓨터 프로그램
CN109993040B (zh) 文本识别方法及装置
CN107291945B (zh) 基于视觉注意力模型的高精度服装图像检索方法及系统
KR102576344B1 (ko) 비디오를 처리하기 위한 방법, 장치, 전자기기, 매체 및 컴퓨터 프로그램
CN114973317B (zh) 一种基于多尺度邻接交互特征的行人重识别方法
WO2020125229A1 (zh) 特征融合方法、装置、电子设备及存储介质
US20240127406A1 (en) Image quality adjustment method and apparatus, device, and medium
WO2023231355A1 (zh) 图像识别方法及装置
CN108154080A (zh) 一种视频设备快速溯源的方法
CN111860683A (zh) 一种基于特征融合的目标检测方法
CN107330387B (zh) 基于图像数据的行人检测方法
CN114581926A (zh) 多行文本识别方法、装置、设备及介质
CN116304179A (zh) 一种获取目标视频的数据处理系统
WO2022042609A1 (zh) 提取热词的方法、装置、电子设备及介质
US20240386714A1 (en) Video representation self-supervised contrastive learning method and apparatus
CN114429633B (zh) 文本识别方法、模型的训练方法、装置、电子设备及介质
WO2023116137A1 (zh) 一种文本识别方法及其相关设备
CN112529018A (zh) 图像局部特征的训练方法、装置及存储介质
CN115730073A (zh) 文本处理方法、装置及存储介质
CN110633717A (zh) 一种目标检测模型的训练方法和装置
CN111814534A (zh) 视觉任务的处理方法、装置和电子系统
CN115331140A (zh) 一种动作识别中基于通道分组的时空特征分离提取方法
CN108537109A (zh) 基于OpenPose的单目相机手语识别方法
CN112348024B (zh) 一种基于深度学习优化网络的图文识别方法及系统
CN114429628A (zh) 图像处理方法、装置、可读存储介质及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22909455

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22909455

Country of ref document: EP

Kind code of ref document: A1