JP7429307B2 - Character string recognition method, device, equipment and medium based on computer vision - Google Patents

Description

The present application relates to a computer vision-based character string recognition method, apparatus, computer equipment, and storage medium.

This application claims the priority of the Chinese application filed on July 3, 2020, whose invention is "Character string recognition method, device, device, and medium based on computer vision" and whose application number is 202010630553.0. The disclosure of that application is incorporated herein by reference in its entirety.

With the development of computer vision technology, recognition of character strings has become one of the practical applications of computer vision technology in life. For example, in industrial scenes, recognition of character strings such as serial numbers, manufacturing dates, stencils, inscriptions, etc. Recognize columns. Generally, in the recognition process for a character string, the position of the character string is first detected, the character string at the detected position is trimmed, and finally, the angle of the trimmed character string image is determined and recognized, and the corresponding text content is Alternatively, the string can be detected as a special target, detected by a classifier and aggregated as a single word based on a model of the image structure, or a neural network algorithm can combine image features and string position. Recognize strings by creating matching relationships with corresponding content.

According to embodiments, a first aspect of the present application includes:
obtaining an image with a character string to be recognized;
a step of acquiring a target area image in which the recognition target character string is located among the images to which the recognition target character string is attached, based on a position detection model constructed in advance;
horizontally correcting the target area image to obtain a horizontal target area image;
acquiring the standing state of the character string in the horizontal target area image based on a pre-built angle determination model;
If the standing state of the character string is an upright state, inputting the horizontal target area image to a content recognition model built in advance to obtain character string content corresponding to the character string to be recognized. A method for character string recognition based on computer vision is provided.

According to embodiments, a second aspect of the present application includes:
an image acquisition module that acquires an image with a recognition target character string;
a position detection module that acquires a target area image in which the recognition target character string is located among the images with the recognition target character string, based on a position detection model built in advance;
a lateral correction module that laterally corrects the target area image to obtain a lateral target area image;
an angle determination module that acquires a standing state of a character string in the horizontal target area image based on a pre-built angle determination model;
When the character string is in an upright state, a content recognition module inputs the horizontal target area image into a pre-built content recognition model and obtains character string content corresponding to the recognition target character string. Provided is a character string recognition device based on computer vision, including the following.

According to embodiments, a third aspect of the present application includes:
A computer device is provided that includes a memory in which a computer program is stored and a processor that, when executed, implements the steps of the method.

According to embodiments, a fourth aspect of the present application includes:
A computer readable storage medium is provided having stored thereon a computer program which, when executed by a processor, implements the steps of the above method.

The details of one or more implementations of the present application are set forth in the drawings and description below. Other features and advantages of the present application will be apparent from the specification, drawings, and claims.

In order to more clearly explain the embodiments of the present application or the technical solutions of the prior art, drawings necessary for the embodiments or conventional explanation will be briefly explained below, but the drawings in the following explanation are part of the present application. It is clear that those skilled in the art can also derive other drawings based on these drawings without any creative effort.

FIG. 2 is a schematic diagram of the flow of a computer vision-based character string recognition method in one embodiment. FIG. 3 is a schematic diagram of a flow of acquiring the standing state of a character string in a horizontal target area image based on a pre-built angle determination model in an embodiment. FIG. 2 is a schematic diagram of a flow of acquiring a target area image in which a recognition target character string is located in an image based on a position detection model built in advance in one embodiment. FIG. 2 is a schematic diagram of a flow of inputting a horizontal target area image into a content recognition model built in advance and acquiring character string content corresponding to a recognition target character string in one embodiment. FIG. 3 is a schematic diagram of the flow of a computer vision-based character string recognition method in another embodiment. FIG. 2 is a schematic diagram of the flow of algorithm training and prediction processing in one application example. FIG. 2 is a structural schematic diagram of an image feature pyramid in one application example. FIG. 2 is a schematic diagram of the flow of a character string angle determination algorithm in one application example. FIG. 2 is a schematic diagram of the flow of a string content recognition algorithm in one application example. 1 is a structural block diagram of a character string recognition device based on computer vision in one embodiment. FIG. FIG. 2 is an internal structural diagram of computer equipment in one embodiment.

Current character string recognition methods are all based on low-dimensional manual features and lack the ability to adapt to changes in image capture angles in industrial scenes, resulting in a low recognition accuracy rate for character strings.

In order to make the objectives, technical solutions and advantages of the present application more clear, the present application will be described in more detail below with reference to the drawings and examples. Note that the specific examples described here are only for the purpose of interpreting the present application, and are not intended to limit the present application.

In one embodiment, a character string recognition method based on computer vision is provided, as shown in FIG. It is understood that the invention may be applied to a system comprising a terminal and a server and may be implemented through interaction between the terminal and the server. In this embodiment, the method includes steps S101 to S105.

In step S101, the terminal acquires an image with a recognition target character string attached.

Here, the recognition target character string refers to a character string that the user should obtain from an image, and the image may be an image taken in an industrial scene. Specifically, the user records images with target character strings attached to them from various scenes using a mobile phone camera or video collection device, stores these images in the terminal, and the terminal records images with target character strings attached to them from various scenes. It is also possible to obtain an attached image.

In step S102, the terminal acquires a target area image in which the recognition target character string is located, out of the images with the recognition target character string, based on a position detection model constructed in advance.

Here, the position detection model mainly detects the position area of the character to be recognized within an image, and the target area image refers to the image of the position area of the character string to be recognized in the image. Specifically, the terminal uses a position detection model built in advance to perform character string position detection on an image with the recognition target character string, thereby detecting the target area image where the recognition target character string is located. may be determined.

In step S103, the terminal corrects the target area image in the horizontal direction to obtain a horizontal target area image.

It is common for users to take images of recognition target character strings from various shooting angles, so in images with recognition target character strings obtained by a terminal, the recognition target character strings are arranged horizontally. It is often expressed at an angle with the horizontal direction, rather than at a horizontal angle. Therefore, in order to improve the accuracy of character string recognition, the terminal needs to obtain a target area image in the horizontal direction by correcting the target area image in the horizontal direction after obtaining the target area image in step S102. be. Within the horizontal target area image, the recognition target character strings are arranged horizontally. Specifically, the terminal may perform lateral correction by performing affine transformation on the target area image to obtain a lateral target area image.

In step S104, the terminal acquires the standing state of the character string in the horizontal target area image based on the angle determination model constructed in advance.

In step S103, after completing the horizontal direction correction on the target area image, the terminal determines that the character string in the obtained horizontal target area image is in an upright position based on the angle of the initial photographed image of the user. In the case of an inverted state, the final character string recognition result is affected by the deviation in the standing state of the character string. Therefore, after obtaining the horizontal target area image, the terminal needs to determine the standing state of the character string in the obtained horizontal target area image. Specifically, the terminal may determine the standing state of the character string in the horizontal target area image by inputting the horizontal target area image into an angle determination model constructed in advance.

In step S105, if the standing state of the character string is an upright state, the terminal inputs the horizontal target area image to a content recognition model built in advance, and acquires the character string content corresponding to the character string to be recognized. do.

On the other hand, if the terminal determines that the standing state of the character string at this time is the upright state, the terminal may directly input the horizontal target area image into a pre-built content recognition model, and the content recognition model mainly uses This method recognizes the content of a character string in a target area image, and therefore, the terminal may use this content recognition model to obtain character string content corresponding to a character string to be recognized.

In the computer vision-based character string recognition method described above, a terminal acquires an image with a recognition target character string attached, and based on a pre-built position detection model, targets the target character string in the image where the recognition target character string is located. Acquire a region image, correct the target region image in the horizontal direction to obtain a horizontal target region image, and determine the standing state of the character string in the horizontal target region image based on a pre-built angle determination model. If the character string is acquired and the standing state of the character string is an upright state, the horizontal target area image is input to a content recognition model built in advance, and character string content corresponding to the character string to be recognized is acquired. In the present application, the terminal corrects the target area image in the horizontal direction, thereby achieving adaptive processing to changes in the image capturing angle in an industrial scene, and improving the recognition accuracy rate for character strings.

In one embodiment, as shown in FIG. 2, step S104 includes step S201 and step S202.

In step S201, the terminal acquires the upright angle of the target area image in the lateral direction based on the angle determination model.

Here, the angle determination model mainly determines the angle of the target area image in the horizontal direction, and since the standing state of the character string is mainly determined by the angle of the initial photographed image of the user, the terminal uses this angle determination model. The standing angle of the target area image in the horizontal direction may be determined by the model, and the standing state of the character string may be determined using the standing angle.

In step S202, the terminal determines the standing state of the character string from the standing angle section to which the standing angle belongs.

On the other hand, in order to avoid a slight deviation between the rising angle of the lateral target area image obtained by the terminal and the standard lateral angle, in step S201, the terminal determines the rising angle using the angle determination model. After that, from the preset standing angle interval table, select the standing angle interval suitable for the standing angle as the standing angle interval to which the standing angle belongs, and use the standing angle interval to determine the standing state of the character string. You may.

Further, the standing angle section may include a first angle section and a second angle section, and the standing state of the character string may include an upright state and an inverted state, and step S202 is performed when the standing angle section is the first angle section. In some cases, the terminal may further include determining the standing state of the character string as an upright state; and, if the standing angle section is a second angle section, determining the standing state of the character string as an inverted state. .

Here, the first angular interval and the second angular interval are two different angular intervals, and represent two types of standing states of the character string, respectively. Specifically, when the upright angle interval to which the upright angle of the horizontal target area image obtained by the terminal belongs is the first angle interval, the terminal sets the horizontal target area image at this time as the upright state. On the other hand, if the upright angle interval to which the upright angle of the horizontal target area image obtained by the terminal belongs is the second angle interval, the terminal may invert the horizontal target area image at this time. It may also be determined as a state.

Further, when the standing state of the character string is an inverted state, the horizontal target area image is rotated to an upright state and input to the content recognition model to obtain the character string content.

If the terminal directly inputs an inverted horizontal target area image into the content recognition model, there is a risk that a discrepancy will occur between the character string content obtained by the content recognition model and the actual character content. Therefore, before inputting the horizontal target area image into the content recognition model, it is necessary to rotate the horizontal target area image so that it is upright. By rotating 180 degrees, the horizontal target area image is rotated to an upright state, and the rotated horizontal target area image is input to the content recognition model to obtain the character string content of the recognition target character string. You can also do this.

In the above embodiment, the terminal may determine the upright state of the character string by obtaining the upright angle of the horizontal target area image using the angle determination model, while the upright state of the character string is the inverted state. In some cases, the terminal may transform the horizontal target area image into an upright state by rotation, and input the upright horizontal target area image into the content recognition model to obtain the string content. , which is advantageous to further improve the accuracy of the obtained string content.

In one embodiment, as shown in FIG. 3, step S102 includes steps S301 to S303.

In step S301, the terminal extracts character area image features from the image using the position detection model.

Here, the character area image feature refers to an image feature for determining a character string position. Specifically, the terminal may extract the character region image features from the obtained image of the recognition target character string using the position detection model.

In step S302, the terminal obtains a prediction mask of the target area image according to the character area image characteristics.

Here, a mask is a selected image, figure, or object, and refers to controlling the processing area or processing process of an image by blocking the image to be processed (globally or locally). Specifically, the terminal may use the character area image features to obtain a prediction mask corresponding to the character area image features.

In step S303, the terminal determines a connected domain and a minimum circumscribed rectangle for the prediction mask, and obtains a target area image.

In step S302, the terminal may obtain a prediction mask of the target area image, and then obtain a communicating domain and a minimum circumscribed rectangle for the mask to obtain the target image.

Furthermore, in order to avoid the problem that the accuracy rate of character string recognition is too low due to lack of sharpness that may exist in the image with the recognition target character string obtained by the terminal and the light irradiation intensity is too low, In one embodiment, step S301 further includes the steps of preprocessing the image and extracting high-dimensional image features from the pre-processed image; The method may also include a step of performing first feature enhancement processing to make the character area image feature.

Here, in the preprocessing process, the terminal filters out small or hard-to-see character string region images in the image with the recognition target character string, thereby reducing the high-dimensionality of the image with the recognition target character string. The terminal may be able to extract image features, and the terminal may perform a first feature enhancement process on the extracted high-dimensional image features using an image feature pyramid, which may be performed on character area images. This is advantageous in improving the ability to express features, and it is possible to generate accurate prediction masks for target area images even in environments where features are unclear.

In the above embodiment, the terminal extracts character area image features from the image, generates a corresponding prediction mask, and obtains an accurate target area image by determining a continuous domain and a minimum circumscribing rectangle for the prediction mask. In addition, in order to avoid problems such as missed recognition or misrecognition of character strings due to unclear features, the terminal performs the first feature enhancement process on the extracted image features using an image feature pyramid. By doing so, it is possible to improve the ability to express character region image features, thereby further improving the accuracy of character string recognition.

In one embodiment, as shown in FIG. 4, step S105 includes steps S401 to S403.

In step S401, the terminal performs global image feature extraction on the horizontal target area image using the content recognition model to obtain character string image features corresponding to the horizontal target area image.

Here, the content recognition model mainly recognizes the character content of the recognition target character string included in the horizontal target area image. Specifically, the terminal uses the content recognition model to perform global image feature extraction on the obtained horizontal target area image to obtain character string image features corresponding to the horizontal target area image. You can also do this.

In step S402, the terminal performs a second feature enhancement process on the character string image feature along the horizontal direction using a row vector convolution kernel.

Here, the second feature enhancement process refers to feature enhancement processing for character string image features. Specifically, in step S401, after obtaining character string image features, a second feature enhancement process is performed on the character string image features in the horizontal direction, that is, along the direction of the character string, using a row vector convolution kernel. You may go.

In step S403, the terminal performs parallel prediction of the recognition target character string based on the character string image feature obtained by the second feature enhancement process to obtain the character string content.

Furthermore, in order to further improve the efficiency of character string recognition, the terminal may perform character string content recognition on the character string image features obtained by the second feature enhancement process, and the recognition process may be performed using parallel prediction. It is possible to make predictions for multiple character strings, and therefore, efficient prediction for character string content is achieved.

In this embodiment, the terminal can accurately recognize the content of the character string using the content recognition model, and perform the second feature enhancement process on the character string image feature, thereby improving the ability to express the feature. This improves the accuracy of character string content recognition, and further improves the efficiency of character string content recognition by making predictions for all character strings using the parallel prediction method.

In one embodiment, as shown in FIG. 5, a character string recognition method based on computer vision is provided, and in this embodiment, a case where the method is applied to a terminal will be explained as an example. The method includes steps S501 to S510.

In step S501, the terminal acquires an image with a recognition target character string attached.

In step S502, the terminal preprocesses the image, extracts high-dimensional image features from the pre-processed image, and performs a first feature enhancement process on the high-dimensional image features using an image feature pyramid; Let the character area be an image feature.

In step S503, the terminal obtains a prediction mask of the target area image according to the character area image characteristics, determines a continuous domain and a minimum circumscribed rectangle for the prediction mask, and obtains a target area image.

In step S504, the terminal corrects the target area image in the horizontal direction to obtain a horizontal target area image.

In step S505, the terminal acquires the upright angle of the target area image in the lateral direction based on the angle determination model.

In step S506, if the standing angle section is the first angle section, the terminal determines the standing state of the character string as an upright state, and if the standing angle section is the second angle section, the terminal determines that the standing state of the character string is the upright state. The standing state of is determined as an inverted state.

In step S507, if the standing state of the character string is an upright state, the terminal inputs the horizontal target area image to a pre-built content recognition model, and if the standing state of the character string is an inverted state, The terminal rotates the horizontal target area image into an upright state and inputs it to the content recognition model.

In step S508, the terminal performs global image feature extraction on the horizontal target area image using the content recognition model to obtain character string image features corresponding to the horizontal target area image.

In step S509, the terminal performs second feature enhancement processing on the character string image feature along the horizontal direction using the row vector convolution kernel.

In step S510, the terminal performs parallel prediction of the recognition target character string based on the character string image features obtained by the second feature enhancement process to obtain character string content.

In the embodiment described above, by the terminal correcting the target area image in the horizontal direction, adaptive processing to changes in the image capturing angle in an industrial scene is achieved, and the recognition accuracy rate for character strings is improved. Further, the terminal may determine the standing state of the character string by obtaining the standing angle of the horizontal target area image using an angle determination model, and if the standing state of the character string is an inverted state, the terminal: The rotation transforms the lateral target area image into an upright state, which is advantageous for further improving the accuracy of the obtained string content. Further, the terminal may use the image feature pyramid to perform first feature enhancement processing on the extracted high-dimensional image features, and may perform second feature enhancement processing on the character string image features. , it is possible to improve the feature representation ability and further improve the accuracy of character string content recognition. Moreover, by predicting all character strings using the parallel prediction method, the efficiency of character string content recognition is further improved.

One application example is to effectively solve problems such as blurring of character recognition algorithms, omission of recognition due to light irradiation or angle changes, and misrecognition in the current industrial scene, and to increase the recognition accuracy rate. In this paper, we further provide an arbitrary angle character string recognition algorithm in industrial scenes. The present application may be placed in an industrial environment where the camera image forming environment is poor, and also ensures the efficiency and accuracy of the recognition algorithm, and supports multi-angle and even inverted character recognition. Here, the flow of algorithm training and prediction processing is shown in FIG. The flow is mainly divided into two processes: algorithm training and prediction. The training process requires training three different models for string position detection, string angle determination, and string content recognition, respectively. In the prediction process, the trained model is input to a test image and processes position detection, angle determination and recognition content in this order, and finally, the string, position and corresponding content are obtained.

Specifically, the flow of processing by each module is as follows.

(1) Training process 1.1 String position detection algorithm The training sample is the entire sample image containing the string, and the corresponding annotation is the coordinate information of the string position, such as the upper left corner of the starting point of the string and This is the location box for the string in the image that contains the bottom right corner information of the ending point. Since there are differences in scale and color distribution between different training samples, it is necessary to perform normalization processing on the samples and filter out small or difficult to see character string position boxes. The data that has undergone image preprocessing is input to the character string position detection algorithm section, and this section performs feature enhancement in conjunction with the image feature pyramid structure via a deep neural network. As shown in Figure 7, conv represents a convolutional layer, stride represents a step size, upsamples the extracted features of each scale, and adds the features previously obtained through the network. The final image features are obtained. In this case, the features carry semantic information in addition to spatial information. The image features obtained by the position detection algorithm are used to predict the mask for the final image string region. A character string position box is obtained by determining a communicating domain and a minimum circumscribed rectangle for the mask.
1.2 Character String Angle Determination Algorithm As shown in FIG. 8, when the angle of a character string is greater than 0 degrees and less than 180 degrees, a character string image corrected in the horizontal direction by affine transformation is obtained. After being corrected in the horizontal direction, depending on the initial shooting angle, it is difficult to ensure that the corrected character string is erect or inverted. Therefore, the angle used to determine whether the corrected character string is inverted is A determination algorithm has been added, which rotates the character string 180 degrees about the center if it is inverted, and outputs it directly without processing if it is upright. In this way, the finally obtained character string image is ensured to be erect, and is used as the output of the next character string content.
1.3 String content recognition algorithm As shown in Figure 9, to recognize string image content, a deep neural network is used to learn about string features, and in order to obtain the features of the entire string, , For the extracted image features, feature emphasis is performed along the character string direction using row vectors as convolution kernels, thereby efficiently predicting character string contents in parallel.

(2) Prediction process A test image is input, and first, the character string position of the test image is detected using a character string position detection algorithm. Next, trimming and affine transformation are performed on the detected image area, and the converted trimming area is supplied to the character string angle determination algorithm. If the cropped area image is determined to be inverted, it is rotated 180 degrees about the center. , if it is determined that the image is erect, no processing is performed. The image area processed by the character string position detection algorithm and the character string angle determination algorithm is input to the character string content recognition network, and finally, the position of the character string in the image and the corresponding text content are obtained by the content recognition network. .

In the application example above, a total of three stages of algorithms: a cascade character string position detection algorithm, a character string angle determination algorithm, and a character string content recognition algorithm are used to change the sharpness of the formed image, change the angle, and change the light irradiation. We obtained an algorithm that can stably and efficiently recognize character strings even in general industrial scenes that change, and laid the foundation for the application of character string recognition in industrial scenes.

Note that although the steps in the flowchart of the present application are shown in the order indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated, there is no strict restriction on the order of performance of these steps, and these steps may be performed in other orders. Furthermore, at least some of the steps in the diagram may include multiple steps or multiple stages, and these steps or stages are not necessarily completed at the same timing, and may be performed at different timings, and these The steps or stages are not necessarily performed sequentially, but may be performed sequentially or alternately with other steps or at least some of the steps or stages in other steps.

In one embodiment, as illustrated in FIG. provide a recognition device;
The image acquisition module 1001 is used to acquire an image with a recognition target character string attached.
The position detection module 1002 is used to obtain a target area image in which a recognition target character string is located in the image based on a position detection model constructed in advance.
The horizontal correction module 1003 is used to horizontally correct the target area image to obtain a horizontal target area image.
The angle determination module 1004 is used to obtain the standing state of a character string in a horizontal target area image based on a pre-built angle determination model.
When the character string is in an erect state, the content recognition module 1005 inputs the horizontal target area image into a pre-built content recognition model and obtains the character string content corresponding to the recognition target character string. used for

In one embodiment, the angle determination module 1004 further obtains the upright angle of the horizontal target area image based on the angle determination model, and determines the upright state of the character string from the upright angle section to which the upright angle belongs. used for

In one embodiment, the standing angle section includes a first angular section and a second angular section, the standing states of the character string include an upright state and an inverted state, and the angle determination module 1004 further determines that the standing angle section includes a first angular section and a second angular section; If it is one angle section, it is used to determine the standing state of the character string as an upright state, and if the standing angle section is the second angle section, it is used to determine the standing state of the character string as an inverted state.

In one embodiment, the content recognition module 1005 further rotates the horizontal target area image to an upright state if the character string is in an inverted state.

In one embodiment, the position detection module 1002 further utilizes the position detection model to extract character region image features from the image with the character string to be recognized , and uses a predictive mask of the target region image according to the character region image features. The connected domain and minimum circumscribing rectangle are obtained for the prediction mask and used to obtain the target area image.

In one embodiment, the position detection module 1002 further preprocesses the image with the character string to be recognized , extracts high-dimensional image features from the preprocessed image, and utilizes an image feature pyramid to perform high-dimensional It is used to perform a first feature enhancement process on the image feature and make it a character area image feature.

In one embodiment, the content recognition module 1005 further utilizes the content recognition model to perform global image feature extraction on the horizontal target region image, and performs global image feature extraction on the horizontal target region image to identify character string image features corresponding to the horizontal target region image. Then, a second feature enhancement process is performed on the character string image features along the horizontal direction using a row vector convolution kernel, and based on the character string image features obtained by the second feature enhancement process, the recognition target is It is used to predict strings in parallel to obtain string content.

Specific limitations on the computer vision-based string recognition device may refer to the limitations on the computer vision-based string recognition method above, so they will not be described in detail here. All or part of each module in the character string recognition device based on computer vision may be implemented by software or a combination of hardware and software. Each of the above modules may be incorporated in the processor of the computer device in the form of hardware, or may be independent from the processor of the computer device, or may be stored in the memory of the computer device in the form of software and called up by the processor. Operations corresponding to each of the above modules may be executed.

In one embodiment, a computer device is provided, the computer device may be a terminal, and the internal structure diagram thereof may be as shown in FIG. 11. The computing equipment includes a processor, memory, communication interface, display screen, and input devices connected through a system bus. The processor of the computer equipment provides computing and control capabilities. The memory of the computer equipment includes non-volatile storage media, internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operating system and computer programs to execute on non-volatile storage media. The communication interface of the computer device is used to perform wired or wireless communication with an external terminal, and the wireless method may be realized by WIFI, an operator's network, NFC (near field communication), or other technology. . The computer program, when executed by a processor, implements a computer vision-based character string recognition method. The display screen of the computer equipment may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment may be a touch layer covered on the display screen or provided on the case of the computer equipment. It may be a button, a trackball, or a touch panel, or it may be an external keyboard, touch panel, mouse, or the like.

As is obvious to those skilled in the art, the structure shown in FIG. 11 is only a block diagram of the structure of a part related to the solution of the present application, and does not limit the computer equipment to which the solution of the present application is applied. In particular, the computing equipment may include fewer or more components, may have combinations of components, or have different arrangements of components than are shown in the figures.

In one embodiment, computer equipment is further provided that includes a memory in which a computer program is stored and a processor that, when executed, implements the steps in each of the method embodiments described above.

One embodiment provides a computer readable storage medium having stored thereon a computer program that, when executed by a processor, implements the steps of each of the method embodiments described above.

As will be apparent to those skilled in the art, all or part of the steps of the example method described above may be implemented by a computer program instructing relevant hardware, the computer program being a non-volatile computer readable device. The computer program may be stored on a storage medium and, when executed, may include the flow of each of the method embodiments described above. All references to memory, storage, database, or other media used in embodiments of this application may include at least one of non-volatile memory and volatile memory. Nonvolatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, and the like. Volatile memory may include random access memory (RAM) or external cache memory. By way of non-limiting illustration, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

The technical features of the above embodiments may be combined arbitrarily, and for convenience of explanation, all possible combinations of the technical features of the above embodiments are not described. should be considered within the scope of this specification unless there is a contradiction.

The examples described above are only some embodiments of the present application, and although the description thereof is specific and detailed, they should not be understood as limiting the patent scope of the present invention. Note that those skilled in the art can make several modifications and improvements without departing from the spirit of the present application, and all of these are included within the patent scope of the present application. Therefore, the patent scope of the present application should conform to the appended claims.

Claims (10)

A character string recognition method based on computer vision,
obtaining an image with a character string to be recognized;
a step of acquiring a target area image in which the recognition target character string is located among the images to which the recognition target character string is attached, based on a position detection model constructed in advance;
horizontally correcting the target area image to obtain a horizontal target area image;
acquiring the standing state of the character string in the horizontal target area image based on a pre-built angle determination model;
If the standing state of the character string is an upright state, the step of inputting the horizontal target area image into a content recognition model built in advance to obtain character string content corresponding to the character string to be recognized. hand,
performing global image feature extraction on the horizontal target area image using the content recognition model to obtain character string image features corresponding to the horizontal target area image;
performing a second feature enhancement process on the character string image feature along the horizontal direction using a row vector convolution kernel;
A character string recognition method, comprising the step of performing parallel prediction on the recognition target character string based on the character string image feature obtained by the second feature enhancement process to obtain the character string content . The step of acquiring the standing state of the character string of the horizontal target area image based on the pre-built angle determination model,
obtaining an upright angle of the horizontal target area image based on the angle determination model;
2. The method according to claim 1, further comprising the step of determining the standing state of the character string from the standing angle section to which the standing angle belongs. The standing angle section includes a first angle section and a second angle section, and the standing state of the character string includes an upright state and an inverted state,
The step of determining the standing state of the character string from the standing angle section to which the standing angle belongs,
when the standing angle section is the first angle section, determining the standing state of the character string as the upright state;
3. The method according to claim 2, further comprising determining the standing state of the character string as the inverted state when the standing angle section is the second angle section. 4. The method of claim 3, further comprising, when the character string is in the inverted state, rotating the horizontal target area image to the upright state. The step of acquiring a target area image in which the recognition target character string is located among the images to which the recognition target character string is attached based on a position detection model constructed in advance,
extracting character area image features from the image with the recognition target character string using the position detection model;
obtaining a prediction mask of the target region image according to the character region image characteristics;
2. The method of claim 1, further comprising determining a minimum circumscribed rectangle for the prediction mask to obtain the target region image. The step of extracting character area image features from the image with the recognition target character string includes:
preprocessing the image with the recognition target character string and extracting high-dimensional image features from the preprocessed image;
6. The method according to claim 5, further comprising the step of performing a first feature enhancement process on the high-dimensional image feature using an image feature pyramid to make it the character area image feature. The pretreatment is
filtering a small or hard-to-see character string region image of the image with the recognition target character string, and extracting the high-dimensional image feature in the image with the recognition target character string; 7. The method of claim 6, characterized in that: A character string recognition device based on computer vision,
an image acquisition module that acquires an image with a recognition target character string;
a position detection module that acquires a target area image in which the recognition target character string is located among the images with the recognition target character string, based on a position detection model built in advance;
a lateral correction module that laterally corrects the target area image to obtain a lateral target area image;
an angle determination module that acquires a standing state of a character string in the horizontal target area image based on a pre-built angle determination model;
When the character string is in an upright state, a content recognition module inputs the horizontal target area image into a pre-built content recognition model and obtains character string content corresponding to the recognition target character string. And,
Performing global image feature extraction on the horizontal target area image using the content recognition model to obtain character string image features corresponding to the horizontal target area image;
performing a second feature enhancement process on the character string image feature along the horizontal direction using a row vector convolution kernel;
and a content recognition module that performs parallel prediction of the recognition target character string to obtain the character string content based on the character string image feature obtained by the second feature enhancement process. String recognizer. memory in which computer programs are stored;
Computer equipment, characterized in that it comprises a processor, which when executing the computer program implements the steps of the method according to any one of claims 1 to 7 . A computer-readable storage medium, characterized in that a computer program is stored thereon, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7 .

Images