WO2023001112A1

WO2023001112A1 - Text beautification method and apparatus, and readable storage medium and electronic device

Info

Publication number: WO2023001112A1
Application number: PCT/CN2022/106276
Authority: WO
Inventors: 胡妍
Original assignee: 维沃移动通信有限公司
Priority date: 2021-07-19
Filing date: 2022-07-18
Publication date: 2023-01-26
Also published as: CN113689525B; CN113689525A

Abstract

The present application belongs to the technical field of communications. Disclosed are a text beautification method and apparatus, and a readable storage medium and an electronic device. The method comprises: performing text recognition on a first image, so as to determine M image regions in the first image, wherein each image region comprises one character; according to the M image regions, performing image processing on the first image, so as to obtain M first single-character images, wherein each first single-character image corresponds to one image region; performing font beautification processing on the M first single-character images, so as to obtain M second single-character images, wherein the font beautification processing comprises at least one of the following: font frame structure processing and font style processing; and synthesizing the M second single-character images with an original background image of the first image, so as to obtain a second image.

Description

Text beautification method, device, readable storage medium and electronic equipment

Cross References to Related Applications

This application claims priority to Chinese Patent Application No. 202110815639.5 filed in China on July 19, 2021, the entire contents of which are hereby incorporated by reference.

technical field

The present application belongs to the technical field of communication, and in particular relates to a text beautification method, device, readable storage medium and electronic equipment.

Background technique

With the development of electronic devices, the functions of electronic devices are becoming more and more abundant. For example, electronic devices can process text in images (specifically, text in image format, hereinafter referred to as text images) through word processing tools. Specifically, the electronic device can adjust the size of the text image and the position of the text image in the image through a word processing tool.

However, since the electronic device processes the text in the image by adjusting the size and/or position of the text image, the effect of processing the text image in the image is poor.

Contents of the invention

The purpose of the embodiments of the present application is to provide a text beautification method, device, readable storage medium and electronic equipment, which can solve the problem of poor processing effect on text images in images.

In the first aspect, the embodiment of the present application provides a text beautification method, the method includes: performing text recognition on the first image, determining M image areas, each of which includes a character; according to the M image areas, Carry out image processing to the first image, obtain M first single-character images, each first single-character image corresponds to an image area; perform font beautification processing on the M first single-character images, obtain M second single-character images, font beautification The processing includes at least one of the following: font frame structure processing, font style processing; combining the M second word images with the original background image of the first image to obtain a second image.

In the second aspect, the embodiment of the present application provides a text beautification device, which may include: a recognition module, a processing module, and a synthesis module; a recognition module, used to perform text recognition on the first image; a processing module, used to Determine M image regions in the first image according to the recognition result of the recognition module, and each image region includes a character; the processing module is also used to perform image processing on the first image according to the M image regions to obtain M The first single-character image, each first single-character image corresponds to an image area; the processing module is also used to perform font beautification processing on the M first single-character images to obtain M second single-character images, and the font beautification processing includes at least one of the following Item: font frame structure processing, font style processing; synthesis module, used to synthesize the M second word images processed by the processing module with the original background image of the first image to obtain the second image.

In the third aspect, the embodiment of the present application provides an electronic device, the electronic device includes a processor, a memory, and a program or instruction stored in the memory and operable on the processor, and the program or instruction is executed by the processor to realize As the steps of the method of the first aspect.

In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or an instruction is stored, and when the program or instruction is executed by a processor, the steps of the method in the first aspect are implemented.

In the fifth aspect, the embodiment of the present application provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the method in the first aspect.

In the embodiment of the present application, text recognition can be performed on the first image, and M image regions are determined, and each image region includes a character; according to the M image regions, image processing is performed on the first image to obtain M first A single character image, each first single character image corresponds to an image area; font beautification processing is performed on the M first single character images to obtain M second single character images, and the font beautification processing includes at least one of the following: frame structure processing between fonts, Font style processing: combining the M second word images with the original background image of the first image to obtain a second image. Through this scheme, since the font beautification process (specifically at least one of font frame structure processing and font style processing) can be carried out to the single-character image in an image (such as the first image), it can ensure that the handwriting of the adjusted single-character image It is more aesthetically pleasing, thus improving the effect of beautifying the text images in the image.

Description of drawings

FIG. 1 is a schematic diagram of a text beautification method provided in an embodiment of the present application;

Fig. 2 is one of the schematic flow charts of processing single-character images using the text beautification method provided by the embodiment of the present application;

FIG. 3 is the second schematic flow diagram of processing single-character images using the text beautification method provided by the embodiment of the present application;

FIG. 4 is a schematic diagram of obtaining an original background image by processing a reference image in an embodiment of the present application;

Fig. 5 is a schematic diagram of determining a fitted straight line in units of text lines;

FIG. 6 is a schematic diagram of the effect of processing single-character images using the text beautification method provided in the embodiment of the present application;

FIG. 7 is the third schematic flow diagram of processing single-character images using the text beautification method provided by the embodiment of the present application;

FIG. 8 is a schematic diagram of a text beautification device provided in an embodiment of the present application;

FIG. 9 is a schematic diagram of an electronic device provided by an embodiment of the present application;

FIG. 10 is a schematic diagram of hardware of an electronic device provided by an embodiment of the present application.

detailed description

The following will clearly describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, but not all of them. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in this application belong to the protection scope of this application.

The terms "first", "second" and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific sequence or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances such that the embodiments of the application can be practiced in sequences other than those illustrated or described herein. In addition, "and/or" in the description and claims means at least one of the connected objects, and the character "/" generally means that the related objects are an "or" relationship.

Firstly, some nouns or terms involved in the claims and description of the present application will be explained below.

The text beautification method, device and electronic equipment provided by the embodiments of the present application will be described in detail below through specific embodiments and application scenarios with reference to the accompanying drawings.

The text beautification method provided in the embodiment of the present application can be applied to the scene of beautifying handwritten text.

With the development of computer vision technology, documents in picture format are widely used due to their characteristics of portability, copying, transmission, management, and viewing. At the same time, a large number of image processing tools have been launched one after another, providing users with many convenient and effective picture editing Tools, such as word processing tools that can process text in image format. Specifically, words can be added to the image or erased by word processing tools. In this way, because the current word processing tools cannot perform handwriting beautification on the words in the image, the processing effect on the words in the image is poor.

In the text beautification method provided in the embodiment of the present application, text recognition can be performed on the first image, M image areas are determined, and each image area includes a character; image processing is performed on the first image according to the M image areas , obtain M first word images, and each first word image corresponds to an image area; carry out font beautification processing to the M first word images, obtain M second word images, and font beautification processing includes at least one of the following: Font frame structure processing and font style processing; combining the M second word images with the original background image of the first image to obtain a second image. Through this scheme, since the font beautification process (specifically at least one of font frame structure processing and font style processing) can be carried out to the single-character image in an image (such as the first image), it can ensure that the handwriting of the adjusted single-character image It is more aesthetically pleasing, thus improving the effect of beautifying the text images in the image.

The embodiment of the present application provides a text beautification method, and FIG. 1 shows a flow chart of the font beautification method provided in the embodiment of the present application. As shown in FIG. 1 , the text beautification method provided by the embodiment of the present application may include the following steps 101 to 104 . The following takes an electronic device executing the method as an example for illustration.

Step 101, the electronic device performs text recognition on the first image, and determines M image areas in the first image.

Wherein, each of the above M image areas includes a character. The characters can be Chinese characters, letters or symbols.

Optionally, in this embodiment of the application, the characters may be handwritten characters or printed characters, which may be determined according to actual usage requirements, and are not limited in this embodiment of the application.

Optionally, in this embodiment of the present application, the electronic device may perform text recognition on the first image based on a text recognition technology, so as to determine the M image regions in the first image.

Optionally, in the embodiment of the present application, after the electronic device imports the first image (such as the image shown in (a) in FIG. 2 ), the electronic device may first perform text detection on the first image to determine the first image The positions (or position information, such as coordinate information) of the M image regions in .

Exemplarily, as shown in (b) in FIG. 2, the electronic device may first use the text line detection technology to detect character lines in the M image regions in the first image (for example, the character lines shown in (b) in FIG. 2 Row 20); then adopt the single word detection technology to detect the area of each character in each line area (the area 21 where the word "wine" is located as shown in (c) in Figure 2).

Step 102: The electronic device performs image processing on the first image according to the M image regions to obtain M first single-character images.

Wherein, each first word image in the M first word images corresponds to one image area in the M image areas. That is, there is a one-to-one correspondence between the M image regions and the M first word images.

In the embodiment of the present application, the electronic device may crop the first image according to M image regions, so as to obtain M first word images.

Optionally, in this embodiment of the present application, before cropping the first image, the electronic device may copy a first image as a reference image.

Step 103, the electronic device performs font beautification processing on the M first word images to obtain M second word images.

Wherein, the aforementioned font beautification may include at least one of the following: (1), font frame structure processing, (2), font style processing.

Optionally, in the embodiment of the present application, the foregoing step 103 may specifically be implemented through the following step 103a.

Step 103a. Based on the font beautification parameters, the electronic device performs font beautification processing on each of the M first individual character images to obtain a second individual character image corresponding to each first individual character image.

Wherein, the above-mentioned font beautification parameters are parameters obtained by training a neural network model using a large number of word image pairs, and each word image pair includes a word image to be beautified and a word image after beautification.

It can be understood that, in the embodiment of the present application, there are differences in font style and/or stroke structure between the above-mentioned single-character image to be beautified and the above-mentioned one beautified single-character image.

For example, taking the difference between the stroke structure of the word image to be beautified and the beautified word image as an example, the stroke structure of the beautified word image is more compact and coordinated than the stroke structure of the word image to be beautified.

Optionally, in the embodiment of the present application, the above font beautification parameters may be parameters of a font beautification model. The font beautification model is obtained by training the convolutional neural network model by using the above-mentioned large number of single-character image pairs; the font beautification model can be used to beautify at least one of the stroke structure and font style of the single-character image.

In order to explain the font beautification method provided by the embodiment of the present application more clearly, the method for training the neural network model by using the above-mentioned large number of image pairs will be exemplarily described below.

1) Prepare the training set:

A large number of word image pairs are collected as a training data set. Wherein, each single-character image pair includes a single-character image to be beautified, and a single-character image after adjusting the stroke structure and/or font style of the single-character image (ie, the beautified single-character image). And the beautified word image in each word image pair is used as the label of the word image pair.

2) Font beautification model training:

i), each layer parameter (weight value and bias) of neural network model is carried out random initialization; And to the individual word image pair in training set, each batch comprises N individual word image pair, and N is a positive integer.

ii), the neural network model after initializing the input of the first batch of single character images in the training set.

After inputting the first batch of single-character image pairs into the first layer of the neural network model, the N single-character images to be beautified in the first batch of single-character image pairs are respectively weighted with the weights of the first layer, and the bias term is added And use the ReLu function to activate to obtain N first result images; use the N first result images as the input of the second layer of the neural network model, and compare the N first result images with the weights of the second layer respectively Weighting operation, and adding bias items and using the ReLu function to activate to obtain N second result images; then use N second result images as the input of the third layer of the neural network model, and repeat this until the obtained neural network model The N jth result images processed by the last layer of , j (indicating that the neural network model includes j layers) is a positive integer.

iii), the Euclidean distance between the N jth result image and the corresponding label image (the corresponding beautified word image), to obtain N Euclidean distances, and then use the average value of the N Euclidean distances as an error to calculate the error The gradient of each network parameter in the neural network model; then the stochastic gradient descent algorithm is used to update the network parameters of the neural network model. In this way, a training of the neural network model is completed.

Then the second batch of single-word image pairs are input into the neural network model to train the neural network model for the second time. Repeat this until the error drops to an appropriate range and tends to be stable, then all the training of the neural network model can be completed. In the embodiment of the present application, the neural network model that has completed the above training is the font beautification model in the present application. The font beautification parameters mentioned above are the parameters of each layer in the trained neural network model.

Optionally, in the embodiment of the present application, the above neural network model may be a convolutional neural network model, or any other neural network model with learning capability, which may be determined according to actual usage requirements, and is not limited in the embodiment of the present application.

Optionally, in the embodiment of the present application, the word images to be beautified in the above training set may be manually handwritten, or the word images to be beautified in the training set may be randomly perturbed by an electronic device to a beautified word image (that is, " uglify") get.

It should be noted that, in actual implementation, the electronic device performs font beautification processing on the i-th first word image among the M first word images based on the font beautification parameters, and obtains the i-th word image corresponding to the i-th first word image. second word image, and then, the electronic device can perform font beautification processing on the i+1 first word image in the M first word images based on the font beautification parameter, and obtain the first word image corresponding to the i+1 first word image The corresponding i+1th second word image, and so on, after the electronic device performs font beautification processing on the M first word image in the M first word images based on the font beautification parameters, it can be obtained. The Mth second word image corresponding to the first word image. That is, after the electronic device executes the above step 103a M times, M second single-character images can be obtained.

In the embodiment of the present application, the electronic device performs font beautification processing on each of the M first word images based on the font beautification parameters, and obtains a second word image corresponding to each first word image, specifically It may be: each first individual character image is sequentially input into the font beautification model for processing, and a second individual character image corresponding to each first individual character image is obtained.

Exemplarily, taking the example of beautifying one of the M first character images through the font beautification model, the electronic device may input a first character image (for example, the character image 30 in FIG. 3 ) into the font beautification model, and The first word image and the weights and bias values (i.e. font beautification parameters) of each layer of the font beautification model (such as the font beautification model 22 in Fig. 3) are calculated, and the activation calculation is performed, and then the font beautification model can output the same The font beautification image corresponding to the first word image is the second word image corresponding to the first word image (such as the word image 31 in FIG. 3 ). It can be understood that in the embodiment of the present application, inputting a single-character image into the font beautification model for processing is essentially processing the single-character image by using the network parameters (ie, font beautification parameters) of each layer of the font beautification model.

In the embodiment of the present application, since the font beautification process can be performed on the first single-character image based on the font beautification parameters obtained by training the neural network model through a large number of single-character image pairs, the author's original writing can be preserved while the handwriting beautification is realized. style and personality.

Further, since the stroke structure and/or font style of a single character image (i.e. the first single-character image) can be adjusted, it can be ensured that the handwriting of the adjusted single-character image is more aesthetically pleasing, thus improving the understanding of the character image in the image. processing effect.

Step 104, the electronic device synthesizes the M second word images and the original background image of the first image to obtain a second image.

Optionally, in the embodiment of the present application, as shown in FIG. 4 , the electronic device can erase the characters in the above-mentioned reference image 40 through text erasing technology to obtain the original background as shown in FIG. 4 Image 41. Alternatively, the electronic device may use images other than the M image areas in the first image as the original background image, which may be determined according to actual usage requirements, and is not limited in this embodiment of the present application.

In the embodiment of the present application, the combined positions of the M second word images in the background image are determined according to the positions of the M image regions in the first image.

Optionally, in this embodiment of the present application, the foregoing step 104 may be specifically implemented through the following steps 104a and 104b.

Step 104a, the electronic device determines the second position information according to the first position information of the M image areas in the first image, and the second position information is used to indicate the composition position of the M second word images in the original background image.

In step 104b, the electronic device synthesizes the M second word images and the background image according to the second location information to obtain a third image.

Wherein, the similarity between the first arrangement and the second arrangement is greater than or equal to the preset threshold, the first arrangement is the arrangement of M second single-character images in the third image, and the second arrangement is the arrangement of the M image regions in the first image.

In the embodiment of the present application, the first position information is used to indicate the positions of the M image regions in the first image.

In the embodiment of the present application, it is assumed that the first image and the original background image are in the same coordinate system, and the position information (such as coordinate information) of the first image and the original background image in the coordinate system is the same, for example, assuming that the first image The coordinate information is: (x1, y1), (x2, y2), (x3, y3), (x4, y4), and the coordinate information of the original background image is also: (x1, y1), (x2, y2), ( x3, y3), (x4, y4); then:

In one manner, the electronic device may use the first position information as the second position information. In this case, the combined positions of the M second word images in the original background image and the positions of the M image areas in the first image area The positions are the same, that is, the positions of the characters in the image remain unchanged before and after the beautification process.

In another way, in order to achieve the effect of neat typesetting, the electronic device may divide the M image regions into at least one unit according to the distribution of the M image regions in the first image during synthesis, and each unit It is a character row or a character column, and the position information (ie, the first position information) of the M image areas is fine-tuned in units of units, and the fine-tuned position information is used as the second position information, so that it can be compared with a single The synthesis positions of the second word images corresponding to the image area groups in the original background image are arranged on a straight line as much as possible.

The method for fine-tuning the position information of M image regions by an electronic device is exemplarily described below by taking a unit as an example of a character line.

i) The electronic device calculates the location information of the central points of the M image areas in the first image (the location information 50 shown in FIG. 5 ), and obtains M pieces of location information (ie, the first location information).

ii) The electronic device divides the M image areas into at least one character line according to the distribution of the M image areas in the first image, and fits the position information of the image area in each character line into A straight line (the straight line 51 shown in FIG. 5). It should be noted that, in the embodiment of the present application, the number of straight lines determined by the electronic device is the same as the number of character lines.

iii) For each character row, it is assumed that a character row (hereinafter referred to as character row o) includes u (u is an integer greater than 1) image areas (that is, u characters), and according to the position information of the u characters The determined straight line is the first straight line, then: the electronic device can use the projection position of the center point of the leftmost image area of the character row o on the first straight line as the starting point of the composition; it will be located at the rightmost of the character row o The projection position of the center point of the image area on the side on the first straight line is used as the synthesis end point; and according to the distance L between the synthesis start point and the synthesis end point, the width of u second word images corresponding to the u image areas Values w1, w2, w3...wu, calculate the gap d between two adjacent second word images, specifically, the gap d can be calculated by the following formula 1, formula 1: d=[L-(w1+w2 +w3…+wu-w1*0.5-wu*0.5)]÷(u-1). It can be understood that the position information of the composition position of the u second word images in the original background image includes: the position information of the composition start point, the position information of the composition end point, and the position information between the composition start point and the composition end point with an interval of d Location information of u - 2 locations. It should be noted that, in the embodiment of the present application, the electronic device may obtain the second position information after performing the above step iii on the image area in each character row.

For the description of determining the second location information in units of character columns, please refer to the related description of determining the second location information in units of character rows. To avoid repetition, details are not repeated here.

It can be understood that in the embodiment of the present application, in the above i~iii, the row coordinate (ie, the X-axis coordinate) and the column coordinate (ie, the Y-axis coordinate) of the position information of the image area in each character row are fine-tuned as follows: An example is used for illustrative purposes. In actual implementation, at least one of the row coordinates and the column coordinates of the position information of the image area may be adjusted.

Illustratively, taking a unit as a character line as an example, assuming that the first image completely overlaps with the original background image, then: the electronic device may use the original position of the leftmost image area in a character line in the first image as the compositing starting point, Paste the second single-character image corresponding to the image area in the character row to the second image in order from left to right, and these single-character images maintain the same gap in the horizontal direction (ie, the average pasting gap d). i.e. only the row coordinates are adjusted.

Also as an example, in conjunction with FIG. 5, as shown in FIG. 6, taking the character line unit as an example, if the distance between the position information of one of the M image areas and the fitted straight line is less than or equal to the first threshold, for example ("small", "garden", "fragrance", "path", "independence", "lingering", "lingering") among Fig. 5, then electronic device will be with the column of the second word image corresponding to this image area The coordinates (Y-axis coordinates) move vertically to the fitting line; if the distance between the position information of an image area in the first image and the fitting line is greater than the first threshold, such as "." in Figure 5; then the electronic device keeps The column coordinates of the second word image corresponding to the image area are the same as the column coordinates of the image area. As shown in Figure 6, Figure 6 is a schematic diagram of the second image, and the column coordinates of the second individual character images "small", "garden", "fragrant", "path", "du", "linger" and "lingering" are Adjust vertically to the fitting straight line, while the row coordinates of the second word image "." are not moved.

The embodiment of the present application provides users with a method for beautifying the handwriting (font frame structure and font style) of the characters in the image. Using the text beautification method provided in the embodiment of the present application can not only beautify the handwriting in the image , and can standardize the typesetting effect of the text in the image, which can improve the look and feel of the (handwritten) text in the image.

In the embodiment of the present application, since the electronic device can fine-tune the typesetting of the M second single-character images, it can ensure that the typesetting of the M second single-character images in the second image is more neat and beautiful.

In the font beautification method provided in the embodiment of the present application, because the font beautification processing (specifically at least one of font frame structure processing and font style processing) can be performed on a single-character image in an image (for example, the first image), it can ensure that The handwriting of the adjusted single-character image is more aesthetically pleasing, thus improving the effect of beautifying the character image in the image.

Optionally, in the embodiment of the present application, after obtaining the M second single-character images, the electronic device may adjust the size of the M second single-character images to reduce the size difference between characters, so that the M The typesetting of the second word image is visually more neat.

Optionally, in the embodiment of the present application, before the above step 104, the font beautification method provided in the embodiment of the present application further includes the following steps 105 and 106.

Step 105, the electronic device divides the M second single-character images into at least one image group according to the distribution of the M image areas in the first image.

In the embodiment of the present application, each image group in the at least one image group includes at least one second single-character image.

In the embodiment of the present application, since the M image areas in the first image, the M first word images, and the M second word images correspond one-to-one, it can be determined according to the area where the M image areas are located in the first image , grouping the M second single-character images to obtain H groups, where H is a positive integer. Specifically, assuming that Q word images in M image areas are in area 1 in the first image, K images in M image areas are in area 2 in the first image, and area 1 is different from area 2, Q The sum of K is equal to M, and Q and K are all positive integers; then: the Q second word images corresponding to these Q word images are one group, and the K second word images corresponding to these K word images are another group. A group.

It can be understood that in actual implementation, grouping can also be performed according to the size of the second individual character images, for example, if the size of 2 second individual character images in the M second individual character images is greater than 2 square centimeters, among the M second individual character images The size of the three second single-character images is less than or equal to 2 square centimeters, then: the two second single-character images form one group, and the three second single-character images form another group.

Step 106 , the electronic device performs size normalization processing on the second single-character images in at least one image group by taking the image group as a unit, and obtains M second single-character images after normalization processing.

Optionally, in the embodiment of the present application, the foregoing step 106 may specifically be implemented through the following steps 106a and 106b, or may be implemented through the following steps 106a and 106c.

Step 106a, for each of the M second single-character images, determine a first difference between the size of a second single-character image and the target size.

Wherein, the target size may be the median of the sizes of all the second single-character images in the first image group, and the first image group is the image group to which the above-mentioned second single-character image belongs.

In the embodiment of the present application, the electronic device can calculate the median of the width values of the second word images in the first image group (hereinafter referred to as the width median), and calculate the height of all the second word images in a group The median of the values (hereafter referred to as the median height), and use the median width and median height as the target size.

In the embodiment of the present application, for each second single-character image in the M second single-character images, the electronic device may first determine the first difference between the size of the second single-character image and the target size; if the first difference is less than or is equal to the preset difference, then the electronic device may execute step 106b below, and if the first difference is greater than the preset difference, then the electronic device may execute step 106c below. It should be noted that, in the embodiment of the present application, after the electronic device executes the process M times, the normalized M second single-character images can be obtained.

Step 106b, the electronic device adjusts the size of a second word image to the first size.

Wherein, the difference between the first size and the target size is smaller than the first difference, for example, the second difference may be 0.

Step 106c, the electronic device keeps the size of a second single character image unchanged.

Optionally, in this embodiment of the application, it is assumed that the first size: width*height=a1*b1, and the second size: width*height=a2*b2, then a2=(1±k)*a1, b2=(1 ±w)*b1, where a1, a2, b1, b2 are numbers greater than 0, k is greater than 0 and less than the median width, w is greater than 0 and less than or equal to the median height. For example, the median width can be 0.2, and the median height can be 0.2. Of course, the first preset value and the second preset value can also be different.

It can be understood that, in the embodiment of the present application, for each individual character image in the M second individual character images, assuming that the size of a second individual character image is the first size, then: if the difference between the first size and the target size (i.e. The above-mentioned first difference) is less than or equal to the target preset value, then the electronic device can adjust the size of the second single-character image from the first size to the second size, so that the size of the second single-character image is closer to the target size move closer. If the difference between the first size and the target size (i.e. the first difference) is greater than the target preset value, the electronic device keeps the size of the second single-character image unchanged to keep the size characteristics of the second single-character image .

In the embodiment of the present application, since the size of the M second single-character images can be moved closer to the target size, the size difference between different second single-character images can be reduced, so that the typesetting of the M second single-character images can be visually more tidy.

Optionally, in this embodiment of the application, in order to ensure that the size of the M first single-character images can meet the size requirements of image processing, before performing font beautification processing on the above M first single-character images, the M first The size of the word image is adjusted.

Optionally, in the embodiment of the present application, before the above step 103, the font beautification method provided in the embodiment of the present application may further include the following step 107.

Step 107, the electronic device performs normalization processing on the M first word images.

Wherein, the size of each first word image after normalization processing is within a preset size range. The preset size range is in line with the image processing size requirements

Optionally, in this embodiment of the present application, the size of a single-character image may be the side length and/or area of the single-character image.

In the embodiment of the present application, if the size of a single-character image is within the preset size range, it means that the size of the single-character image meets the image processing size requirements; if the size of a single-character image is outside the preset size range, it means that the size of the single-character image Does not meet image processing size requirements.

It can be understood that in the embodiment of the present application, as shown in FIG. 7 , the electronic device normalizes the individual character images detected from the character row one by one (for example, in the order from "small" to "diameter") to a preset size range , and then input it into the font beautification model for processing to obtain the processed "Small Garden Fragrant Path Lonely Wandering".

It should be noted that, in the embodiment of the present application, in order to make the typesetting relatively neat, after performing font beautification processing on the above-mentioned M first word images after normalization processing to obtain M second word images, the The size of the M second word images is restored to the original size.

In the embodiment of the present application, since the size of the word image to be beautified can be normalized before processing the word image to be beautified, the success rate of processing the word image to be beautified can be ensured.

8, the embodiment of the present application provides a text beautification device 80, the text beautification device 80 may include: a recognition module 81, a processing module 82 and a synthesis module 83; the recognition module 81 is used to perform text processing on the first image Recognition; the processing module 82 is used to determine M image regions in the first image according to the recognition result of the recognition module 81, and each image region includes a character; the processing module 82 is also used to determine the M image regions according to the M image regions Perform image processing on an image to obtain M first word images, and each first word image corresponds to an image area; the processing module 82 is also used to perform font beautification processing on the M first word images to obtain M second word images Image, the font beautification process includes at least one of the following: frame structure processing between fonts, font style processing; synthesis module 83, used to synthesize M second word images obtained by processing module 82 with the original background image of the first image, to obtain second image.

In a possible implementation manner, the above-mentioned processing module 82 is specifically configured to perform font beautification processing on each first individual character image based on font beautification parameters, to obtain a second individual character image corresponding to each first individual character image.

Wherein, the font beautification parameters are parameters obtained by training the neural network model using a large number of word image pairs, and each word image pair includes a word image to be beautified and a word image after beautification.

In a possible implementation, the above-mentioned processing module 82 is also used to synthesize the M second word images with the original background image of the first image in the synthesis module 83 to obtain the second image, according to the M image regions at the first According to the distribution situation in an image, the M second single-character images are divided into at least one image group; and taking the image group as a unit, the second single-character images in at least one image group are subjected to size normalization processing to obtain normalization The processed M second word images.

In a possible implementation manner, the above-mentioned processing module 82 is specifically configured to, for each second single-character image in the M second single-character images, when the first difference between the size of a second single-character image and the target size is less than or When it is equal to the preset difference, the size of a second single character image is adjusted to the first size, and the difference between the first size and the target size is less than the first difference; or, the processing module 82 is specifically used for M For each second single-character image in the second single-character image, when the first difference between the size of the second single-character image and the target size is greater than the preset difference, the size of the second single-character image remains unchanged. Wherein, the target size is the median of the sizes of all the second single-character images in the first image group, and the first image group is the image group to which a second single-character image belongs.

In a possible implementation, the synthesis module 83 includes a determination sub-module and a synthesis sub-module; the determination sub-module is configured to determine the second position information according to the first position information of the M image regions in the first image, the second The second position information is used to indicate the synthesis position of the M second single-character images in the original background image; the synthesis submodule is used to synthesize the M second single-character images and the background image according to the second position information determined by the determination sub-module, Get the third image. Wherein, the similarity between the first arrangement and the second arrangement is greater than or equal to the preset threshold, the first arrangement is the arrangement of M second single-character images in the third image, and the second arrangement is the arrangement of the M image regions in the first image.

In the text beautification device provided in the embodiment of the present application, since the font beautification processing (specifically at least one of font frame structure processing and font style processing) can be performed on a single-character image in an image (for example, the first image), it can ensure that The handwriting of the adjusted single-character image is more aesthetically pleasing, thus improving the effect of beautifying the character image in the image.

For the beneficial effects of the various implementations in this embodiment, refer to the beneficial effects of the corresponding implementations in the foregoing method embodiments. To avoid repetition, details are not repeated here.

The text beautification in this embodiment of the present application may be a device, or a component, an integrated circuit, or a chip in a terminal. The device may be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a notebook computer, a handheld computer, a vehicle electronic device, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant). assistant, PDA), etc., non-mobile electronic devices can be servers, network attached storage (Network Attached Storage, NAS), personal computer (personal computer, PC), television (television, TV), teller machine or self-service machine, etc., this application Examples are not specifically limited.

The text beautifying device in the embodiment of the present application may be a device with an operating system. The operating system may be an Android (Android) operating system, an ios operating system, or other possible operating systems, which are not specifically limited in this embodiment of the present application.

The text beautifying device provided in the embodiment of the present application can realize various processes realized by the method embodiments in FIG. 1 to FIG. 6 , and details are not repeated here to avoid repetition.

As shown in Figure 9, the embodiment of the present application also provides an electronic device 200, including a processor 202, a memory 201, a program or instruction stored in the memory 201 and operable on the processor 202, the program or instruction is processed When executed by the device 202, each process of the above text beautification method embodiment can be realized, and the same technical effect can be achieved. To avoid repetition, details are not repeated here.

It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

FIG. 10 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.

The electronic device 1000 includes, but is not limited to: a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, and a processor 1010, etc. part.

Those skilled in the art can understand that the electronic device 1000 can also include a power supply (such as a battery) for supplying power to various components, and the power supply can be logically connected to the processor 1010 through the power management system, so that the management of charging, discharging, and function can be realized through the power management system. Consumption management and other functions. The structure of the electronic device shown in FIG. 10 does not constitute a limitation to the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange different components, and details will not be repeated here. .

Wherein, the processor 1010 is configured to perform text recognition on the first image; and determine M image areas in the first image, each image area includes a character; and perform image processing on the first image to obtain M number A single-character image, each first single-character image corresponds to an image area; and performing font beautification processing on M first single-character images to obtain M second single-character images, font beautification processing includes at least one of the following: frame structure processing between fonts, Font style processing; the processor 1010 is configured to combine the M second word images obtained by the processor 1010 with the original background image of the first image to obtain a second image.

In a possible implementation manner, the above-mentioned processor 1010 is specifically configured to perform font beautification processing on each first individual character image based on font beautification parameters, to obtain a second individual character image corresponding to each first individual character image.

In a possible implementation manner, the above-mentioned processor 1010 is further configured to combine the M second word images with the original background image of the first image to obtain the second image, according to the M image regions in the first image The distribution of the M second single-character images is divided into at least one image group; and taking the image group as a unit, the size normalization process is carried out to the second single-character images in at least one image group to obtain the normalized M second word images.

In a possible implementation manner, the above-mentioned processor 1010 is specifically configured to, for each second single-character image in the M second single-character images, when the first difference between the size of a second single-character image and the target size is less than or When it is equal to the preset difference, adjust the size of a second single-character image to the first size, and the difference between the first size and the target size is less than the first difference; or, the processor 1010 is specifically used for M For each second single-character image in the second single-character image, when the first difference between the size of the second single-character image and the target size is greater than the preset difference, the size of the second single-character image remains unchanged.

Wherein, the target size is the median of the sizes of all the second single-character images in the first image group, and the first image group is the image group to which a second single-character image belongs.

In a possible implementation manner, the processor 1010 is configured to determine the second position information according to the first position information of the M image regions in the first image, and the second position information is used to indicate that the M second word images are in Compositing positions in the original background image; and according to the determined second position information, compositing M second word images with the background image to obtain a third image.

It should be understood that, in the embodiment of the present application, the input unit 1004 may include a graphics processor (Graphics Processing Unit, GPU) 10041 and a microphone 10042, and the graphics processor 10041 is used for the image capture device ( Such as the image data of the still picture or video obtained by the camera) for processing. The display unit 1006 may include a display panel 10061, and the display panel 10061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 1007 includes a touch panel 10071 and other input devices 10072 . The touch panel 10071 is also called a touch screen. The touch panel 10071 may include two parts, a touch detection device and a touch controller. Other input devices 10072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, and joysticks, which will not be repeated here. The memory 1009 can be used to store software programs as well as various data, including but not limited to application programs and operating systems. The processor 1010 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the foregoing modem processor may not be integrated into the processor 1010 .

The embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, each process of the above text beautification method embodiment can be achieved, and the same Technical effects, in order to avoid repetition, will not be repeated here.

Wherein, the above-mentioned processor is the processor in the electronic device in the above-mentioned embodiment. The above-mentioned readable storage medium includes a computer readable storage medium, such as a computer read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and the like. The embodiment of the present application further provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to realize the various processes of the above text beautification method embodiments, and can achieve the same To avoid repetition, the technical effects will not be repeated here.

It should be understood that the chips mentioned in the embodiments of the present application may also be called system-on-chip, system-on-chip, system-on-a-chip, or system-on-a-chip.

It should be noted that, in this document, the term "comprising", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved. Functions are performed, for example, the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on such an understanding, the technical solution of the present application can be embodied in the form of computer software products, which are stored in a storage medium (such as ROM/RAM, magnetic disk, etc.) , optical disc), including several instructions to enable a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

The embodiments of the present application have been described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementations. The above-mentioned specific implementations are only illustrative and not restrictive. Those of ordinary skill in the art will Under the inspiration of this application, without departing from the purpose of this application and the scope of protection of the claims, many forms can also be made, all of which belong to the protection of this application.

Claims

A text beautification method, said method comprising:

performing text recognition on the first image, determining M image areas in the first image, each image area including a character;

Perform image processing on the first image according to the M image areas to obtain M first word images, each of the first word images corresponds to an image area;

Perform font beautification processing on the M first single-character images to obtain M second single-character images, and the font beautification processing includes at least one of the following: font frame structure processing, font style processing;

Combining the M second word images with the original background image of the first image to obtain a second image.
The method according to claim 1, wherein said performing font beautification processing on said M first single-character images to obtain M second single-character images comprises:

Based on font beautification parameters, performing font beautification processing on each of the first individual character images to obtain a second individual character image corresponding to each of the first individual character images;

Wherein, the font beautification parameters are parameters obtained by training a neural network model using a large number of word image pairs, and each word image pair includes a word image to be beautified and a word image after beautification.
The method according to claim 1 or 2, wherein said synthesis of said M second word images and the original background image of said first image, before obtaining the second image, said method also includes:

Divide the M second word images into at least one image group according to the distribution of the M image areas in the first image;

Taking an image group as a unit, performing size normalization processing on the second single-character images in the at least one image group to obtain M second single-character images after normalization processing.
The method according to claim 3, wherein the size normalization process is performed on the second single-character images in the at least one image group in units of image groups, and M second word images after normalization processing are obtained. Word images, including:

For each second single-character image in the M second single-character images, if the first difference between the size of one second single-character image and the target size is less than or equal to the preset difference, the one second single-character image The size of the two single-character images is adjusted to a first size, and the difference between the first size and the target size is smaller than the first difference; or,

In the case where the first difference between the size of a second single-character image and the target size is greater than a preset difference, keep the size of the second single-character image unchanged;

Wherein, the target size is the median value of the sizes of all the second single-character images in the first image group, and the first image group is the image group to which the one second single-character image belongs.
The method according to claim 1, wherein said synthesis of said M second word images and the original background image of said first image to obtain a second image comprises:

According to the first position information of the M image areas in the first image, determine second position information, the second position information is used to indicate that the M second word images are in the original background image synthetic position;

According to the second position information, combining the M second word images with the background image to obtain the third image;

Wherein, the similarity between the first arrangement and the second arrangement is greater than or equal to a preset threshold, and the first arrangement is the arrangement of the M second single-character images in the third image , the second arrangement is the arrangement of the M image regions in the first image.
A text beautification device, said device comprising: a recognition module, a processing module and a synthesis module;

The recognition module is configured to perform text recognition on the first image;

The processing module is configured to determine M image regions in the first image according to the recognition result of the recognition module, and each image region includes a character;

The processing module is further configured to perform image processing on the first image according to the M image regions to obtain M first word images, and each first word image corresponds to an image region;

The processing module is further configured to perform font beautification processing on the M first single-character images to obtain M second single-character images, and the font beautification processing includes at least one of the following: font frame structure processing and font style processing;

The synthesizing module is configured to synthesize the M second word images processed by the processing module with the original background image of the first image to obtain a second image.
The apparatus of claim 6, wherein,

The processing module is specifically configured to perform font beautification processing on each of the first individual character images based on font beautification parameters, to obtain a second individual character image corresponding to each of the first individual character images;

Wherein, the font beautification parameters are parameters obtained by training a neural network model using a large number of word image pairs, and each word image pair includes a word image to be beautified and a word image after beautification.
Apparatus according to claim 6 or 7, wherein,

The processing module is further configured to combine the M second word images with the original background image of the first image by the synthesis module to obtain the second image, according to the M image regions in the For the distribution in the first image, divide the M second single-character images into at least one image group; and use the image group as a unit to normalize the size of the second single-character images in the at least one image group normalized processing to obtain M second single-character images after normalized processing.
The apparatus of claim 8, wherein,

The processing module is specifically configured to, for each second single-character image in the M second single-character images, when the first difference between the size of a second single-character image and the target size is less than or equal to the preset difference In this case, the size of the second single-character image is adjusted to a first size, and the difference between the first size and the target size is smaller than the first difference; or,

The processing module is specifically configured to, for each of the M second single-character images, when the first difference between the size of a second single-character image and the target size is greater than a preset difference , keeping the size of the second word image unchanged;

Wherein, the target size is the median value of the sizes of all the second single-character images in the first image group, and the first image group is the image group to which the one second single-character image belongs.
The device according to claim 6, wherein the synthesis module comprises a determination sub-module and a synthesis sub-module;

The determination submodule is configured to determine second position information according to the first position information of the M image regions in the first image, and the second position information is used to indicate the M second words the composite position of the image within said original background image;

The synthesis submodule is configured to synthesize the M second word images and the background image according to the second position information determined by the determination submodule to obtain the third image;

Wherein, the similarity between the first arrangement and the second arrangement is greater than or equal to a preset threshold, and the first arrangement is the arrangement of the M second single-character images in the third image , the second arrangement is the arrangement of the M image regions in the first image.
An electronic device, comprising a processor, a memory, and a program or instruction stored on the memory and operable on the processor, when the program or instruction is executed by the processor, claims 1 to 5 are realized The steps of the text beautification method described in any one.
A readable storage medium, storing programs or instructions on the readable storage medium, and implementing the steps of the text beautification method according to any one of claims 1 to 5 when the programs or instructions are executed by a processor.
A computer program product, the computer program product is executed by at least one processor to implement the text beautification method according to any one of claims 1 to 5.
An electronic device configured to execute the text beautification method according to any one of claims 1 to 5.
A chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the text beautification method described in any one of 1 to 5 .