WO2024066490A1 - Character beautification method and electronic device - Google Patents

Abstract

Provided in the present application are a character beautification method and an electronic device. The method comprises: an electronic device generating, in response to a first operation by a user on a touch screen, a first character track, and recognizing the first character track so as to obtain a first style of the first character track; and also acquiring a second style of a target font; then generating a third style according to the first style and the second style; and then beautifying the first character track according to the third style so as to obtain a second character track, and displaying the second character track on the touch screen. In this way, the electronic device can fuse the first style of the first character track with the second style of the target font, so as to adjust the font structure of character tracks written by a user on the touch screen while retaining the original writing style of the user, thereby achieving a style-adaptive beautification effect, and satisfying different requirements of different users for character beautification.

Description

Text beautification method and electronic device

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the priority of the Chinese patent application filed with the China Patent Office on September 26, 2022, with application number 202211180551.1 and application name “A Text Beautification Method and Electronic Device”, all contents of which are incorporated by reference in this application.

Technical Field

The present application relates to the field of computer vision technology, and in particular to a text beautification method and electronic equipment.

Background technique

With the rapid development of science and technology, more and more electronic devices for writing text (such as electronic whiteboards) have gradually replaced traditional writing boards (such as traditional ordinary whiteboards) in corporate meetings, medical or educational scenarios. However, due to the materials of electronic devices and styluses, as well as the touch acquisition and display principles of electronic device screens, the writing content on electronic devices is composed of simple lines, so there will be large differences in text outline, text thickness, etc. compared with paper writing, resulting in the font of the user's writing content on the electronic device not being beautiful enough. At the same time, because the writing posture on the electronic device is different from the writing posture on paper, the handwriting on the electronic device will be deformed and difficult to read. In view of this, in order to effectively increase the readability and beauty of the user's handwritten text, it is necessary to beautify the writing content of the user on the electronic device.

At present, in a text beautification scheme, the purpose of beautifying the text input by the user is achieved by beautifying the pen tip of the text input by the user and rendering the stroke of the text input by the user. However, this scheme of only changing the stroke outline cannot achieve a good beautification effect.

Summary of the invention

The embodiments of the present application provide a text beautification method and an electronic device, so as to meet different needs of different users for text beautification.

In the first aspect, an embodiment of the present application provides a text beautification method, which can be applied to an electronic device with a touch screen. In this method, the electronic device generates a first text track in response to a first operation of a user on the touch screen, and can identify the first text track to obtain a first style of the first text track, wherein the first style is used to characterize the font features of the first text track. And, the electronic device can also obtain a second style of the target font, wherein the second style is used to characterize the font features of the text track of the target font. Afterwards, the electronic device can generate a third style based on the first style and the second style. Then, the electronic device can beautify the first text track according to the third style to obtain a second text track, and can display the second text track on the touch screen.

In the above design, the electronic device can achieve the complementarity of two different styles by fusing the first style of the first text track formed by the user on the touch screen with the second style of the target font automatically selected or manually selected by the user, and then beautify the first text track formed by the user on the touch screen according to the third style obtained by the fusion. It can be achieved on the basis of retaining the user's original writing style and adjusting the font structure of the text track formed by the user on the touch screen, so as to achieve the beautification effect of adaptive style, thereby meeting the different needs of different users for text beautification.

In a possible design, recognizing the first text track to obtain the first style of the first text track includes:

The first text track is input into the style recognition model to obtain a first style.

In the above design, the electronic device recognizes the first text track through the pre-trained style recognition model, and can obtain the first style of the first text track in a timely and accurate manner.

In a possible design, obtaining the second style of the target font includes:

The text model corresponding to the target font is input into the style recognition model to obtain the second style; or the second style of the target font is determined from the style library.

In the above design, there are two implementation methods that can enable the electronic device to obtain the second style of the target font. One implementation method is that the electronic device recognizes the text model corresponding to the target font through a pre-trained style recognition model, and can obtain the second style of the target font in a timely and accurate manner; the other implementation method is that the electronic device can obtain the second style of the target font in a timely and accurate manner through a pre-created style library, which stores multiple fonts and the styles of the multiple fonts.

In one possible design, the method further includes:

The target font is selected in a font library, wherein the font library contains at least one font; or the target font is acquired in response to a second operation of the user on the touch screen.

In the above design, there are two implementation methods for realizing the selection of the target font, and the font selection method is relatively flexible. That is, one implementation method is that the electronic device selects a font matching the first text track in the font library as the target font, which helps to realize the automatic selection of the target font, and the accuracy of the automatic selection is also relatively high; another implementation method is that the electronic device provides the user with the function of selecting the target font. For example, the target font can be selected by the user through the selection operation on the touch screen, which helps to determine the target font according to the actual needs of the user, and largely meets the user's needs for text beautification. It can be adapted to the different understandings of text beautification by different users, thereby improving the user experience.

In one possible design, a target font is selected in a font library, including:

calculating a similarity between the first text track and each font of at least one font;

In the font library, a font having the highest similarity to the first text track is selected as the target font.

In the above design, the electronic device can accurately select a target font that matches the first text track by calculating the similarity between the first text track and each font in at least one font in the font library.

In a possible design, calculating the similarity between the first text track and each font in at least one font includes:

Inputting the first text trajectory into a first handwriting feature extraction model to obtain a first handwriting feature vector of the first text trajectory;

The similarity between the first text track and each font in the at least one font is calculated according to the first handwriting feature vector and the handwriting feature vector of the at least one font.

In the above design, the electronic device performs feature extraction on the first text trajectory through a pre-trained first handwriting feature extraction model, and can timely and accurately extract the first handwriting feature vector, so that the similarity between the first text trajectory and each font in at least one font can be timely and effectively calculated through the first handwriting feature vector and the handwriting feature vector of at least one font.

In one possible design, the method further includes:

A first font is determined, and the determined first font is updated to a font library.

In the above design, the electronic device can update the font library in real time or periodically. For example, the electronic device can update the font library with a new font that is determined each time.

In a possible design, determining the first font includes:

The third text track is input into the first handwriting feature extraction model to obtain the second handwriting feature vector of the third text track, and the second handwriting feature vector is input into the font identification model to obtain the font of the third text track as the first font; wherein, The third text track is generated according to a third operation triggered by any user on the touch screen; or

The target text model is input into the second handwriting feature extraction model to obtain a third handwriting feature vector of the target text model, and the third handwriting feature vector is input into the font identification model to obtain that the font corresponding to the target text model is the first font.

In the above design, the electronic device can determine the first font in two implementation methods, namely, one implementation method is that the electronic device performs font identification on the text track generated by the user's online operation to determine the first font; the other implementation method is that the electronic device performs font identification on the target text model in the form of an image or picture to determine the first font. In this way, through the above design, the font identification needs in different scenarios can be met, and the coverage of the font library can be relatively wide, so that different font selection needs can be met to a certain extent.

In a possible design, in response to a second operation of the user on the touch screen, obtaining a target font includes:

When detecting a user operation on a font selection icon on the touch screen, displaying a plurality of candidate fonts;

In response to a second operation in which the user selects any one of the plurality of candidate fonts, the candidate font selected by the user is used as the target font.

In the above design, the electronic device provides the user with the function of selecting a certain font as the target font. For example, when the electronic device detects that the user has triggered a font selection operation on the touch screen, it will display multiple candidate fonts for the user to choose from. The font selection flexibility is relatively high, which helps to meet the user's different font selection needs in different scenarios, thereby improving the user experience.

In a possible design, a third style is generated according to the first style and the second style, including:

Obtaining a beautification parameter value for adjusting a beautification degree of the first text track;

The beautification parameter value, the first style, and the second style are input into the feature fusion model to obtain the third style.

In the above design, the electronic device provides the user with a function of selecting the degree of text beautification. For example, the user can select or input a beautification parameter value on the touch screen according to his or her text beautification needs, which helps to meet the different beautification needs of different users, thereby meeting the different beautification needs of different users. Optionally, the beautification parameter value can also be pre-configured in the electronic device, so that when the electronic device needs to beautify a certain text track, the pre-configured beautification parameter value can be directly used.

In the second aspect, the embodiment of the present application provides an electronic device, and the beneficial effects can be found in the description of the first aspect, which will not be repeated here. The electronic device has the function of implementing the behavior in the method example of the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In a possible design, the electronic device includes an acquisition module and a processing module, the processing module is used to generate a first text track in response to a first operation of a user on a touch screen; the processing module is also used to identify the first text track and obtain the first style of the first text track; wherein the first style is used to characterize the font features of the first text track; the acquisition module is used to obtain the second style of the target font; the second style is used to characterize the font features of the text track of the target font; the processing module is also used to generate a third style according to the first style and the second style; the processing module is also used to beautify the first text track according to the third style to obtain the second text track; the processing module is also used to display the second text track on the touch screen. These modules can perform the corresponding functions in the method example of the first aspect, and refer to the detailed description in the method example for details, which will not be repeated here.

In a third aspect, an embodiment of the present application provides an electronic device, the electronic device comprising a communication interface and a processor, and optionally, a memory. The memory is used to store computer programs or instructions, and the processor is coupled to the memory and the communication interface. When the processor executes the computer program or instruction, the electronic device executes the method in any possible design of the first aspect.

In a fourth aspect, an embodiment of the present application provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are run on a computer, the computer executes a method in any possible design of the first aspect above.

In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program or instructions are stored. When the computer program or instructions are executed by a computer, the computer executes a method in any possible design of the first aspect above.

In a sixth aspect, an embodiment of the present application further provides a chip, which is coupled to a memory, and is used to read a computer program stored in the memory to execute a method in any possible design of the first aspect above.

In a seventh aspect, an embodiment of the present application further provides a chip system, which includes a processor for supporting a computer device to implement the method in any possible design of the first aspect. In one possible design, the chip system also includes a memory, which is used to store the necessary programs and data of the computer device. The chip system can be composed of a chip, or it can include a chip and other discrete devices.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 exemplarily shows a schematic diagram of an application scenario provided by an embodiment of the present application;

FIG2 exemplarily shows a schematic structural diagram of an electronic device provided in an embodiment of the present application;

FIG3 exemplarily shows a schematic diagram of a functional module structure of an electronic device provided in an embodiment of the present application;

FIG4a exemplarily shows a slide bar type beautification parameter value selection box provided in an embodiment of the present application;

FIG4b exemplarily shows a selection box including multiple beautification parameter values provided in an embodiment of the present application;

FIG4c exemplarily shows a sliding button type beautification parameter value selection box provided in an embodiment of the present application;

FIG5 exemplarily shows a flow chart of a text beautification method provided in an embodiment of the present application;

FIG6a exemplarily shows a schematic diagram of a style recognition model provided in an embodiment of the present application recognizing a text track;

FIG6b exemplarily shows a schematic diagram of generating a third style provided by an embodiment of the present application;

FIG6c exemplarily shows a schematic diagram of generating a second text trajectory provided by an embodiment of the present application;

FIG7 exemplarily shows a structural diagram of a possible electronic device provided in an embodiment of the present application;

FIG8 exemplarily shows a schematic structural diagram of a possible electronic device provided in an embodiment of the present application.

Detailed ways

Before introducing the technical solution provided by the present application, some of the terms involved in the present application are first explained to facilitate understanding by those skilled in the art.

(1) Artificial intelligence (AI) algorithm: Artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, acquire knowledge and use knowledge to obtain the best results. It attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is also the study of the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

Generally speaking, AI algorithms can include machine learning algorithms and deep learning algorithms. Among them, machine learning algorithms can include decision trees, logistic regression (LR), naive Bayes (NB) classification algorithms, random forest (RF) algorithms, support vector machines (SVM) classification algorithms, etc. Deep learning algorithms can include neural networks, deep neural networks, convolutional neural networks (CNN), generative adversarial networks (GAN), etc. Deep Deep learning can be understood as an optimization of machine learning, which can use a large number of training samples to obtain more accurate calculation results. There are certain differences between machine learning and deep learning. For example, when the amount of training data is small, machine learning is more effective, and when the amount of training data is large, deep learning may be more effective.

The following is a brief introduction using neural networks, deep neural networks, convolutional neural networks, and generative adversarial networks as examples.

(2) Neural network: A neural network can be composed of neural units. In simple terms, a neural unit refers to a computing unit with computing capabilities. A neural network can be formed by connecting many of the above-mentioned single neural units together. Different neural units in a neural network can be connected. For example, the output of one neural unit can be the input of another neural unit. In this way, the final output signal is obtained by passing through all neural units one by one.

(3) Deep Neural Network (DNN): also known as multi-layer neural network. Generally, the neural network inside a DNN can be divided into three categories: input layer, hidden layer, and output layer. The first layer is the input layer, which is used to receive input data, and the last layer is the output layer, which is used to output data. The layers between the first and last layers are all hidden layers. The layers are fully connected, that is, any neuron in the i-th layer must be connected to any neuron in the i+1-th layer.

It should be noted that the input layer does not have a weight matrix. Since there are many DNN layers, there are also many weight matrices and offset vectors. The process of training a deep neural network is the process of learning the weight matrix, which makes the output of the trained deep neural network closer to the ideal result. In a deep neural network, more hidden layers allow the network to better describe complex situations in the real world.

(4) Convolutional Neural Network (CNN): It is a deep neural network with a convolutional structure. Unlike deep neural networks (DNNs), weights in convolutional neural networks (CNNs) are replaced by convolution kernels. The convolution kernel can be a two-dimensional or three-dimensional matrix, which is used to perform convolution operations with input data (such as output images) to extract feature information from the input data. If the input data is an image, the extracted feature information can be feature information such as color, brightness, exposure, grayscale, etc. on the image.

Exemplarily, the neural network inside CNN can also be divided into three categories: input layer, hidden layer, and output layer. Each layer corresponds to a feature extractor, which includes a convolution kernel. The same layer can correspond to one or more feature extractors, that is, one or more convolution kernels, and the feature extractors corresponding to different layers can be the same or different. It should be noted that the input layer may not have a feature extractor. Each layer can use the feature extractor corresponding to the layer to extract features from the input image input to the layer.

Generally speaking, the more convolution kernels there are, the richer the image information reflected by the convolution operation, which can improve the accuracy of image recognition in the subsequent process. Moreover, a specific feature extractor (i.e., a specific convolution kernel) can extract specific feature information from the input data, so the design of the feature extractor is particularly important. Therefore, the training process of CNN is the process of training the convolution kernel in the feature extractor.

(5) Generative Adversarial Network: It is an important generative model in the field of deep learning. That is, two networks (generator and discriminator) are trained at the same time and compete in the minimax algorithm. This adversarial approach avoids some difficulties of traditional generative models in practical applications. It cleverly approximates some unsolvable loss functions through adversarial learning and has been widely used in the generation of data such as images, videos, natural language and music.

For example, the generator randomly samples from the latent space as input, and its output needs to imitate the real samples in the training set as much as possible. The input of the discriminator is the real sample or the output of the generator network, and its purpose is to distinguish the output of the generator from the real sample as much as possible. The generator should deceive the discriminant network as much as possible. The two networks compete with each other and constantly adjust parameters, and the ultimate goal is to make it impossible for the discriminator to determine whether the output of the generator is real.

(6) Loss function: Taking the training of CNN as an example, in the process of training CNN, because we hope that the output of CNN is as close to the desired value as possible, we can compare the current CNN output value with the desired target value, and then update the weight of each layer of the neural network in CNN according to the difference between the two (of course, there is usually an initialization process before the first update, that is, pre-configuring parameters for each layer of the network). For example, if the output value is high, adjust the weight to make its output value lower, and continue to adjust until the desired target value or a value very close to the desired target value can be obtained. Therefore, it is necessary to predefine "how to compare the difference between the predicted value and the target value", which is the loss function (loss function) or objective function (objective function). They are important equations used to measure the difference between the output value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference, so the training process becomes a process of minimizing this loss as much as possible.

In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. It should be understood that the specific operating methods in the method embodiments can also be applied to device embodiments or system embodiments. Moreover, in the description of the present application, "at least one" refers to one or more, and multiple refers to two or more. In view of this, "multiple" can also be understood as "at least two" in the embodiments of the present invention. "And/or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and/or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the characters "three kinds, unless otherwise specified, generally indicate that the objects associated before and after are in an "or" relationship. In addition, it should be understood that in the description of the present application, words such as "first" and "second" are only used to distinguish the purpose of description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.

The embodiments of the present application are described in detail below with reference to the accompanying drawings.

The technical solution of the text beautification method disclosed in this application can be applied to electronic devices equipped with touch screens. In some embodiments of this application, the electronic device can be a portable electronic device with a processor, such as a mobile phone, a tablet computer, a wearable device with wireless communication function (such as a smart watch, etc.), a vehicle-mounted device, etc. Exemplary embodiments of portable electronic devices include but are not limited to devices equipped with Or a portable electronic device with other operating systems. The portable electronic device may also be a laptop computer (Laptop) with a touch-sensitive surface (such as a touch panel, etc.). It should be understood that in some other embodiments of the present application, the electronic device may not be a portable device, for example, it may be an electronic whiteboard, such as a cabinet electronic whiteboard or a wall-mounted electronic whiteboard, or it may be a desktop computer, such as a PC, or it may be a television, etc.

FIG1 exemplarily shows a schematic diagram of an application scenario provided by an embodiment of the present application. The application scenario includes a user 100 and an electronic device 200. Optionally, the touch screen can be set in the electronic device 200, or the touch screen can also be set independently of the electronic device, for example, the touch screen is set as an independent entity, and the touch screen is connected to an electronic device including a processor. Optionally, multiple applications can be installed in the electronic device 200, such as video conferencing, screen writing, whiteboard, file management, camera, image and other applications. Exemplarily, taking the case where the user 100 selects the whiteboard application in the electronic device 200 for text writing, when the user 100 writes one or more words on the whiteboard, the electronic device 200 can beautify the one or more words written by the user 100 using the text beautification method provided in the embodiment of the present application, and display the beautified one or more words on the touch screen. At this time, the user 100 can see that the one or more words displayed on the touch screen are beautified, which brings the user a visual impact of readability and aesthetic comfort of the text.

It should be noted that FIG. 1 only schematically provides an application scenario, and does not limit the application scenario of the text beautification method provided in this application. The text beautification method provided in this application can be applied to more scenarios. Let me list them one by one.

Based on the application scenario shown in FIG1 , an embodiment of the present application provides an electronic device. Referring to FIG2 , the electronic device may include a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 251, a wireless communication module 252, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, an earphone interface 270D, a sensor module 280, a button 290, a motor 291, an indicator 292, a camera 293, a display screen 294, and a subscriber identification module (SIM) card interface 295, etc. Among them, the sensor module 280 may include a pressure sensor 280A, a gyroscope sensor 280B, an air pressure sensor 280C, an acceleration sensor 280D, a proximity light sensor 280E, an ambient light sensor 280F, a fingerprint sensor 280G, a temperature sensor 280H and a touch sensor 280I. Of course, the electronic device 200 may also include other sensors, such as a magnetic sensor, a distance sensor, a bone conduction sensor, etc.

It should be noted that the antenna 1 and antenna 2 shown in FIG. 2 are examples, and other antennas may also be included in other embodiments. It should be understood that the electronic device shown in FIG. 2 is only an example and does not constitute a limitation on the electronic device. For example, the electronic device 200 may have more or fewer components than those shown in the figure, may combine two or more components, or may have different component configurations. The various components shown in the figure may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application-specific integrated circuits.

The following is a detailed introduction to some components of the electronic device 200 in conjunction with FIG. 2 :

The processor 210 in FIG. 2 may include one or more processing units. For example, the processor 210 may include an application processor (AP), a baseband processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP) and/or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. The controller may be the nerve center and command center of the electronic device 200. The controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions.

The processor 210 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 210 is a cache memory. The memory may store instructions or data that the processor 210 has just used or cyclically used. For example, the memory may store program codes of a text beautification method. If the processor 210 needs to use the instruction or data again, it may be directly called from the memory, thereby avoiding repeated access, reducing the waiting time of the processor 210, and thus improving the efficiency of the system.

The processor 210 can run the text beautification method provided in the present application, and the processor 210 can respond to the user's operation on the touch screen and start the function corresponding to the user's operation. When the processor 210 integrates different devices, such as a CPU and a GPU, the CPU and the GPU can cooperate to execute the text beautification method provided in the present application to obtain faster processing efficiency.

The external memory interface 220 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 200. The external memory card communicates with the processor 210 through the external memory interface 220 to implement a data storage function. For example, files such as pictures and videos can be saved in the external memory card.

The internal memory 221 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 221 may include a program storage area and a data storage area. The program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), and may store files, The program code of the word beautification method. The data storage area can store data (such as audio data, phone book, etc.) created during the use of the electronic device 200. In addition, the internal memory 221 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 210 executes various functional applications and data processing of the electronic device 200 by running instructions stored in the internal memory 221 and/or instructions stored in a memory provided in the processor.

In some embodiments, the processor 210 may include one or more interfaces. For example, the interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver/transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input/output (GPIO) interface, a subscriber identity module (SIM) interface, and/or a universal serial bus (USB) interface.

The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 210 may include multiple groups of I2C buses. The processor 210 may be coupled to the touch sensor 280I, the charger, the flash, the camera 293, etc. through different I2C bus interfaces. For example, the processor 210 may be coupled to the touch sensor 280I through the I2C interface, so that the processor 210 communicates with the touch sensor 280I through the I2C bus interface, thereby realizing the touch function of the electronic device 200.

The I2S interface can be used for audio communication. In some embodiments, the processor 210 can include multiple groups of I2S buses. The processor 210 can be coupled to the audio module 270 via the I2S bus to achieve communication between the processor 210 and the audio module 270. In some embodiments, the audio module 270 can transmit an audio signal to the wireless communication module 252 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.

The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 270 and the wireless communication module 252 can be coupled via a PCM bus interface. In some embodiments, the audio module 270 can also transmit audio signals to the wireless communication module 252 via the PCM interface to realize the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 210 and the wireless communication module 252. For example, the processor 210 communicates with the Bluetooth module in the wireless communication module 252 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 270 can transmit an audio signal to the wireless communication module 252 through the UART interface to implement the function of playing music through a Bluetooth headset.

The MIPI interface can be used to connect the processor 210 with peripheral devices such as the display screen 294 and the camera 293. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 210 and the camera 293 communicate via the CSI interface to realize the shooting function of the electronic device 200. The processor 210 and the display screen 294 communicate via the DSI interface to realize the display function of the electronic device 200.

The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 210 with the camera 293, the display 294, the wireless communication module 252, the audio module 270, the sensor module 280, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

The USB interface 230 is an interface that complies with the USB standard specification, and can be a Mini USB interface, a Micro USB The USB interface 230 may be used to connect a charger to charge the electronic device 200, or to transmit data between the electronic device 200 and a peripheral device. The USB interface 230 may also be used to connect headphones to play audio. The interface may also be used to connect other electronic devices, such as AR devices.

It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 200. In other embodiments of the present application, the electronic device 200 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

The charging management module 240 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 240 may receive charging input from a wired charger through the USB interface 230. In some wireless charging embodiments, the charging management module 240 may receive wireless charging input through a wireless charging coil of the electronic device 200. While the charging management module 240 is charging the battery 242, it may also power the electronic device 200 through the power management module 241.

The power management module 241 is used to connect the battery 242, the charging management module 240 and the processor 210. The power management module 241 receives input from the battery 242 and/or the charging management module 240, and supplies power to the processor 210, the internal memory 221, the display screen 294, the camera 293, and the wireless communication module 252. The power management module 241 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 241 can also be set in the processor 210. In other embodiments, the power management module 241 and the charging management module 240 can also be set in the same device.

The wireless communication function of the electronic device 200 can be implemented through the antenna 1, the antenna 2, the mobile communication module 251, the wireless communication module 252, the modem processor and the baseband processor.

Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 200 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

The mobile communication module 251 can provide solutions for wireless communications including 2G/3G/4G/5G applied to the electronic device 200. The mobile communication module 251 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 251 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 251 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 251 can be set in the processor 210. In some embodiments, at least some of the functional modules of the mobile communication module 251 can be set in the same device as at least some of the modules of the processor 210.

The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 270A, a receiver 270B, etc.), or displays an image or video through a display screen 294. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 210 and be set in the same device as the mobile communication module 251 or other functional modules.

The wireless communication module 252 can provide wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM) and other communication functions applied to the electronic device 200. The wireless communication module 252 can be one or more devices integrating at least one communication processing module. The wireless communication module 252 receives electromagnetic waves via the antenna 2, modulates the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 210. The wireless communication module 252 can also receive the signal to be sent from the processor 210, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

In some embodiments, the antenna 1 of the electronic device 200 is coupled to the mobile communication module 251, and the antenna 2 is coupled to the wireless communication module 252, so that the electronic device 200 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), new radio access technology (NR), BT, GNSS, WLAN, NFC, FM, and/or IR technology.

Optionally, GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and/or a satellite based augmentation system (SBAS).

The electronic device 200 implements the display function through a GPU, a display screen 294, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 294 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 210 may include one or more GPUs, which execute program instructions to generate or change display information.

The display screen 294 is used to display images, videos, etc. The display screen 294 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diodes (QLED), etc.

NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the electronic device 200 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

The pressure sensor 280A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 280A can be set on the display screen 294. There are many types of pressure sensors 280A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. A capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 280A, the capacitance between the electrodes changes. The electronic device 200 determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 294, the electronic device 200 detects the intensity of the touch operation according to the pressure sensor 280A. The electronic device 200 can also calculate the position of the touch based on the detection signal of the pressure sensor 280A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions.

The gyro sensor 280B can be used to determine the motion posture of the electronic device 200. In some embodiments, the angular velocity of the electronic device 200 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 280B. The gyro sensor 280B can be used for shooting anti-shake. The gyro sensor 280B can also be used for navigation and somatosensory game scenes.

The air pressure sensor 280C is used to measure air pressure. In some embodiments, the electronic device 200 calculates the altitude through the air pressure value measured by the air pressure sensor 280C to assist in positioning and navigation.

The acceleration sensor 280D can detect the magnitude of the acceleration of the electronic device 200 in all directions (generally three axes). When the electronic device 200 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

The proximity light sensor 280E may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 200 emits infrared light outward through the light emitting diode. The electronic device 200 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 200. When insufficient reflected light is detected, the electronic device 200 can determine that there is no object near the electronic device 200. The electronic device 200 can use the proximity light sensor 280E to detect that the user holds the electronic device 200 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 280E can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

The ambient light sensor 280F is used to sense the ambient light brightness. The electronic device 200 can adaptively adjust the brightness of the display screen 294 according to the perceived ambient light brightness. The ambient light sensor 280F can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 280F can also cooperate with the proximity light sensor 280E to detect whether the electronic device 200 is in a pocket to prevent accidental touch.

The fingerprint sensor 280G is used to collect fingerprints. The electronic device 200 can use the collected fingerprint characteristics to realize fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc. For example, a fingerprint sensor can be configured on the front of the electronic device 200 (below the display screen 294), or a fingerprint sensor can be configured on the back of the electronic device 200 (below the rear camera). In addition, the fingerprint recognition function can also be realized by configuring a fingerprint sensor in the touch screen, that is, the fingerprint sensor can be integrated with the touch screen to realize the fingerprint recognition function of the electronic device 200. In this case, the fingerprint sensor can be configured in the touch screen, can be a part of the touch screen, or can be configured in the touch screen in other ways.

In addition, the fingerprint sensor can also be implemented as a full-panel fingerprint sensor, so the touch screen can be regarded as a panel where fingerprints can be collected at any position. In some embodiments, the fingerprint sensor can process the collected fingerprint (for example, whether the fingerprint is verified) and send it to the processor 210, and the processor 210 makes corresponding processing according to the fingerprint processing result. In other embodiments, the fingerprint sensor can also send the collected fingerprint to the processor 210 so that the processor 210 processes the fingerprint (for example, fingerprint verification, etc.). The fingerprint sensor in this application can adopt any type of sensing technology, including but not limited to optical, capacitive, piezoelectric or ultrasonic sensing technology.

The temperature sensor 280H is used to detect temperature. In some embodiments, the electronic device 200 uses the temperature detected by the temperature sensor 280H to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 280H exceeds a threshold, the electronic device 200 reduces the performance of a processor located near the temperature sensor 280H to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 200 heats the battery 242 to avoid abnormal shutdown of the electronic device 200 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 200 performs a boost on the output voltage of the battery 242 to avoid abnormal shutdown caused by low temperature.

The touch sensor 280I is also called a "touch panel". The touch sensor 280I can be set on the display screen 294. The touch sensor 280I and the display screen 294 form a touch screen, also called a "touch screen". The touch sensor 280I is used to detect a touch operation acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 294. In other embodiments In the embodiment, the touch sensor 280I may also be disposed on the surface of the electronic device 200, which is different from the position of the display screen 294.

Based on the electronic device shown in FIG2 , the embodiment of the present application further provides a functional module structure of the electronic device. Referring to FIG3 , according to the logical functions of the electronic device, the following modules can be divided: a target font selection module, a text beautification generation module, and a beautification degree module. The functions of each module are described below.

The target font selection module is used to determine the target font in response to the user's manual selection, or to extract handwriting features of the text track generated by the corresponding operation triggered by the user on the touch screen, and determine the corresponding target font from the font library based on the extracted handwriting feature vector.

For example, taking the case where the user manually selects and determines the target font, the target font selection module responds to the user's corresponding operation on the touch screen (such as the user clicking the target font button on the touch screen) and displays a target font selection box on the touch screen. The user can select a standard font such as Kaishu, Caoshu or Songti from the target font selection box, or can also select a popular online font from the target font selection box. Then, the target font selection module responds to the user's operation of selecting a font in the target font selection box and determines that the font is the target font required by the user.

The text beautification generation module is used to extract the first style of the text track generated by the user on the touch screen through the style recognition model, and extract the second style of the text model corresponding to the target font through the style recognition model. Then, the text beautification generation module merges the first style and the second style to generate a third style, and performs beautification processing on the text track generated by the user on the touch screen according to the third style to generate a beautified text track.

Exemplarily, a user writes a certain word (such as "永") on the touch screen, and it is assumed that the target font is regular script. The text beautification generation module can extract the style of the word "永" written by the user through a style recognition model, such as a pre-trained GAN-based deep neural network model, and can extract the style of a regular script word (such as "永") through a style recognition model. Then, the style of the word "永" written by the user is merged with the style of the regular script word "永" to generate a merged style, and the word "永" written by the user is beautified according to the merged style to generate a beautified word "永".

Of course, the text beautification generation module can also extract the styles of other regular script characters (such as the regular script characters "you", "I" or "he", etc.) through the style recognition model, and merge the styles of other regular script characters with the style of the text "永" written by the user. The embodiment of the present application is not limited to this.

Optionally, in the process of fusing the first style and the second style, the text beautification generation module may also obtain a configured beautification parameter value for adjusting the beautification degree of the text track generated by the user on the touch screen, or may also obtain a beautification parameter value selected or input by the user for adjusting the beautification degree of the text track generated by the user on the touch screen. Then, the text beautification generation module fuses the beautification parameter value, the first style and the second style to generate a third style.

The beautification degree module is used to adjust the text track generated by the user on the touch screen according to the beautification parameter value selected or input by the user.

For example, the beautification degree module responds to a certain operation of the user on the touch screen (such as the user clicking a beautification parameter value input button on the touch screen) and displays a beautification parameter value input box, in which the user can input a corresponding beautification parameter value.

Exemplarily, the beautification degree module displays a beautification parameter value input box in response to the user clicking a beautification parameter value input button on the touch screen. For example, if the user enters 5 in the beautification parameter value input box, 5 can be used as the beautification parameter value for adjusting the beautification degree of the text trajectory generated by the user on the touch screen.

For another example, the beautification degree module responds to other operations of the user on the touch screen (such as the user clicking on the touch screen). Beautification parameter value selection button), displaying a beautification parameter value selection box, for example, a slide bar type beautification parameter value selection box as shown in FIG4a (the user can adjust the slide bar button to select the corresponding beautification parameter value), a selection box containing multiple beautification parameter values as shown in FIG4b (the user can select the corresponding beautification parameter value from multiple beautification parameter values), a slide button type beautification parameter value selection box as shown in FIG4c (the user can select the corresponding beautification parameter value by sliding the button), etc.

Exemplarily, taking a slider-type beautification parameter value selection box as an example, the beautification degree module displays a slider-type beautification parameter value selection box in response to the user clicking a beautification parameter value selection button on the touch screen. For example, the value range of the slider-type beautification parameter value selection box is 1-10, and the user can select a beautification parameter value (for example, the beautification parameter value is 3) by adjusting the slider button. Then 3 can be used as a beautification parameter value for adjusting the beautification degree of the text trajectory generated by the user on the touch screen.

A text beautification method provided by an embodiment of the present application is described below in conjunction with the accompanying drawings. The method is applicable to an electronic device with a touch screen, such as the electronic device 200 shown in FIG. 1 . As shown in FIG. 5 , the method includes:

Step 501: In response to a first operation of a user on a touch screen, a first text track is generated.

Optionally, when a text writing application is installed in the electronic device, if the user needs to perform a text writing operation through the electronic device, the user can click the text writing application (such as a whiteboard application) on the touch screen of the electronic device, and the text writing application responds to the user's click operation and displays a text writing interface. Then, the user can write one or more characters on the text writing interface, so that the one or more characters form corresponding text tracks on the text writing interface.

Optionally, when the electronic device itself is used as a text writing device (such as an electronic whiteboard), the user can directly perform text writing operations on the touch screen of the electronic device, for example, the user can write on the blackboard on the touch screen of the electronic device, or the user can also write meeting content on the touch screen of the electronic device. In this way, one or more characters written by the user on the touch screen of the electronic device form corresponding text tracks.

For example, the electronic device is an electronic whiteboard. For example, when a user attends a corporate meeting held by a company and needs to handwrite relevant meeting content, the user can write the corresponding meeting content on the touch screen of the electronic whiteboard. For example, the user writes multiple words "Second quarter corporate sales" on the touch screen of the electronic whiteboard. In this way, multiple words "Second quarter corporate sales" form corresponding text tracks on the touch screen of the electronic whiteboard.

Step 502: Recognize the first text track to obtain the first style of the first text track.

Optionally, the electronic device may identify the first text track through a style recognition model to obtain a first style of the first text track. The first style may be used to characterize the font features of the first text track. It should be understood that the style recognition model may be obtained through supervised learning based on samples and stored in the electronic device.

Optionally, the electronic device may also perform style extraction on the first text track through a style extraction model to obtain a first style of the first text track. Exemplarily, the style extraction model may be a pre-trained VGG-16 (Visual Geometry Group Network 16) model or a pre-trained RNN (Recurrent Neural Network) model or a pre-trained GAN-based deep neural network model or a pre-trained CNN model, etc., and the embodiments of the present application are not limited to this. It should be understood that the style extraction model may be obtained through supervised learning based on samples and stored in the electronic device.

For example, see Fig. 6a, which is a schematic diagram of a style recognition model recognizing a text track according to an embodiment of the present application. For example, taking the word "霁" written by a user on a touch screen of an electronic device as an example, the recognition process of the style recognition model recognizing a text track is introduced.

Optionally, the electronic device inputs the text track corresponding to the word "霁" written by the user on the touch screen of the electronic device By inputting it into the style recognition model, we can get the style of the word "霁", such as style a.

The following takes the training of the style recognition model as an example to introduce the training process of the style recognition model.

Optionally, the training method may include supervised training, unsupervised training, etc. The embodiment of the present application takes supervised training as an example to introduce the training process of the style recognition model.

First, obtain the training sample set.

For example, the training sample set may include text samples and the label styles of the text samples.

Exemplarily, the label style may be a manual annotation result, which is used to indicate the style to which the text sample belongs. For example, the first text sample in the training sample set is manually annotated, and its manual annotation result is the style corresponding to Kaishu, and the second text sample in the training sample set is manually annotated, and its manual annotation result is the style corresponding to Songti. In this way, each text sample in the training sample set corresponds to a manual annotation result.

Furthermore, the first text sample in the training sample set is input into the initial style recognition model to obtain the style of the first text sample.

Exemplarily, the first text sample is any text sample in the training sample set.

Then, the loss value between the style of the first text sample and the label style of the first text sample is calculated through the loss function.

Exemplarily, after obtaining the loss value, if the loss value is greater than the loss threshold, back propagation is performed to update the initial style recognition model. Then, the updated style recognition model recalculates the loss value between the style of the second text sample and the label style of the second text sample through the loss function. The second text sample is any other text sample in the training sample set except the first text sample. If the loss value calculated again is less than or equal to the loss threshold, the training is stopped, otherwise the back propagation is continued to continue updating until the loss value between the style of the text sample obtained by the updated style recognition model and the label style of the text sample is less than or equal to the loss threshold.

Step 503: Obtain the second style of the target font.

Exemplarily, in a specific implementation process, the electronic device may obtain the second style of the target font in the following manner.

Method 1: After generating the first text trajectory, the electronic device can select a font that matches the first text trajectory from the font library as the target font. Then, the electronic device can determine the second style of the target font from the style library, or input the text model corresponding to the target font into the style recognition model to obtain the second style of the target font. Optionally, the font library contains at least one font, or may also contain a handwriting feature vector of at least one font. The second style can be used to characterize the font features of the text trajectory of the target font.

Among them, the style library includes multiple fonts and styles of the multiple fonts. Optionally, the style library can be created offline. It should be understood that the style library can be updated regularly. For example, in an implementation method of creating a style library, for any existing font, the electronic device can input the text model corresponding to the font into the style recognition model to obtain the style of the font, and can store the font and the style of the font in the style library.

For example, taking the regular script as an example, the electronic device can input the text model corresponding to the regular script (such as the regular script characters "文", "中" or "书", etc.) into the style recognition model to obtain the style of the regular script.

Based on the above content, the following introduces an implementation method in which the electronic device selects a font matching the first text track in a font library as a target font.

Optionally, the electronic device inputs the first text track into a first handwriting feature extraction model to obtain a first handwriting feature vector of the first text track. Afterwards, the electronic device can calculate the similarity between the first text track and each font in at least one font based on the first handwriting feature vector and the handwriting feature vector of at least one font in the font library, so as to obtain at least one similarity. Optionally, the electronic device can also calculate the first text track by other means. The similarity with each font in at least one font, for example, without extracting the first handwriting feature vector of the first text track, directly calculates the similarity between the first text track and any font in the font library through a similarity matching algorithm. Then, the electronic device can select the greatest similarity from at least one similarity, and can use the font corresponding to the greatest similarity as the target font. Exemplarily, the first handwriting feature extraction model can be a neural network model with memory function, such as an LSTM (Long Short-Term Memory) model.

Exemplarily, the electronic device may use a similarity matching algorithm, such as a Euclidean distance algorithm, a Pearson correlation coefficient algorithm, or a cosine similarity algorithm, to calculate the similarity between the first text track and each font in the at least one font.

For example, taking the cosine similarity algorithm as the similarity matching algorithm, it is assumed that there are three fonts in the font library, namely font a, font b and font c, and it is assumed that the handwriting feature vector of the first text track is f, the handwriting feature vector of font a is a1, the handwriting feature vector of font b is b1, and the handwriting feature vector of font c is c1. The electronic device uses the cosine similarity algorithm to calculate the handwriting feature vector f and the handwriting feature vector a1, and calculate the similarity k1 between the first text track and font a; calculate the handwriting feature vector f and the handwriting feature vector b1, and calculate the similarity k2 between the first text track and font b; calculate the handwriting feature vector f and the handwriting feature vector c1, and calculate the similarity k3 between the first text track and font c. For example, assuming k1＜k3＜k2, the electronic device can determine that the similarity between the first text track and font b is the largest by comparing k1, k2 and k3, so font b can be used as the target font.

Exemplarily, the font library may include multiple fonts and/or handwriting feature vectors of the multiple fonts. Optionally, the font library may be created offline. It should be understood that the font library may be updated in real time or regularly.

For example, in the implementation of creating a font library, for any third text track, the electronic device inputs the third text track into the first handwriting feature extraction model to obtain the second handwriting feature vector of the third text track. Afterwards, the electronic device inputs the second handwriting feature vector into a font identification model (such as a font identification model obtained by training CNN) to obtain the font of the third text track. Then, the electronic device can store the font of the third text track and/or the second handwriting feature vector of the third text track in the font library.

The third text track is generated according to a third operation triggered by any user on the touch screen, such as a user writing one or more characters on the touch screen of the electronic device to form a corresponding text track. It should be understood that the third text track is a track formed by the characters written online in the online mode, and belongs to online handwriting.

Exemplarily, a user writes a word "you" on the touch screen of an electronic device, and the word "you" can form a corresponding text track on the touch screen of the electronic device. Afterwards, the electronic device inputs the text track corresponding to the word "you" into the LSTM model, and a handwriting feature vector of the word "you" can be obtained. Then, the electronic device inputs the handwriting feature vector of the word "you" into the font identification model, and the font of the word "you" can be obtained. Finally, the electronic device can store the font of the word "you" and/or the handwriting feature vector of the word "you" in the font library.

For another example, for any target text model, the electronic device inputs the target text model into the second handwriting feature extraction model to obtain the third handwriting feature vector of the target text model. Afterwards, the electronic device inputs the third handwriting feature vector into the font identification model to obtain the font of the target text model. Then, the electronic device can store the font corresponding to the target text model and/or the third handwriting feature vector of the target text model in the font library. Optionally, the target text model can be a text image in offline mode, which belongs to offline handwriting. Optionally, the target text model can be a text image corresponding to a standard font (such as regular script, Song script, etc.), or it can also be a text image corresponding to a popular font on the Internet, etc., and the embodiments of the present application are not limited to this. Exemplarily, the second handwriting feature extraction model can be a neural network model without a memory function, such as a handwriting feature extraction model obtained by training CNN.

For example, for any offline text image, such as the image of the regular script character “我”, the electronic device inputs the image of the regular script character “我” into the second handwriting feature extraction model, and can obtain the image corresponding to the regular script character “我”. Then, the electronic device inputs the handwriting feature vector corresponding to the image of the regular script character "I" into the font identification model, and the font corresponding to the image of the regular script character "I" can be obtained. Finally, the electronic device can store the font corresponding to the image of the regular script character "I" and/or the handwriting feature vector corresponding to the image of the regular script character "I" in the font library.

Method 2: The electronic device obtains the target font in response to the second operation of the user on the touch screen. Then, the electronic device can determine the second style of the target font from the style library, or input the text model corresponding to the target font into the style recognition model to obtain the second style of the target font.

Based on the above content, the following introduces an implementation method for the electronic device to obtain the target font in response to the second operation of the user on the touch screen.

Optionally, a font selection application for the user to select a target font is installed on the electronic device. When the user's operation on the font selection icon corresponding to the font selection application is detected on the touch screen, the electronic device displays multiple candidate fonts. Afterwards, the user can select a candidate font from the multiple candidate fonts as the target font. For example, the user can select which candidate font to use as the target font according to his or her needs. Then, in response to the user's operation of selecting a candidate font from the multiple candidate fonts, the electronic device uses the candidate font selected by the user as the target font.

For example, when the electronic device detects that the user operates the font selection icon corresponding to the font selection application, multiple candidate fonts may be displayed, such as "Regular Script", "Cursive Script", "Running Script" and "Song Ti". Assuming that the user selects "Cursive Script" according to his needs, the electronic device may use the "Cursive Script" selected by the user as the target font.

Step 504: Generate a third style according to the first style and the second style.

In one implementation, the electronic device may directly fuse the first style with the second style to generate a third style. For example, the electronic device may input the first style and the second style into a feature fusion model to obtain the third style. Exemplarily, the feature fusion model may be a style transfer model based on an attention mechanism or an Attentive Font Morphing module (AFM) based on an attention mechanism.

In another embodiment, a beautification parameter value for adjusting the beautification degree of the text track may be introduced to perform weighted processing on the first style and the second style to generate a third style.

For example, the electronic device can input a preconfigured beautification parameter value (such as a default beautification parameter value), a first style, and a second style into a feature fusion model to obtain a third style, or it can input multiple preconfigured beautification parameter values, a first style, and a second style into a feature fusion model to obtain a third style.

For another example, the electronic device may also obtain one or more beautification parameter values selected by the user in a beautification parameter value selection box or one or more beautification parameter values entered by the user in a beautification parameter value input box, and may input the one or more beautification parameter values selected or entered by the user, the first style, and the second style into a feature fusion model to obtain a third style.

Exemplarily, the weighted processing of the first style and the second style by the electronic device may satisfy the following form.
V _x ′＝V _x1 *α+V _x2 *β

Wherein, V _x ′ represents the third style obtained after weighted processing, V _x1 represents the first style, V _x2 represents the second style, and α and β represent beautification parameter values selected or input by the user.

For example, see Fig. 6b, which is a schematic diagram of generating a third style provided by an embodiment of the present application. For example, taking the word "霁" written by a user on a touch screen of an electronic device as an example, and taking the word model "永" corresponding to the target font as an example, the process of generating the third style is introduced.

Optionally, the electronic device inputs the text track corresponding to the word "霁" written by the user on the touch screen of the electronic device Into the style recognition model, the style of the word "霁" can be obtained, such as style a. In addition, the electronic device inputs the word model "永" corresponding to the target font into the style recognition model to obtain the style of the target font, such as style b. Then, the electronic device inputs a beautification parameter value (such as beautification parameter value 1) for adjusting the beautification degree of the text track, style a, and style b into the style transfer model based on the attention mechanism to obtain a fused style, such as style c.

Step 505: Beautify the first text track according to the third style to obtain a second text track.

Optionally, after generating the first text track, the electronic device may also extract the characters of the first text track through OCR (Optical Character Recognition) technology. Then, after obtaining the third style, the electronic device decodes the third style and the characters of the first text track through a decoder to obtain the beautified first text track, that is, the second text track.

Exemplarily, the decoder may correspond to the style recognition model as the encoder, or the decoder may correspond to the style extraction model or the style recognition model as the encoder. For example, the decoder may be composed of a deconvolution layer, a normalization layer, an activation layer, and the like.

Optionally, after generating the first text track, the electronic device may also input the first text track into a character recognition model (such as a character recognition model obtained by training CNN) to obtain the characters of the first text track. Then, after obtaining the third style, the electronic device decodes the third style and the characters of the first text track through a decoder to obtain the beautified first text track, i.e., the second text track.

For example, see Fig. 6c, which is a schematic diagram of generating a second text track provided by an embodiment of the present application. For example, taking the word "霁" written by a user on the touch screen of an electronic device as an example, the process of generating the second text track is introduced.

For example, the electronic device decodes the characters of the word "霁" and the fused style (such as style c) corresponding to the word "霁" through a decoder to obtain the beautified word "霁".

Step 506: Display a second text track on the touch screen.

Optionally, after obtaining the second text track, the electronic device displays the second text track through the touch screen. For example, taking the text "霁" written by the user on the touch screen of the electronic device as an example, after the electronic device performs beautification processing on the text track corresponding to the text "霁", the beautified text "霁" is obtained. Then, the electronic device can display the beautified text "霁" on the touch screen.

It should be noted that each step involved in the above embodiments can be performed by a corresponding device, or by a chip, processor or chip system in the device, and the embodiments of the present application do not limit them. The above embodiments are only described by taking the corresponding device as an example.

It should be noted that in the above embodiments, some steps can be selected for implementation, and the order of the steps in the diagram can be adjusted for implementation, and this application does not limit this. It should be understood that executing some steps in the diagram, adjusting the order of the steps, or combining them for specific implementation all fall within the scope of protection of this application.

It is understandable that, in order to implement the functions in the above embodiments, the various devices involved in the above embodiments include hardware structures and/or software modules corresponding to the execution of the various functions. It should be easily appreciated by those skilled in the art that, in combination with the units and method steps of the various examples described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

It should be noted that the "steps" in the embodiments of the present application are merely illustrative and are a method of expression used to better understand the embodiments. They do not constitute a substantial limitation on the execution of the solutions of the present application. For example, the "steps" can also be understood as In addition, this step does not constitute any limitation on the execution order of the solution of this application. Any new technical solution formed by any change in the order of steps, merging steps, or splitting steps made on this basis that does not affect the implementation of the overall solution is also within the scope of the disclosure of this application.

Based on the same technical concept, the embodiment of the present application also provides a possible electronic device, which is suitable for the application scenario shown in Figure 1. The electronic device is used to implement the text beautification method provided in the above embodiment, or the module (such as a chip) of the electronic device is used to implement the text beautification method provided in the above embodiment, so the beneficial effects of the above embodiment can also be achieved. In the embodiment of the present application, the electronic device can be an electronic device 200 as shown in Figure 1.

Referring to FIG. 7 , the electronic device 700 includes an acquisition module 701 and a processing module 702. The processing module 702 is used to generate a first text track in response to a first operation of a user on the touch screen; the processing module 702 is used to identify the first text track and obtain a first style of the first text track; wherein the first style is used to characterize the font features of the first text track; the acquisition module 701 is used to obtain a second style of the target font; the second style is used to characterize the font features of the text track of the target font; the processing module 702 is also used to generate a third style based on the first style and the second style; the processing module 702 is also used to beautify the first text track according to the third style to obtain the second text track; the processing module 702 is also used to display the second text track on the touch screen.

For a more detailed description of the acquisition module 701 and the processing module 702, please refer to the relevant description in the above method embodiment, which will not be repeated here.

It should be understood that the acquisition module 701 in the embodiment of the present application can be implemented by a communication interface or a communication interface related circuit component, and the processing module 702 can be implemented by a processor or a processor related circuit component.

It should be noted that the division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

Based on the same concept, the embodiment of the present application also provides a possible electronic device, which is suitable for the application scenario shown in Figure 1. The electronic device is used to implement the text beautification method provided in the above embodiment, and therefore can also achieve the beneficial effects of the above method embodiment. As shown in Figure 8, the electronic device 800 includes: a communication interface 801 and a processor 802. Optionally, the electronic device also includes a memory 803. Among them, the communication interface 801, the processor 802 and the memory 803 are interconnected. When the electronic device 800 is used to implement the method provided in the above embodiment, the communication interface 801 can be used to implement the function of the above acquisition module 701, and the processor 802 is used to implement the function of the above processing module 702.

Optionally, the communication interface 801, the processor 802 and the memory 803 are interconnected via a bus 804. The bus 804 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, FIG8 is represented by only one thick line, but it does not mean that there is only one bus or one type of bus.

The communication interface 801 is used to receive and send data, realize communication with other functional components in the electronic device shown in FIG1, or realize communication with other devices other than the electronic device. Optionally, the communication interface 801 can be an input and output interface. For example, the electronic device can use the communication interface to communicate with a touch screen, or the electronic device can also use the communication interface to communicate with other electronic devices.

The functions of the processor 802 can refer to the description in the above embodiments, and will not be repeated here. Among them, the processor 802 can be a central processing unit (CPU), a network processor (NP) or a combination of CPU and NP, etc. The processor 802 can further include a hardware chip. The above-mentioned hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above-mentioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof. When the processor 802 realizes the above-mentioned functions, it can be realized by hardware, and of course, it can also be realized by executing the corresponding software through hardware.

The memory 803 is used to store program instructions, etc. Specifically, the program instructions may include program codes, and the program codes include computer operation instructions. The memory 803 may include random access memory (RAM), and may also include non-volatile memory (non-volatile memory), such as at least one disk memory. The processor 802 executes the program instructions stored in the memory 803 to implement the above functions, thereby implementing the method provided in the above embodiment.

Based on the same concept, an embodiment of the present application further provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are executed on a computer, the computer executes the method provided in the above embodiment.

Based on the same concept, an embodiment of the present application also provides a computer-readable storage medium, in which a computer program or instruction is stored. When the computer program or instruction is executed by a computer, the computer executes the method provided in the above embodiment.

The storage medium may be any available medium that can be accessed by a computer. For example, but not limited to, a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer.

Based on the same concept, an embodiment of the present application further provides a chip, which is coupled to a memory and is used to read a computer program stored in the memory to implement the method provided in the above embodiment.

Based on the same concept, the embodiment of the present application also provides a chip system, which includes a processor for supporting a computer device to implement the functions involved in the electronic device in the above embodiment. In a possible design, the chip system also includes a memory, which is used to store the necessary programs and data for the computer device. The chip system can be composed of chips, or it can include chips and other discrete devices.

The methods provided in the embodiments of the present application may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, they may be implemented in whole or in part in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a high-density digital video disc (DVD)), or a semiconductor medium (e.g., a solid state drive (SSD)), etc.

The steps of the method described in the embodiments of the present application can be directly embedded in the software unit executed by the hardware, the processor, or a combination of the two. The software unit can be stored in a RAM, ROM, EEPROM, register, hard disk, removable disk, CD-ROM or other storage media of any form in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and the storage medium can be arranged in an ASIC.

These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims (15)

1. A text beautification method, characterized in that it is applied to an electronic device with a touch screen, and the method comprises:

   In response to a first operation of a user on the touch screen, generating a first text track;

   Recognize the first text track to obtain a first style of the first text track; wherein the first style is used to characterize the font features of the first text track;

   Acquire a second style of the target font; the second style is used to characterize the font features of the text track of the target font;

   generating a third style according to the first style and the second style;

   Beautify the first text track according to the third style to obtain a second text track;

   The second text track is displayed on the touch screen.
2. The method according to claim 1, wherein identifying the first text track to obtain the first style of the first text track comprises:

   The first text trajectory is input into a style recognition model to obtain the first style.
3. The method according to claim 1 or 2, characterized in that obtaining the second style of the target font comprises:

   The text model corresponding to the target font is input into a style recognition model to obtain the second style; or the second style of the target font is determined from a style library.
4. The method according to any one of claims 1 to 3, characterized in that the method further comprises:

   The target font is selected in a font library, wherein the font library contains at least one font; or the target font is acquired in response to a second operation of the user on the touch screen.
5. The method according to claim 4, wherein selecting the target font in a font library comprises:

   Calculating the similarity between the first text track and each font of the at least one font;

   In the font library, a font having the highest similarity to the first text track is selected as the target font.
6. The method according to claim 5, wherein calculating the similarity between the first text trajectory and each font of the at least one font comprises:

   Inputting the first text trajectory into a first handwriting feature extraction model to obtain a first handwriting feature vector of the first text trajectory;

   The similarity between the first text track and each font in the at least one font is calculated according to the first handwriting feature vector and the handwriting feature vector of the at least one font.
7. The method according to any one of claims 4 to 6, characterized in that the method further comprises:

   A first font is determined, and the determined first font is updated to a font library.
8. The method of claim 7, wherein determining the first font comprises:

   Inputting a third text track into the first handwriting feature extraction model to obtain a second handwriting feature vector of the third text track, and inputting the second handwriting feature vector into a font identification model to obtain that the font of the third text track is the first font; wherein the third text track is generated according to a third operation triggered by any user on the touch screen; or

   The target text model is input into the second handwriting feature extraction model to obtain a third handwriting feature vector of the target text model, and the third handwriting feature vector is input into the font identification model to obtain that the font corresponding to the target text model is the first font.
9. The method according to claim 4, wherein, in response to the second operation of the user on the touch screen, obtaining the target font comprises:

   When detecting that the user operates a font selection icon on the touch screen, displaying a plurality of candidate fonts;

   In response to the second operation of the user selecting any one candidate font from the plurality of candidate fonts, the candidate font selected by the user is used as the target font.
10. The method according to any one of claims 1 to 9, characterized in that generating a third style according to the first style and the second style comprises:

    Obtaining a beautification parameter value for adjusting the beautification degree of the first text track;

    The beautification parameter value, the first style, and the second style are input into a feature fusion model to obtain the third style.
11. An electronic device, characterized in that it comprises a module for executing the method as claimed in any one of claims 1 to 10.
12. An electronic device, characterized in that it comprises a memory and a processor;

    The memory is used to store computer program instructions;

    The processor is used to execute and call the computer program instructions in the memory so that the electronic device performs the operating steps of the method as described in any one of claims 1-10.
13. A computer-readable storage medium, characterized in that a computer program or instruction is stored in the computer-readable storage medium, and when the computer program or instruction is executed by a computer, the computer executes the operating steps of the method according to any one of claims 1 to 10.
14. A computer program product, characterized in that the computer program product comprises a computer program or instructions, and when the computer program or instructions are run on a computer, the computer is caused to execute the operating steps of the method according to any one of claims 1 to 10.
15. A chip, characterized in that the chip is coupled to a memory, and the chip reads a computer program stored in the memory to execute the operating steps of the method according to any one of claims 1 to 10.