WO2023097446A1

WO2023097446A1 - Video processing method, smart terminal, and storage medium

Info

Publication number: WO2023097446A1
Application number: PCT/CN2021/134410
Authority: WO
Inventors: 冯宇傲
Original assignee: 深圳传音控股股份有限公司
Priority date: 2021-11-30
Filing date: 2021-11-30
Publication date: 2023-06-08

Abstract

The present application provides a video processing method, a smart terminal, and a storage medium. The video processing method comprises: obtaining first information corresponding to at least one video image frame in a target video; and processing the first information according to a preset rule to determine or generate target information corresponding to the video image frame. In the present application, the target information may be obtained by processing the first information in the video image frame, thereby improving the video resource utilization rate and improving user experience.

Description

Video processing method, intelligent terminal and storage medium

technical field

The present application relates to the technical field of video processing, and in particular to a video processing method, an intelligent terminal and a storage medium.

Background technique

In some implementations, in order to facilitate the understanding of video content, generally in the video, a subtitle display function is supported, and some also support a subtitle translation function to help users better understand the video content. Preset information (such as text information, etc.) in different languages may appear in the video image, which may be difficult for the user to understand, thereby affecting the user's viewing experience.

technical solution

The main purpose of this application is to provide a video processing method, an intelligent terminal and a storage medium, so that users can quickly understand the preset information in the video according to the converted target information.

In order to achieve the above object, the application provides a video processing method, the video processing method includes the following steps:

Obtain first information corresponding to at least one video image frame in the target video;

The first information is processed according to a preset rule to determine or generate target information corresponding to the video image frame.

Optionally, the step of processing the first information according to preset rules to determine or generate target information corresponding to the video image frame includes:

identifying the scene information of the video image frame, and determining or generating feature information according to the scene information;

The target information is determined or generated according to the characteristic information and the first information.

Optionally, the method also includes:

Obtaining second information corresponding to the voice information in the target video;

Performing a first preset process on the first information according to the second information to determine or generate the target information.

Optionally, the step of acquiring the second information corresponding to the voice information in the target video includes:

Acquiring initial information corresponding to the voice information;

performing a second preset process on the initial information according to the first information to obtain the second information.

Optionally, the first information includes first content and/or a first location, and the step of processing the first information according to preset rules includes:

Acquiring descriptive information and/or associated information for the first content in the second information;

Determine or generate identification information corresponding to the first content according to the description information and/or the association information;

Identifying the first content according to the identification information to determine or generate the target content.

Optionally, the step of processing the first information according to preset rules includes:

When the language type corresponding to the first content does not match the preset language type, convert the first content into the target content corresponding to the preset language type; and/or determine or generate the target content according to the preset position rule The target location corresponding to the first location.

Optionally, the preset location rules include at least one of the following:

taking a position separated by a preset distance from the first position as a target position;

In response to a preset operation within or outside of a first location, the target location is determined or generated based on the preset operation.

Optionally, the preset operation may be a drag operation, or other operations.

Optionally, after the step of processing the first information according to preset rules, the method further includes:

Determine or generate target information according to the target content and/or target position, and associate the target information with the video image frame; and/or, according to the target content, the time stamp of the video image frame and/or the target The location is determined or the target information is generated.

Optionally, the method also includes:

When detecting the playback request of the target video, acquiring the target information;

During the playing of the target video, the target content is displayed at the target position corresponding to the video image frame.

Optionally, when the play request of the target video is detected, the step of acquiring the target information includes:

Obtain the target information according to the association relationship between the video image frame in the target video and the target information, and/or, when a playback request of the target video is detected, according to the correspondence between the playback time point and the timestamp in the target information The relationship acquires the target information.

Optionally, the step of displaying the target content at the position corresponding to the target image frame during the playback of the target video includes:

Acquiring target display parameters of the target information;

During the playing process of the target video, the target content is displayed at the target position corresponding to the video image frame with the target display parameters.

Optionally, the step of acquiring target display parameters of the target information includes:

Output a selection interface;

In response to a trigger selection operation on the selection interface, the target display parameter is determined according to the selection operation.

Optionally, the target display parameters include at least one of text display duration, text display mode, and text target display position. Optionally, the text display manner includes displaying the target content, and/or simultaneously displaying the target content and the content.

Optionally, the step of acquiring first information corresponding to at least one video image frame in the target video includes:

judging whether preset information exists in the video image frame;

If yes, acquire the first information from the video image frame.

Optionally, the preset information may be text information, may also be subtitle information, and may also be content text in an image frame, and the like.

The present application also provides a video processing method, the method comprising:

Acquiring first information corresponding to at least one video image frame in the target video and acquiring second information corresponding to voice information in the target video;

Determine or generate target information corresponding to the target video according to the first information and the second information.

Optionally, the target information includes image text information and/or voice text information.

Optionally, the step of determining or generating target information corresponding to the target video according to the first information and the second information includes:

Performing third preset processing on the first information according to the second information to obtain image text information, and/or performing fourth preset processing on the second information according to the first information to obtain voice text messages;

The target information is determined or generated according to the image text information and/or the voice text information.

Optionally, the third preset processing may be calibration processing or other processing; the fourth preset processing may be calibration processing or other processing.

Optionally, the step of performing third preset processing on the first information according to the second information to obtain processed image text information includes:

Acquiring the content to be processed in the first information according to a preset acquisition rule;

Determine the voice processing content corresponding to the content to be processed in the second information;

Performing a third preset process on the content to be processed according to the voice processing content, so as to determine or generate the image text information.

Optionally, the step of performing third preset processing on the first information according to the second information to obtain image text information includes: obtaining descriptive information and/or related information;

Determine or generate identification information corresponding to the first content according to the description information and/or associated information;

Identifying the first content according to the identification information to determine or generate the image text information.

Optionally, the step of performing fourth preset processing on the second information according to the first information to obtain voice text information includes:

judging whether the voice information corresponds to the first information;

If yes, perform a fourth preset process on the second information according to the first information, so as to obtain voice-to-text information.

The present application also provides an intelligent terminal, including: a memory and a processor, wherein a video processing program is stored in the memory, and when the video processing program is executed by the processor, the steps of any one of the methods described above are implemented.

The present application also provides a storage medium, the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the above methods are implemented.

Optionally, the storage medium may be a computer-readable storage medium.

In the present application, the first information corresponding to at least one video image frame in the target video is obtained; and the first information is processed according to preset rules to determine or generate the target information corresponding to the video image frame. Through the above technical solution, it can be realized that when playing a video, the first information corresponding to the video image frame is processed as the target information, and the user can quickly understand the original preset information (such as the original text) according to the target information, which solves the problem that the user is difficult to understand the original preset information. Set information problems, thereby improving the user experience.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description serve to explain the principles of the application. In order to more clearly illustrate the technical solutions of the embodiments of the present application, the accompanying drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, for those of ordinary skill in the art, the Under the premise, other drawings can also be obtained based on these drawings.

FIG. 1 is a schematic diagram of a hardware structure of an intelligent terminal implementing various embodiments of the present application;

FIG. 2 is a system architecture diagram of a communication network provided by an embodiment of the present application;

Fig. 3 is a schematic diagram of the hardware structure of the controller 140 shown according to the first embodiment;

Fig. 4 is a schematic diagram of a hardware structure of a network node 150 shown according to the first embodiment;

FIG. 5 is a schematic diagram of a hardware structure of a network node 160 shown according to the first embodiment;

FIG. 6 is a schematic diagram of the hardware structure of the controller 170 shown according to the second embodiment;

FIG. 7 is a schematic diagram of a hardware structure of a network node 180 according to a second embodiment;

Fig. 8 is a schematic flowchart of a video processing method according to the first embodiment;

FIG. 9 is a schematic flowchart of step S20 of the video processing method according to the first embodiment;

Fig. 10 is a diagram of the playback interface of the video processing method shown according to the first embodiment;

FIG. 11 is a schematic flowchart of step S20 of the video processing method according to the second embodiment;

Fig. 12 is a schematic flowchart of a video processing method according to a third embodiment;

Fig. 13 is a schematic flowchart of a video processing method according to a fourth embodiment;

Fig. 14 is a schematic flowchart of step S20 of the video processing method according to the fifth embodiment.

Fig. 15 is a schematic flowchart of a video processing method according to a sixth embodiment;

Fig. 16 is a schematic diagram of an image frame according to a sixth embodiment;

Fig. 17 is a schematic flowchart of a video processing method according to a seventh embodiment;

Fig. 18 is a schematic flowchart of a video processing method according to a seventh embodiment;

FIG. 19 is a schematic flowchart of a video processing method S70 according to the eighth embodiment;

Fig. 20 is a schematic flowchart of a video processing method according to a ninth embodiment.

The realization, functional features and advantages of the present application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. By means of the above drawings, specific embodiments of the present application have been shown, which will be described in more detail hereinafter. These drawings and text descriptions are not intended to limit the scope of the concept of the application in any way, but to illustrate the concept of the application for those skilled in the art by referring to specific embodiments.

Embodiment of this application

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present application as recited in the appended claims.

It should be noted that, in this document, the term "comprising", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or device. Without further limitations, an element defined by the statement "comprising a..." does not exclude the existence of other identical elements in the process, method, article or device that includes the element. Optionally, the present application Components, features, and elements with the same name in different embodiments may have the same meaning, or may have different meanings, and the specific meaning shall be determined based on the explanation in the specific embodiment or further combined with the context in the specific embodiment.

It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this document, first information may also be called second information, and similarly, second information may also be called first information. Depending on the context, the word "if" as used herein may be interpreted as "at" or "when" or "in response to a determination". Furthermore, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context indicates otherwise. It should be further understood that the terms "comprising", "comprising" indicate the presence of stated features, steps, operations, elements, components, items, species, and/or groups, but do not exclude one or more other features, steps, operations, The existence, occurrence or addition of an element, component, item, species, and/or group. The terms "or", "and/or", "comprising at least one of" and the like used in this application may be interpreted as inclusive, or mean any one or any combination. For example, "including at least one of the following: A, B, C" means "any of the following: A; B; C; A and B; A and C; B and C; A and B and C", another example, " A, B or C" or "A, B and/or C" means "any of the following: A; B; C; A and B; A and C; B and C; A and B and C". Exceptions to this definition will only arise when combinations of elements, functions, steps or operations are inherently mutually exclusive in some way.

It should be understood that although the various steps in the flow chart in the embodiment of the present application are displayed sequentially as indicated by the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figure may include multiple sub-steps or multiple stages, these sub-steps or stages are not necessarily executed at the same time, but may be executed at different times, and the execution order is not necessarily sequential Instead, it may be performed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

Depending on the context, the words "if", "if" as used herein may be interpreted as "at" or "when" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrases "if determined" or "if detected (the stated condition or event)" could be interpreted as "when determined" or "in response to the determination" or "when detected (the stated condition or event) )" or "in response to detection of (a stated condition or event)".

It should be noted that, in this article, step codes such as S10 and S20 are used, the purpose of which is to express the corresponding content more clearly and concisely, and does not constitute a substantive limitation on the order. Those skilled in the art may, during specific implementation, S20 will be executed first, followed by S10, etc., but these should be within the scope of protection of this application.

It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

In the following description, the use of suffixes such as 'module', 'part' or 'unit' for denoting elements is only for facilitating the description of the present application and has no specific meaning by itself. Therefore, 'module', 'part' or 'unit' may be mixedly used.

Smart terminals can be implemented in various forms. For example, the smart terminals described in this application may include mobile phones, tablet computers, notebook computers, palmtop computers, personal digital assistants (Personal Digital Assistant, PDA), portable media players (Portable Media Player, PMP), navigation devices, Smart terminals such as wearable devices, smart bracelets, and pedometers, as well as fixed terminals such as digital TVs and desktop computers.

In the subsequent description, a mobile terminal will be taken as an example, and those skilled in the art will understand that, in addition to elements specially used for mobile purposes, the configurations according to the embodiments of the present application can also be applied to fixed-type terminals.

Please refer to FIG. 1 , which is a schematic diagram of the hardware structure of a mobile terminal implementing various embodiments of the present application. The mobile terminal 100 may include: an RF (Radio Frequency, radio frequency) unit 101, a WiFi module 102, an audio output unit 103, an A /V (audio/video) input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, processor 110, and power supply 111 and other components. Those skilled in the art can understand that the structure of the mobile terminal shown in Figure 1 does not constitute a limitation on the mobile terminal, and the mobile terminal may include more or less components than those shown in the figure, or combine some components, or different components layout. Each component of the mobile terminal is specifically introduced below in combination with FIG. 1:

The radio frequency unit 101 can be used for sending and receiving information or receiving and sending signals during a call. Specifically, after receiving the downlink information of the base station, it is processed by the processor 110; in addition, the uplink data is sent to the base station. Generally, the radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency unit 101 can also communicate with the network and other devices through wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication, Global System for Mobile Communications), GPRS (General Packet Radio Service, General Packet Radio Service), CDMA2000 (Code Division Multiple Access 2000 , Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access, Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access, Time Division Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term Evolution, frequency division duplex long-term evolution), TDD-LTE (Time Division Duplexing-Long Term Evolution, time-division duplex long-term evolution) and 5G, etc.

WiFi is a short-distance wireless transmission technology. The mobile terminal can help users send and receive emails, browse web pages, and access streaming media through the WiFi module 102, which provides users with wireless broadband Internet access. Although Fig. 1 shows the WiFi module 102, it can be understood that it is not an essential component of the mobile terminal, and can be completely omitted as required without changing the essence of the invention.

The audio output unit 103 can store the audio received by the radio frequency unit 101 or the WiFi module 102 or stored in the memory 109 when the mobile terminal 100 is in a call signal receiving mode, a call mode, a recording mode, a voice recognition mode, a broadcast receiving mode, or the like. The audio data is converted into an audio signal and output as sound. Also, the audio output unit 103 can also provide audio output related to a specific function performed by the mobile terminal 100 (eg, call signal reception sound, message reception sound, etc.). The audio output unit 103 may include a speaker, a buzzer, and the like.

The A/V input unit 104 is used to receive audio or video signals. The A/V input unit 104 may include a graphics processing unit (Graphics Processing Unit, GPU) 1041 and a microphone 1042, and the graphics processing unit 1041 is used for still pictures or The image data of the video is processed. The processed image frames may be displayed on the display unit 106 . The image frames processed by the graphics processor 1041 may be stored in the memory 109 (or other storage media) or sent via the radio frequency unit 101 or the WiFi module 102 . The microphone 1042 can receive sound (audio data) via the microphone 1042 in a phone call mode, a recording mode, a voice recognition mode, and the like operating modes, and can process such sound as audio data. The processed audio (voice) data can be converted into a format transmittable to a mobile communication base station via the radio frequency unit 101 for output in case of a phone call mode. The microphone 1042 may implement various types of noise cancellation (or suppression) algorithms to cancel (or suppress) noise or interference generated in the process of receiving and transmitting audio signals.

The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, a motion sensor, and other sensors. Optionally, the light sensor includes an ambient light sensor and a proximity sensor. Optionally, the ambient light sensor can adjust the brightness of the display panel 1061 according to the brightness of the ambient light, and the proximity sensor can turn off the display when the mobile terminal 100 moves to the ear. panel 1061 and/or backlight. As a kind of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when it is stationary, and can be used for applications that recognize the posture of mobile phones (such as horizontal and vertical screen switching, related Games, magnetometer attitude calibration), vibration recognition related functions (such as pedometer, tap), etc.; as for mobile phones, fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, Other sensors such as thermometers and infrared sensors will not be described in detail here. The display unit 106 is used to display information input by the user or information provided to the user. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display (Liquid Crystal Display, LCD), an organic light-emitting diode (Organic Light-Emitting Diode, OLED), or the like.

The user input unit 107 can be used to receive input numbers or character information, and generate key signal input related to user settings and function control of the mobile terminal. Optionally, the user input unit 107 may include a touch panel 1071 and other input devices 1072 . The touch panel 1071, also referred to as a touch screen, can collect touch operations of the user on or near it (for example, the user uses any suitable object or accessory such as a finger or a stylus on the touch panel 1071 or near the touch panel 1071). operation), and drive the corresponding connection device according to the preset program. The touch panel 1071 may include two parts, a touch detection device and a touch controller. Optionally, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device and converts it into contact coordinates , and then sent to the processor 110, and can receive the command sent by the processor 110 and execute it. In addition, the touch panel 1071 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1071 , the user input unit 107 may also include other input devices 1072 . Optionally, other input devices 1072 may include, but are not limited to, one or more of physical keyboards, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, joysticks, etc., which are not specifically described here. limited.

Optionally, the touch panel 1071 may cover the display panel 1061. When the touch panel 1071 detects a touch operation on or near it, it transmits to the processor 110 to determine the type of the touch event, and then the processor 110 determines the touch event according to the touch event. The corresponding visual output is provided on the display panel 1061 . Although in FIG. 1, the touch panel 1071 and the display panel 1061 are used as two independent components to realize the input and output functions of the mobile terminal, in some embodiments, the touch panel 1071 and the display panel 1061 can be integrated. The implementation of the input and output functions of the mobile terminal is not specifically limited here.

The interface unit 108 serves as an interface through which at least one external device can be connected with the mobile terminal 100 . For example, an external device may include a wired or wireless headset port, an external power (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, audio input/output (I/O) ports, video I/O ports, headphone ports, and more. The interface unit 108 can be used to receive input from an external device (for example, data information, power, etc.) transfer data between devices.

The memory 109 can be used to store software programs as well as various data. The memory 109 can mainly include a storage program area and a storage data area. Optionally, the storage program area can store an operating system, at least one function required application program (such as a sound playback function, an image playback function, etc.) etc.; the storage data area can be Store data (such as audio data, phone book, etc.) created according to the use of the mobile phone. In addition, the memory 109 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices.

The processor 110 is the control center of the mobile terminal, and uses various interfaces and lines to connect various parts of the entire mobile terminal, by running or executing software programs and/or modules stored in the memory 109, and calling data stored in the memory 109 , execute various functions of the mobile terminal and process data, so as to monitor the mobile terminal as a whole. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor. Optionally, the application processor mainly processes operating systems, user interfaces, and application programs, etc. The demodulation processor mainly handles wireless communication. It can be understood that the foregoing modem processor may not be integrated into the processor 110 .

The mobile terminal 100 can also include a power supply 111 (such as a battery) for supplying power to various components. Preferably, the power supply 111 can be logically connected to the processor 110 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. and other functions. Although not shown in FIG. 1 , the mobile terminal 100 may also include a Bluetooth module, etc., which will not be repeated here.

In order to facilitate understanding of the embodiments of the present application, the following describes the communication network system on which the mobile terminal of the present application is based.

Please refer to FIG. 2. FIG. 2 is a structure diagram of a communication network system provided by an embodiment of the present application. The communication network system is an LTE system of general mobile communication technology. ) 201, E-UTRAN (Evolved UMTS Terrestrial Radio Access Network, Evolved UMTS Terrestrial Radio Access Network) 202, EPC (Evolved Packet Core, Evolved Packet Core Network) 203 and the operator's IP service 204.

Optionally, the UE 201 may be the above-mentioned terminal 100, which will not be repeated here.

E-UTRAN 202 includes eNodeB 2021 and other eNodeB 2022 and so on. Optionally, the eNodeB 2021 can be connected to other eNodeB 2022 through a backhaul (for example, X2 interface), the eNodeB 2021 is connected to the EPC 203 , and the eNodeB 2021 can provide access from the UE 201 to the EPC 203 .

EPC203 may include MME (Mobility Management Entity, Mobility Management Entity) 2031, HSS (Home Subscriber Server, Home Subscriber Server) 2032, other MME2033, SGW (Serving Gate Way, Serving Gateway) 2034, PGW (PDN Gate Way, packet data Network Gateway) 2035 and PCRF (Policy and Charging Rules Function, Policy and Charging Functional Entity) 2036, etc. Optionally, MME2031 is a control node that processes signaling between UE201 and EPC203, and provides bearer and connection management. HSS2032 is used to provide some registers to manage functions such as home location register (not shown in the figure), and store some user-specific information about service characteristics, data rate, etc. All user data can be sent through SGW2034, PGW2035 can provide UE 201 IP address allocation and other functions, PCRF2036 is the policy and charging control policy decision point of service data flow and IP bearer resources, it is the policy and charging execution function A unit (not shown) selects and provides available policy and charging control decisions.

The IP service 204 may include Internet, Intranet, IMS (IP Multimedia Subsystem, IP Multimedia Subsystem) or other IP services.

Although the LTE system is used as an example above, those skilled in the art should know that this application is not only applicable to the LTE system, but also applicable to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA and future new wireless communication systems. The network system (such as 5G), etc., is not limited here.

FIG. 3 is a schematic diagram of a hardware structure of a controller 140 provided in the present application. The controller 140 includes: a memory 1401 and a processor 1402, the memory 1401 is used to store program instructions, and the processor 1402 is used to call the program instructions in the memory 1401 to execute the steps performed by the controller in the first method embodiment above, and its implementation principle and beneficial effects are similar, and will not be repeated here.

Optionally, the foregoing controller further includes a communication interface 1403 , and the communication interface 1403 may be connected to the processor 1402 through a bus 1404 . The processor 1402 can control the communication interface 1403 to implement the receiving and sending functions of the controller 140 .

FIG. 4 is a schematic diagram of a hardware structure of a network node 150 provided in the present application. The network node 150 includes: a memory 1501 and a processor 1502, the memory 1501 is used to store program instructions, and the processor 1502 is used to call the program instructions in the memory 1501 to execute the steps performed by the first node in the first method embodiment above, and its implementation principle and beneficial effects are similar, and will not be repeated here.

Optionally, the foregoing controller further includes a communication interface 1503 , and the communication interface 1503 may be connected to the processor 1502 through a bus 1504 . The processor 1502 can control the communication interface 1503 to realize the functions of receiving and sending of the network node 150 .

FIG. 5 is a schematic diagram of a hardware structure of a network node 160 provided in the present application. The network node 160 includes: a memory 1601 and a processor 1602, the memory 1601 is used to store program instructions, and the processor 1602 is used to call the program instructions in the memory 1601 to execute the steps performed by the intermediate node and the tail node in the first method embodiment above, The implementation principles and beneficial effects are similar, and will not be repeated here. Optionally, the foregoing controller further includes a communication interface 1603 , and the communication interface 1603 may be connected to the processor 1602 through a bus 1604 . The processor 1602 can control the communication interface 1603 to realize the functions of receiving and sending of the network node 160 .

FIG. 6 is a schematic diagram of a hardware structure of a controller 170 provided in the present application. The controller 170 includes: a memory 1701 and a processor 1702, the memory 1701 is used to store program instructions, and the processor 1702 is used to call the program instructions in the memory 1701 to execute the steps performed by the controller in the second method embodiment above, and its implementation principle and beneficial effects are similar, and will not be repeated here.

FIG. 7 is a schematic diagram of a hardware structure of a network node 180 provided in the present application. The network node 180 includes: a memory 1801 and a processor 1802, the memory 1801 is used to store program instructions, and the processor 1802 is used to invoke the program instructions in the memory 1801 to execute the steps performed by the head node in the second method embodiment above, and its implementation principle and beneficial effects are similar, and will not be repeated here.

The above-mentioned integrated modules implemented in the form of software function modules can be stored in a computer-readable storage medium. The above-mentioned software function modules are stored in a storage medium, and include several instructions to enable a computer device (which may be a personal computer, server, or network device, etc.) or a processor (English: processor) to execute the methods of the various embodiments of the present application. partial steps.

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. A computer can be a general purpose computer, special purpose computer, computer network, or other programmable device. Computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, e.g. Coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server, a data center, etc. integrated with one or more available media. Available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, solid state disk, SSD), etc.

Based on the above mobile terminal hardware structure and communication network system, various embodiments of the present application are proposed.

first embodiment

Please refer to FIG. 8 . FIG. 8 is a schematic flowchart of a video processing method according to a first embodiment. The methods include:

S10. Obtain first information corresponding to at least one video image frame in the target video;

S20. Process the first information according to a preset rule to determine or generate target information corresponding to the video image frame.

In this embodiment of the application, the execution body of the video processing method of this embodiment may be a smart terminal and/or a server, and the smart terminal may be the smart terminal in the above embodiment. Optionally, the smart terminal may be a smart phone, tablet, computer etc.

Optionally, the target video may be stored in the smart terminal in advance, or may be a video loaded in real time through a video playing application. Optionally, the method of triggering the video processing of the target video may be that when the user actively triggers a playback request for the target video, the user actively triggers a video processing instruction, and the smart terminal calls the target video, and performs the processing on the target video. Executing video processing on the target video; optionally, the method of triggering video processing on the target video may also be to trigger a video processing instruction when the target video is stored in the smart terminal, and then perform video processing on the target video .

Optionally, the target video is composed of at least one video image frame, the video image frame includes a video image frame containing preset information, and the video image frame further includes a video image frame not containing preset information. Optionally, before identifying the first information in the video image frame of the target video, a video image frame containing preset information may be determined from each video image frame of the target video, and then according to the The video image frame acquires the first information.

Optionally, the step S10 includes: judging whether preset information exists in the video image frame; if yes, acquiring the first information from the video image frame.

Optionally, the specific manner of determining the video image frame containing preset information from each image frame of the target video may be that the smart terminal starts from the initial image frame of the video image frame and identifies The content of each video image frame, and then judge whether the video image frame has preset information according to the content of the video image frame. Optionally, when it is recognized that the video image frame has preset information, the video image frame is determined as the video image frame containing the preset information, and then the information of the video image frame containing the preset information is obtained , using the information as the first information. By identifying the information of the video image frames containing the preset information in the target video, video processing is performed only on the video image frames containing the preset information, and the efficiency of video processing is improved.

Optionally, after the video image frame containing preset information is acquired, there may be other video image frames containing information that contain the same information. Optionally, after obtaining the video image frames containing preset information, the present application performs a similarity analysis on each of the video image frames containing preset information, so as to obtain the similarity between each video image frame, and then Similar image frames are determined based on the similarity, and then the similar image frames are screened to obtain video image frames to be processed. Optionally, the higher the similarity between the video image frames, the higher the probability that the information contained in the video image frames is consistent, and the lower the similarity between the video image frames, it means that the video image frames The information contained in is different information.

Optionally, the method of obtaining the video image frame to be processed may also be to first extract the first image frame containing preset information in the video frame, and then obtain each of the video frames based on the extracted first image frame. The degree of similarity between the first image frames, further determining similar image frames corresponding to the first image frames based on the similarity, and further performing screening processing based on the similar image frames to obtain video image frames. Optionally, this embodiment is based on first determining the first image frame containing preset information, and then filtering the first image frame again to obtain fewer video image frames, reducing video processing resources, and further improving improve the efficiency of video processing.

Optionally, after the acquisition of the video image frame containing preset information, the manner of acquiring the first information from the video image frame may be: based on an optical character recognition (OCR) algorithm from the video image The frame acquires the first information.

Optionally, the OCR algorithm is used to translate the shape into computer text through a character recognition method, that is, to recognize information in the image frame and generate a corresponding text recognition result.

Optionally, after the video image frames are obtained, OCR recognition is performed on each of the video image frames to obtain an image text recognition result of the image text information of the video image frames. Optionally, the text displayed on the video image frame is "the sun is shining today", and after performing OCR recognition on the video image frame, the obtained text is an image text recognition result of "the sun is shining today".

Optionally, the first information includes first content and/or a first location. Optionally, the first content is text content in a video image frame, such as "it is sunny today." Optionally, the first location It is the display position of the text in the video image frame.

Optionally, in the process of performing OCR recognition on the video image frame, while obtaining the image text recognition result (text), the display position of the text in the video image frame is obtained, and the display position is used as the image text A character position of the recognition result, and then determine the character position as the first position.

Optionally, after the first information of the video image frame is acquired, the language type of the first content or the first location of the first information may not meet the needs of the user. Optionally, the language type of the first content may be German, but the user cannot read German but only Chinese, then the first content does not meet the needs of the user, or the first location is in edge of the video image frame, but the user cannot see the edge clearly, then the first position does not meet the needs of the user. Optionally, in this embodiment of the present application, after the first information is acquired, the first information is processed according to a preset rule, so as to process the first information into target information meeting user needs.

Optionally, the target information includes target content and/or target location, and the preset rules include preset language type conversion rules and/or preset location rules, and the preset language type conversion rules are used to convert the For the language type corresponding to the first content, the preset position rule is used to adjust the first position.

Optionally, referring to FIG. 9 , the S20 includes: S21, when the language type corresponding to the first content does not match the preset type, converting the first content into a target corresponding to the preset type content, and/or, determine or generate a target location corresponding to the first location according to preset location rules.

Optionally, it is determined whether the language type corresponding to the first content is a preset language type. Optionally, the preset language type may be a system language, and/or the preset language type may be a set language. Optionally, the set language can be configured by the user. The user can modify the system language of the smart terminal by himself, or the user can modify the language setting page of the video playback page by himself, or the user can Inputting language control instructions to the smart terminal. Optionally, after receiving the language control instruction, the smart terminal determines a preset language type according to the language control instruction. Optionally, a way for the user to input voice control instructions to the smart terminal may be voice input, and optionally, the preset language types may include at least one language type.

Optionally, after acquiring the language type corresponding to the first content, it is judged whether the language type matches the preset language type, and when the language type matches the preset language type, there is no need to convert the the language type of the first content; when the language type does not match the preset language type, convert the first content into the target content corresponding to the preset language type. Optionally, the first content is "London Road", the language type is English, the preset language type is Chinese, and the target content corresponding to converting the first content into the preset language type is "London Road".

Optionally, the first text content is converted into the target content, so that when the target video is played later, the target content is displayed at the first text position corresponding to the first information in the video image frame . Optionally referring to Figure 10, the left side of Figure 10 is the image frame before video processing, the right side of Figure 10 is the image frame after video processing, and the image frame on the left side of Figure 5 has a road sign "London Road", after video processing , "London Road" is displayed at the original position of the image frame on the left side of Fig. 10 .

Optionally, the embodiment of the present application also proposes a method of determining or generating a target position corresponding to the first position according to a preset position rule, where the preset position rule includes at least one of the following:

Optionally, the target position may be the same as the first position, or may be different from the first position.

Optionally, the target location may be the same as the first location, and optionally, the preset location rule includes determining the first location as the target location.

Optionally, the target position may be different from the first position, and the preset position rule includes taking the first position as an origin, obtaining a position separated from the first position by a preset distance, and setting the location as the target location. Optionally, the preset distance may be a user-defined setting, and the target position may be directly above the first position, may be to the left of the first position, or may be directly below the first position, Also to the right of said first position.

Optionally, the preset position rule may further include receiving a user's preset operation on the first content, and determining the target location according to the preset operation. Optionally, the user may perform a preset operation within or outside the first location based on the first content. Optionally, the preset position rule also includes acquiring the distance between the first position and important information of the video image frame, the important information may be a person in the video image frame, or the video image frame When the subtitle information of the image frame is too close to the important information, the important information may be blocked or the important information may be confused with the target information. Optionally, when the distance is less than a preset distance, determine a position whose distance from the important information is greater than or equal to a preset distance, and use the position as the target position.

Optionally, the preset position rule may also include obtaining the distance between the first position and the edge side of the video image frame, and obtaining the distance from the edge side when the distance is less than the preset distance A position greater than or equal to a preset distance is used as the target position.

It can be understood that the preset location rules include but are not limited to the above-mentioned several ways.

In the embodiment of the present application, after acquiring the first information corresponding to the video image frame, the first information includes the first content and/or the first location, and the first content is converted into the target corresponding to the preset language type content; and/or, determine or generate the target position corresponding to the first position according to the preset position rules, and when the subsequent user plays the target video, the target information can be obtained directly, and then the target video in the target video The target information is displayed in the video image frame, so that the user can quickly understand the text according to the target information, it is easier to understand the video content, and the user experience is improved, and by adjusting the first position, it is convenient for the user to view the target information, which meets the needs of the user. need. second embodiment

Optionally, referring to FIG. 11 , based on the first embodiment, the S20 includes:

Step S22, identifying the scene information of the video image frame, and determining or generating feature information according to the scene information;

Step S23, determining or generating the target information according to the feature information and the first information.

Optionally, when the user is watching a video, in order to enable the user to quickly obtain part of the content displayed by the video image frame, the embodiment of the present application also proposes a method for automatically marking the first information according to the scene information of the video image frame .

Optionally, the video image frame is a video image frame corresponding to the first information, and the scene information includes the scene type in the video image frame, and the scene type can be indoor, traffic, natural scenery, humanities Landscapes, cities, villages, fields, etc.

Optionally, the specific implementation manner of identifying the scene information of the video image frame includes: acquiring the scene feature information of the video image frame according to a preset feature extraction algorithm, the scene feature information including image display parameter information, object One of the feature information and the environment feature information, compare the scene feature information with the preset scene feature information of the preset scene one by one, and then determine the target preset scene feature information corresponding to the scene feature information, according to the The target preset scene feature information determines the target preset scene corresponding to the target preset scene feature information, and then determines the scene type of the video image frame according to the target preset scene, and then determines the scene type according to the scene type. For scene information, optionally, the preset scene includes several preset scenes, and the feature information of the preset scene information corresponding to different preset scenes is different.

Optionally, after the scene information is acquired, feature information corresponding to the video image frame is generated according to the scene information, the feature information may be the scene information, for example, the scene information is a hospital, and " "Hospital" as the feature information; the feature information may include a scene icon converted according to the scene information, for example, the scene information is a hospital scene, and the hospital icon corresponding to the hospital scene is acquired; the feature information also The target object corresponding to the scene information may be included, and the target object category is a target object in the video image frame, for example, the scene information is a group photo scene, and the video image frame displays person A and person B, That is, person A and person B are the target objects, and then "person A and person B" is determined as the characteristic information.

It can be understood that the manner of determining the feature information includes but not limited to the above three manners. The feature information may also include sequence numbers of the video image frames and the like.

Optionally, after the characteristic information is acquired, the target information is determined or generated according to the characteristic information and the first information.

Optionally, the manner of determining or generating the target information according to the feature information and the first information may be to use the feature information as the tag content of the first information, and then according to the first information and the The target information is determined or generated jointly with the marked content. In the embodiment of the present application, the scene feature information of the video image frame is extracted by a preset feature extraction algorithm, and then the scene type corresponding to the video image frame is determined according to the scene feature information, and then the scene is determined according to the scene type information, and then determine or generate feature information according to the scene information, and then combine the feature information with the first information to generate the target information, so as to combine the first information with the scene type of the video image frame The association is quickly established, so that the user can quickly understand the scene type of the video image frame according to the target information.

third embodiment

Optionally, referring to FIG. 12, based on the first embodiment, the method further includes:

Step S30, acquiring second information corresponding to the voice information in the target video;

Step S40, performing a first preset process on the first information according to the second information, so as to determine or generate target information.

Optionally, the first preset processing may be calibration processing or other processing. Wherein, the calibration processing may be based on voice calibration processing, and may also be text calibration processing.

Optionally, when the first content of the first information is rough, when the first information is recognized based on OCR, the first information may be recognized inaccurately. Optionally, in this embodiment of the present application, the voice information corresponding to each of the video image frames of the target video is obtained, and the first preset processing is performed on the first information according to the voice information, so as to determine or generate the target video. information, and the voice information is in one-to-one correspondence with the video image frames.

Optionally, the target information includes first content in the first information after first preset processing.

Optionally, after the voice information is acquired, the voice information is converted into corresponding second information through voice-to-text technology according to the voice information, and then the first information is performed a second time based on the second information. A preset process, the second information is in one-to-one correspondence with the video image frames.

Optionally, the manner of performing the first preset processing on the first information according to the second information may be to acquire the content to be processed in the first content according to a preset acquisition rule, and then in the second information Determining the voice processing content corresponding to the content to be processed in the second information; performing a first preset process on the content to be processed according to the voice processing content, so as to determine or generate the target content.

Optionally, the preset acquisition rule may be to receive a user's processing instruction for the first content, the processing instruction includes content to be processed, and the preset acquisition rule may also be that the smart terminal intelligently acquires the first content. Pending content. Optionally, the way for the smart terminal to intelligently acquire the content to be processed may be to use the first content as the content to be processed, or to obtain a probability coefficient corresponding to the first content. Optionally, when the probability coefficient is less than or equal to a preset probability coefficient, the first content is used as the content to be processed.

Optionally, after the content to be processed is acquired, the voice processing content corresponding to the content to be processed is determined in the second information.

Optionally, the second information is in one-to-one correspondence with the time stamps of the video image frames. Optionally, after the content to be processed is acquired, the time stamp of the video image frame where the content to be processed is located is acquired, and the second information corresponding to the time stamp is determined according to the time stamp. Stamp the corresponding second information as the voice processing content.

Optionally, the voice processing content is voice-converted content.

Optionally, after acquiring the voice processing content, perform a first preset processing (such as calibration) on the content to be processed according to the voice processing content, so as to obtain the content to be processed after the first preset processing , further determining the first content after the first preset processing according to the content to be processed after the first preset processing, and then determining or Generate said target content. Optionally, the content to be processed includes "Londo Avenue", the speech processing content includes "London Avenue", and then the first content is calibrated as "London Avenue".

Optionally, in yet another embodiment, after the content to be processed is acquired, the video image frame corresponding to the content to be processed is acquired, and then the voice information corresponding to the video image frame is acquired, and the The content to be processed is subjected to a first preset processing (such as calibration). Optionally, the content to be processed includes "Londo Avenue", the voice information includes "London Avenue", and then the to-be-processed content is calibrated as "London Avenue".

Optionally, in yet another embodiment, after the first preset processed first content is acquired, the first information is determined according to the first preset processed first content, and then The first information is processed according to a preset rule to determine or generate target information corresponding to the video image frame.

In this embodiment of the present application, by acquiring the second information corresponding to the voice information in the target video, and then performing the first preset processing on the first content in the first information according to the second information, and then according to the second information The first content after the preset processing determines the first information after the first preset processing, and then processes the first content after the first preset processing according to preset rules to determine or generate The target information corresponding to the video image frame is used to improve the accuracy of image recognition text, thereby improving the accuracy of the target information, so that the user can correctly understand the text according to the target information, it is easier to understand the video content, and the user experience is improved.

Fourth embodiment

Optionally, referring to FIG. 13 , based on the second embodiment, the S30 includes:

S31. Obtain initial information corresponding to the voice information;

S32. Perform a second preset process on the initial information according to the first information to obtain the second information.

Optionally, the preset processing may be calibration processing or other processing.

In this embodiment of the application, when the user is playing the target video, in order to facilitate the user to watch the video, the smart terminal will generate corresponding subtitles according to the voice information corresponding to the target video, and then display the subtitles on the frame of the video image. Below, it is convenient for users to easily understand the voice information of the target video, but there may be dialects in the voice information, or when the voice information includes some professional terms, it is easy to cause the generated subtitles to be inaccurate. Based on this, the embodiment of this application A method for preset processing of subtitles is proposed.

Optionally, the initial information is information converted through speech-to-speech technology according to the voice information, and after the initial information is acquired, preset processing is performed on the initial information according to the first information to obtain the The processed initial information, and then determine the second information according to the processed initial information.

Optionally, the specific implementation manner of performing preset processing on the initial information according to the first information includes: acquiring text to be processed corresponding to the initial information, combining the text to be processed with the first information Match the first content of the first content to obtain the target information in the first content that matches the text to be processed, the similarity between the target information and the text to be processed is greater than or equal to a preset threshold, and the target Information replaces the pending text. Optionally: the text to be processed includes "P to P", the first content includes "P2P", the similarity between "P to P" and "P2P" is 98%, and the preset threshold is 95%, Then prove that the "P2P" is the target information, and then replace the target information with "P to P".

Optionally, after replacing the to-be-processed text with target information, the second information is generated according to the replaced to-be-processed text.

In the embodiment of the present application, the initial information corresponding to the voice information is preset through the first information to obtain accurate second information, so as to improve the accuracy of voice recognition and improve user experience.

fifth embodiment

Optionally, referring to FIG. 14, based on the foregoing embodiment, the S20 includes:

S24. Obtain descriptive information and/or associated information for the first content in the second information;

S25. Determine or generate identification information corresponding to the first content according to the description information and/or associated information;

S26. Identify the first content according to the identification information, so as to determine or generate the target content.

In this embodiment of the present application, the description information is description information used to describe the first content, the description information may include attribute information corresponding to the first content, and the description information may also include the second content Status information corresponding to a content. Optionally, the description information includes but not limited to the attribute information and status information, and the information may also include a description associated with the first content. Optionally, the first content is "London Street", the second information includes attribute information for the "London Street", the attribute information is "London Street is 1 km long", and the "length is 1 km" as the description information. Optionally, the second information includes an adjective associated with the "London Street", for example, "London Street is a food street", and then "a food street" is used as the description information.

Optionally, the description information is associated with the first content.

Optionally, after the description information is acquired, corresponding identification information is generated according to the description information. Optionally, when the description information is "a food street", the identification information may be a food identification; when the description information is "the weather in this area is rainy", the identification information may be a rain identification.

Optionally, the associated information includes attribute information corresponding to the first information, and the attribute information may include language type information, customs and culture, religious culture information, location information, text interpretation information, and the like.

Optionally, after acquiring the association information, corresponding identification information is generated according to the association information. Optionally, the first information is "London Street", the associated information is "The language type of London Street is A, the religious and cultural information is "Christianity", and the description information is "The climate of London Street is rainy" , the generated identification information may be: "voice type is A, religious culture is Christianity, rainy day".

Optionally, when acquiring the description information and/or the association information, the identification information may also be determined according to the description information and the association information.

It can be understood that when the first information appears in the video image frame, if only the first information is translated or the first position corresponding to the first information is adjusted, if the user sees the first information for the first time The content in the information may not be known to the content, so when viewing the video image frame, the user can only view the content, but cannot obtain information related to the content. However, in this embodiment of the present application, the description information for the first content and associated information corresponding to the first content are obtained through the second information, and then the identification information is generated according to the description information and the description information, Furthermore, the identification information identifies the first content, so that the user can view the identification information while viewing the first content, so as to better understand the first content and quickly obtain information related to the first content. Relevant information improves user experience.

In this embodiment of the present application, the description information for the first content and associated information corresponding to the first content are obtained according to the second information, and then the first content is generated according to the description information and the associated information. The identification information corresponding to a content, and then identify the first content according to the identification information, so that when the target video is subsequently played, the identification information is displayed while the first content is displayed, so that the user can update A good understanding of the first content and quick acquisition of information related to the first content improve user experience.

Sixth embodiment

Optionally, referring to FIG. 15 , based on the first embodiment, the video processing method of this application proposes a sixth embodiment. After the S20, also includes:

S50. Determine or generate target information according to the target content and/or the target position, associating the target information with the video image frame; and/or, according to the target content, the time stamp of the video image frame And/or the target location is determined or target information is generated. In this embodiment of the present application, after the first information is processed according to preset rules, the target content and/or the target location is obtained, and target information is determined or generated according to the target content and target location, and Associating the target information with the video image frame, so that in the process of playing the target video, the target information associated with the video image frame is directly acquired, and then the corresponding target position in the video image frame The target content corresponding to the target information is displayed at .

Optionally, the video image frame includes a corresponding time stamp, and the time stamp is used to represent a playback time stamp of the video image frame in the target video. Optionally, the target video is composed of several video image frames, and each video image frame corresponds to a different time stamp. When playing the target video, according to the time stamp corresponding to each of the video image frames, the The large chronological order plays the individual video image frames sequentially. Referring to Figure 16, the left side of Figure 16 is a schematic diagram of each video image frame corresponding to the target video, including "F1, F2, F3, F4, F5, F6, F7, F8, F9, ...", each video The image frames correspond to their respective time stamps, "T1, T2, T3, T4, T5, T6, T7, T8, T9, ...", and the right side of Figure 16 is the corresponding video image frame of the target video schematic diagram.

In this embodiment of the present application, the method of determining or generating target information may also be to obtain the time stamp of the video image frame, and then determine according to the target content, the time stamp of the video image frame and/or the target position Or generate target information for later in the process of playing the target video, when the current playback time point is consistent with the timestamp, acquire the target information, and determine the target content corresponding to the timestamp according to the target information , and then display the target content at the corresponding target position in the video image frame. Optionally, when the playback time point is "01:30", and the time stamp of the video image frame in the target video is also "01:30", then the target information is obtained directly, and then according to The target information determines the target content and/or target position corresponding to "01:30", and then displays the target content at the target position corresponding to the video image frame.

Optionally, the method of determining or generating the target information may also be to generate subtitles according to the target content and the time stamp of the video image frame, so that in the subsequent video playback, the current playback time point reaches the time When stamping, the target information is obtained, and then the target content is displayed at a preset position of the video image frame, and the preset position may be set by the user.

Optionally, the method of determining or generating the target information may also be to determine or generate the target information according to the target content and/or the preset position of the target information, and then combine the target information with the video image frame association, so as to directly acquire the target information associated with the video image frame in subsequent playing of the video, and then display the target content corresponding to the target information at a preset position in the video image frame.

Optionally, each video image frame corresponding to the target video includes its own frame number, and the way of determining or generating the subtitle may also be based on the target content, the frame number of the video image frame and the target position Generate the target information.

Optionally, the manners of determining or generating the target information include but are not limited to the above several manners.

Optionally, after the target information corresponding to the target video is generated, the target information is saved for acquisition when a subsequent user needs to play the target video.

Optionally, the way of saving the target information may be to save the target information and the target video together in the same folder, and the target information is named according to preset rules to ensure that the target video is played. The target information in the video can be loaded normally.

Optionally, the way of saving the target information may also be to compress the video stream and voice stream corresponding to the target video into an audio file, and then package the audio file and the target information.

Optionally, the way of saving the target information may also be to embed the target information in the video image frame in an embedded manner, and during the specific implementation process, the target information is obtained, and the The target information, the video stream, and the audio stream jointly generate a video file of the target video. At this time, the target information is integrated into the coded stream of the video file, and when the target video is played subsequently, it is not necessary to acquire the target information, and the video file can be played directly.

In the embodiment of the present application, by generating the corresponding target information of the target content and/or target location, when the subsequent user plays the target video, the target information can be directly obtained, and then the video in the target video The corresponding target position in the image frame displays the target content, so that the user can quickly understand the text according to the target content, understand the video content more easily, and improve user experience.

Seventh embodiment

Referring to Figure 17, based on all the above-mentioned embodiments, the present application proposes a seventh embodiment, the method further includes:

S60. Acquiring the target information when a play request of the target video is detected;

S70. During playing the target video, display the target content at the target position corresponding to the video image frame.

In this embodiment of the application, when the user initiates a playback request of the target video to the smart terminal, the target information of the target video is obtained according to the playback request, and then the target video is played, and the target video is played according to the target information in the The target content is displayed in the video image frame of the target video.

Optionally, when the target information is stored in such a way that the target information is embedded in the video image frame, the target video may be directly acquired, and then the target video may be played. At this point, the target content of the target information has been presented in the video image frame.

Optionally, when the target information is saved in the same folder as the target information and the target video, when the target video is acquired, the target video is automatically loaded. target information in the same folder, and then display the target content of the target information in the video image frame.

Optionally, when the target information is saved in the form of compressing the video stream and voice stream corresponding to the target video into an audio file, and then saving the audio file and the target information package, the audio is loaded simultaneously file and the target information, and then display the target content of the target information in the video image frame.

Optionally, based on the target information corresponding to the video image frame, after the target information is loaded, the target information can be acquired when the target video is played to the video image frame, so as to reduce the The amount of data processed when playing the target video.

Optionally, referring to FIG. 18 , when the play request of the target video is detected, the step of acquiring the target information includes:

Step S61: Obtain the target information according to the association relationship between the video image frame and the target information in the target video, and/or, when a playback request of the target video is detected, according to the playback time point and the time in the target information The corresponding relationship of stamps is used to obtain the target information.

Optionally, when the target information includes the target content and/or the target location, the present application associates the target information with the video image frame, and then generates the target information and the video image frame frame associations. Optionally, the association relationship includes the video image frame and target content appearing in the video image frame. Optionally, when playing to the video image frame, determine the target content corresponding to the video image frame according to the association relationship, and then display the target content at the target position in the video image frame .

Optionally, when the target information includes the target content, the time stamp of the video image frame and/or the target position, when a playback request of the target video is detected, during the process of playing the target video In, obtain the playback time point, determine the timestamp corresponding to the playback time point according to the correspondence between the playback time point and the timestamp in the target information, and then obtain the target information according to the timestamp, and then according to The target information determines the video image frame corresponding to the time stamp and the corresponding target content, and the target content is displayed at the target position in the video image frame.

In the embodiment of the present application, when the play request of the target video is detected, the target information is acquired, and then the target video is played, so that when the corresponding video frame is played, the The target content is displayed at the position of the text on the screen, so that the user can quickly understand the text according to the target content, make it easier to understand the video content, and improve user experience.

Eighth embodiment

Referring to FIG. 19, based on the seventh embodiment, the embodiment of the present invention proposes an eighth embodiment, and the S70 includes:

S71. Obtain target display parameters of the target information;

S72. During the playing process of the target video, display the target content at the target position corresponding to the video image frame with the target display parameters.

In the embodiment of the present application, the target display parameters include at least one of text display duration, text display mode, and text target display position, and the text display mode includes displaying the target content, and/or simultaneously displaying the target content and the first content. Optionally, the text display time corresponds to the time stamp of the video image frame. It is understandable that the same information may appear in consecutive image frames. Optionally, the text display duration may be the duration of only displaying and playing a preset number of image frames, that is, selecting a preset number of initial image frames in the continuous image frames, and selecting a preset number of initial image frames in the preset number of initial image frames The target content is displayed, and the target content is no longer displayed when the image frame next to the initial image frame is played. The preset number can be 1 or 2. Optionally, when the preset number is 3, the duration of 3 image frames is 5s, and the text display duration is 5s. Optionally, the text display manner includes displaying the target content alone or superimposing the target content and/or the first content. When the text display mode is to display the target content alone, the target content is overlaid on the target position in a preset manner. Optionally, when the text display method is to superimpose the target content and/or the first content, the target content may be displayed above, below, left or right of the first content .

Optionally, the text display method further includes highlighting the target content separately, that is, highlighting the target content after the target content is overlaid on the target position in a preset manner; or, the The text display method further includes superimposing and displaying the target content and/or the first content, and highlighting the target content.

Optionally, the text target display position is the target display position of the target content in the video image frame, and the target display position may be different from the target position or different. Optionally, when the target display position is different from the target position, during the process of playing the target video, the target content is displayed at the target display position of the video image frame without displaying the target content at the target display position of the video image frame. The target position is displayed at the target position, so as to realize the user's function of adaptively adjusting the display position of the target content.

Optionally, the target display parameter may be a default display parameter, or may be set by the user through a selection interface, and the selection interface is used to provide the user with a function of adjusting the display parameter. Optionally, the output mode of the selection interface may be that when the user inputs a play request of the target video, the smart terminal pops up the selection interface by itself. Optionally, the output mode of the selection interface may also be that the user manually opens the selection interface during the process of watching the target video.

Optionally, an embodiment of the present application proposes a method for acquiring the target display parameters, and the S71 includes: outputting a selection interface; in response to a trigger selection operation on the selection interface, determining the target display parameter according to the selection operation parameter.

Optionally, the selection interface may display at least one display parameter and the adjustment range corresponding to each display parameter, the display parameters include at least one of text display duration, text display mode, and text display position, and the display parameters may also be Including language, display color, display area, etc. Optionally, after the selection interface is output, the user may trigger a selection operation on the selection interface according to his own needs, and the smart terminal determines the target display parameter according to the selection operation when detecting the selection operation.

Optionally, when the display parameter is text display duration, the adjustment range is "1 image frame, 2 image frames, 3 image frames..."; when the display parameter is text display mode , the adjustment range is "display the target content alone, superimpose the target content and/or text, highlight the target content alone..."; when the display parameter is language, the adjustment range is "Chinese, Japanese, German, Thai...".

Optionally, the selection interface may also display various combined display parameters, and each combined display parameter includes several display parameters. For example: "Combined display parameter 1, combined display parameter 2, combined display parameter 3..." is displayed in the selection interface, and combined display parameter 1 includes "Text display display: 1 image frame; text display method: display the target content alone ;Text target display position: below the text; Language: Chinese". Optionally, the user can select any combination of display parameters based on their own needs, and then trigger the selection operation of the selection interface. After the smart terminal detects the selection operation, it determines the display parameters of the target combination based on the selection operation, and stores the target The combined display parameter is determined as the target display parameter.

Optionally, when the smart terminal detects the user's selection operation, it can save the target display parameters corresponding to the selection parameters, and the next time the user triggers to play the video, it can directly play the video according to the target display parameters.

In this embodiment of the present application, by providing users with a selection interface for users to adaptively adjust the display parameters of the target content based on the selection interface based on their own viewing needs, so that when the target video is played, the target display parameters Displaying the target content for the user not only facilitates the user to quickly understand the video content, but also satisfies the user's viewing needs, thereby improving the user's viewing experience.

Ninth embodiment

The present application also provides a video processing method. Referring to FIG. 20, the video processing method includes:

S80. Obtain first information corresponding to at least one video image frame in the target video and obtain second information corresponding to voice information in the target video;

S90. Determine or generate target information corresponding to the target video according to the first information and the second information.

In this embodiment of the present application, the first information is preset information existing in the video image frame, and the preset information may be text information on the video image frame. Specifically, it is determined that the video image frame contains Whether there is preset information, and if so, acquire the first information from the video image frame. Optionally, the method of obtaining the first information from the video image frame is to obtain the first information from the video image frame based on an optical character recognition (OCR) algorithm, and optionally, the first information includes First content and/or first location.

Optionally, the second information is a speech recognition result obtained by recognizing corresponding speech information in the target video based on speech-to-text technology. Optionally, after the voice information is acquired, the voice information is converted into corresponding second information through a voice-to-speech technology according to the voice information, and the second information is in one-to-one correspondence with the video image frames.

Optionally, after the first information and the second information are acquired, the target information is determined or generated according to the first information and the second information. Optionally, the target information may include the processed image text information of the first information, the target information may also include the processed voice text information of the second information, and the target information may also include The image text information and the voice text information.

Optionally, the step of determining or generating target information corresponding to the target video according to the first information and the second information includes: performing a third preset on the first information according to the second information processing to obtain processed image text information, and/or perform fourth preset processing on the second information according to the first information to obtain processed voice text information.

Optionally, the third preset processing may be calibration processing or other processing; the fourth preset processing may be calibration processing or other processing; the calibration processing may be voice calibration processing, and may also be text calibration processing.

Optionally, the first information includes first content. When the first content of the first information is rough, when the first content is recognized based on OCR, it may appear that the first information is not recognized exact situation. In this embodiment of the present application, by acquiring the voice information corresponding to each of the video image frames of the target video, and performing preset processing on the first information according to the voice information, to determine or generate the target content, the voice information One-to-one correspondence with the video image frames.

Optionally, after the voice information is acquired, the voice information is converted into corresponding second information through voice-to-speech technology according to the voice information, and then the third information is performed on the first information according to the second information. Preset processing, the second information is in one-to-one correspondence with the video image frames.

Optionally, the step of performing third preset processing on the first information according to the second information includes: acquiring content to be processed in the first information according to preset acquisition rules; The voice processing content corresponding to the content to be processed is determined in the information; and the content to be processed is subjected to preset processing according to the voice processing content to determine or generate the target information.

Optionally, the preset acquisition rule may be to receive a user's processing instruction for the first content, the processing instruction includes content to be processed, and the preset acquisition rule may also be that the smart terminal intelligently acquires the first content. Pending content. Optionally, the way for the smart terminal to intelligently acquire the to-be-processed content may be to use the first content as the to-be-processed content, or to acquire a probability coefficient corresponding to the first content. Optionally, when the probability coefficient is less than or equal to a preset probability coefficient, the first content is used as the content to be processed.

Optionally, the voice processing content is voice-converted second information.

Optionally, after the voice processing content is acquired, preset processing (such as calibration) is performed on the content to be processed according to the voice processing content, so as to obtain the content to be processed after preset processing, and then according to the The content to be processed after the preset processing determines the first content after the preset processing, and then determines or generates the target content according to the first information after the preset processing. Optionally, the content to be processed includes "Londo Avenue", the speech processing content includes "London Avenue", and then the first content is calibrated as "London Avenue".

Optionally, in yet another embodiment, after the content to be processed is acquired, the video image frame corresponding to the content to be processed is acquired, and then the voice information corresponding to the video image frame is acquired, and the The content to be processed is processed by default. Optionally, the content to be processed includes "Londo Avenue", the voice information includes "London Avenue", and then the to-be-processed content is calibrated as "London Avenue".

Optionally, when the user is playing the target video, in order to facilitate the user to watch the video, the smart terminal will generate corresponding subtitles according to the dialogue content corresponding to the target video, and then display the subtitles below the video image frame to It is convenient for users to easily understand the voice information of the target video, but there may be dialects in the voice information, or when the voice information includes some professional terms, it is easy to cause inaccurate generated subtitles. Based on this, the embodiment of this application proposes a A method for preset processing of subtitles.

Optionally, the second information is information converted through speech-to-speech technology according to the voice information, and after the second information is acquired, preset processing is performed on the second information according to the first information, so as to The processed second information is obtained, and then the voice information is determined according to the processed second information.

Optionally, the specific implementation manner of performing the fourth preset processing on the second information according to the first information includes: acquiring the text to be processed corresponding to the second information, combining the text to be processed with the Match the first content of the first information to obtain the target text content in the first content that matches the text to be processed. Optionally, the similarity between the target text content and the text to be processed is greater than or equal to a preset threshold, and the text to be processed is replaced by the target text content. Optionally, the text to be processed includes "P to P", the first content includes "P2P", the similarity between "P to P" and "P2P" is 98%, and the preset threshold is 95%. Then prove that the "P2P" is the target content, and then replace "P to P" with "P2P".

Optionally, after the text to be processed is replaced by the target information, the phonetic text information is determined or generated according to the replaced text to be processed.

Optionally, before performing the fourth preset processing on the second information according to the first information, it is necessary to judge in advance whether there is a target text matching the text to be processed in the first content of the first information content, if there is no target text content matching the text to be processed in the first content, it is not necessary to perform fourth preset processing on the second information based on the first information. Based on this, the embodiment of the present application A method for judging whether to perform fourth preset processing on the second information is also proposed.

Optionally, the step of performing fourth preset processing on the second information according to the first information to obtain the processed voice-text information includes:

judging whether the voice information corresponds to the first information;

If yes, perform a fourth preset process on the second information according to the first information, so as to obtain processed voice-text information.

Optionally, the manner of judging whether the voice information corresponds to the first information may be to obtain the voice feature parameters corresponding to the voice information and the voice recognition parameters corresponding to the first information, and convert the voice feature parameters to Compared with the speech recognition parameters, and then obtain the target speech recognition parameters matching the speech feature parameters, it can be understood that the speech recognition parameters corresponding to the first information include the target speech recognition parameters matching the speech feature parameters When the voice recognition parameter is used, it is determined that the target content corresponding to the voice information exists in the first information.

Optionally, the manner of obtaining the speech feature parameters corresponding to the speech information may be to perform word segmentation processing on the speech information to obtain several sub-speech information, and then obtain the speech feature parameters corresponding to each of the sub-speech information, Further, the speech feature parameters corresponding to each sub-speech information are determined as the speech feature parameters corresponding to the speech information.

Optionally, the manner of obtaining the speech recognition parameters corresponding to the first information may be to perform word segmentation processing on the first content to obtain several sub-contents, and then obtain the speech recognition parameters corresponding to each of the sub-contents, Further, the speech recognition parameters corresponding to the first information are determined according to the speech recognition parameters corresponding to each sub-content.

Optionally, after obtaining the target recognition parameters that match the speech feature parameters, it is determined that the speech information corresponds to the first information, and then the text to be processed corresponding to the second information is obtained, and then according to the Target recognition parameter acquisition determines the target text content corresponding to the text to be processed, and then replaces the text to be processed with the target text content, so as to complete the fourth preset processing of the second information, and then obtain the Voice text messages.

In the embodiment of the present application, by obtaining the first information and/or the second information corresponding to the target video, and then performing preset processing on the second information according to the first information, so as to obtain the voice text information, and /or perform preset processing on the first information according to the second information to obtain image text information, and then determine or generate target information according to the image text information and/or voice text information for subsequent playback When the target video is described, the target information can be obtained directly, and then the image text information and/or voice text information is displayed in the video image frame in the target video, so that the user can quickly understand it according to the image text The text information on the video image frame can accurately understand the dialogue content in the target video according to the voice text information, which makes it easier to understand the video content and improves user experience.

The embodiment of the present application also provides an intelligent terminal, the intelligent terminal includes a memory and a processor, and the XX program is stored in the memory, and when the XX program is executed by the processor, the steps of the video processing method in any of the foregoing embodiments are implemented.

An embodiment of the present application further provides a storage medium, on which a video processing program is stored, and when the video processing program is executed by a processor, the steps of the video processing method in any of the foregoing embodiments are implemented.

In the embodiments of the intelligent terminal and the computer-readable storage medium provided in the embodiments of the present application, all the technical features of any of the above-mentioned XX method embodiments may be included, and the expansion and explanation of the description are basically the same as the embodiments of the above-mentioned methods. This will not be repeated here.

An embodiment of the present application further provides a computer program product, the computer program product includes computer program code, and when the computer program code is run on the computer, the computer is made to execute the methods in the above various possible implementation manners.

The embodiment of the present application also provides a chip, including a memory and a processor. The memory is used to store a computer program, and the processor is used to call and run the computer program from the memory, so that the device installed with the chip executes the above various possible implementation modes. Methods.

It can be understood that the above scenario is only an example, and does not constitute a limitation on the application scenario of the technical solution provided by the embodiment of the present application, and the technical solution of the present application can also be applied to other scenarios. For example, those skilled in the art know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

The serial numbers of the above embodiments of the present application are for description only, and do not represent the advantages and disadvantages of the embodiments.

The steps in the methods of the embodiments of the present application can be adjusted, combined and deleted according to actual needs.

Units in the device in the embodiment of the present application may be combined, divided and deleted according to actual needs.

In this application, descriptions of the same or similar terms, concepts, technical solutions and/or application scenarios are generally only described in detail when they appear for the first time, and when they appear repeatedly later, for the sake of brevity, they are generally not repeated. When understanding the technical solutions and other contents of the present application, for the same or similar term concepts, technical solutions and/or application scenario descriptions that are not described in detail later, you can refer to the previous relevant detailed descriptions.

In this application, the description of each embodiment has its own emphasis. For the parts that are not detailed or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

The various technical features of the technical solution of the present application can be combined arbitrarily. For the sake of concise description, all possible combinations of the various technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, all It should be regarded as the scope described in this application.

Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence or in other words, the part that contributes to the prior art, and the computer software product is stored in one of the above storage media (such as ROM/RAM, magnetic CD, CD), including several instructions to make a terminal device (which may be a mobile phone, computer, server, controlled terminal, or network device, etc.) execute the method of each embodiment of the present application.

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. The computer can be a general purpose computer, special purpose computer, a computer network, or other programmable apparatus. Computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, e.g. Coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.) to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server, a data center, etc. integrated with one or more available media. Usable media may be magnetic media, (eg, floppy disk, memory disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, Solid State Disk (SSD)), among others.

The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structures or equivalent process transformations made by using the description of the application and the accompanying drawings are directly or indirectly used in other related technical fields. , are all included in the patent protection scope of the present application in the same way.

Claims

A video processing method, wherein the method comprises:

Obtain first information corresponding to at least one video image frame in the target video;

The first information is processed according to a preset rule to determine or generate target information corresponding to the video image frame.
The method according to claim 1, wherein the step of processing the first information according to preset rules to determine or generate target information corresponding to the video image frame comprises:

identifying the scene information of the video image frame, and determining or generating feature information according to the scene information;

The target information is determined or generated according to the characteristic information and the first information.
The method of claim 1, wherein the method further comprises:

Obtaining second information corresponding to the voice information in the target video;

Performing a first preset process on the first information according to the second information to determine or generate the target information.
The method according to claim 3, wherein the step of obtaining the second information corresponding to the voice information in the target video comprises:

Acquiring initial information corresponding to the voice information;

performing a second preset process on the initial information according to the first information to obtain the second information.
The method according to claim 3, wherein the step of processing the first information according to preset rules includes:

Acquiring descriptive information and/or associated information for the first content in the second information;

Determine or generate identification information corresponding to the first content according to the description information and/or the association information;

Identifying the first content according to the identification information to determine or generate the target content.
The method according to any one of claims 1 to 5, wherein the step of processing the first information according to preset rules includes:

When the language type corresponding to the first content does not match the preset language type, convert the first content into the target content corresponding to the preset language type; and/or determine or generate the target content according to the preset position rule The target location corresponding to the first location.
The method according to claim 6, wherein the preset location rules include at least one of the following:

taking a position separated by a preset distance from the first position as a target position;

In response to a preset operation within or outside of a first location, the target location is determined or generated based on the preset operation.
The method according to any one of claims 1 to 5, wherein, after the step of processing the first information according to preset rules, the method further comprises:

Determine or generate target information according to the target content and/or target position, and associate the target information with the video image frame; and/or, according to the target content, the time stamp of the video image frame and/or the target The location is determined or the target information is generated.
The method according to any one of claims 1 to 5, wherein said method further comprises:

When detecting the playback request of the target video, acquiring the target information;

During the playing of the target video, the target content is displayed at the target position corresponding to the video image frame.
The method according to claim 9, wherein, when the play request of the target video is detected, the step of obtaining the target information comprises:

Obtain the target information according to the association relationship between the video image frame in the target video and the target information, and/or, when a playback request of the target video is detected, according to the correspondence between the playback time point and the timestamp in the target information The relationship acquires the target information.
The method according to claim 9, wherein the step of displaying the target content at the position corresponding to the target image frame during the playback of the target video comprises:

Acquiring target display parameters of the target information;

During the playing process of the target video, the target content is displayed at the target position corresponding to the video image frame with the target display parameters.
The method according to claim 11, wherein said step of obtaining target display parameters of said target information comprises:

Output a selection interface;

In response to a trigger selection operation on the selection interface, the target display parameter is determined according to the selection operation.
The method according to any one of claims 1 to 5, wherein the step of obtaining the first information corresponding to at least one video image frame in the target video includes:

judging whether preset information exists in the video image frame;

If yes, acquire the first information from the video image frame.
A video processing method, wherein the method includes:

Acquiring first information corresponding to at least one video image frame in the target video and acquiring second information corresponding to voice information in the target video;

Determine or generate target information corresponding to the target video according to the first information and the second information.
The method according to claim 14, wherein the step of determining or generating target information corresponding to the target video according to the first information and the second information includes:

Performing third preset processing on the first information according to the second information to obtain image text information, and/or performing fourth preset processing on the second information according to the first information to obtain voice text messages;

The target information is determined or generated according to the image text information and/or the voice text information.
The method according to claim 15, wherein the step of performing third preset processing on the first information according to the second information to obtain processed image text information includes:

Acquiring the content to be processed in the first information according to a preset acquisition rule;

Determine the voice processing content corresponding to the content to be processed in the second information;

Performing a third preset process on the content to be processed according to the voice processing content, so as to determine or generate the image text information.
The method according to claim 15, wherein the step of performing a third preset process on the first information according to the second information to obtain image text information includes:

Acquiring descriptive information and/or associated information for the first content in the second information;

Determine or generate identification information corresponding to the first content according to the description information and/or associated information;

Identifying the first content according to the identification information to determine or generate the image text information.
The method according to claim 15, wherein said step of performing fourth preset processing on said second information according to said first information to obtain voice text information comprises:

judging whether the voice information corresponds to the first information;

If yes, perform a fourth preset process on the second information according to the first information, so as to obtain voice-to-text information.
An intelligent terminal, wherein the intelligent terminal comprises: a memory and a processor, wherein a video processing program is stored on the memory, and when the video processing program is executed by the processor, the method according to claim 1 is realized. Steps of a video processing method.
A storage medium, wherein a computer program is stored on the storage medium, and when the computer program is executed by a processor, the steps of the video processing method according to claim 1 are implemented.