CN115631131A - Image processing method, image processing device, electronic equipment and readable storage medium - Google Patents

Image processing method, image processing device, electronic equipment and readable storage medium Download PDF

Info

Publication number
CN115631131A
CN115631131A CN202211103552.6A CN202211103552A CN115631131A CN 115631131 A CN115631131 A CN 115631131A CN 202211103552 A CN202211103552 A CN 202211103552A CN 115631131 A CN115631131 A CN 115631131A
Authority
CN
China
Prior art keywords
image
image sequence
frame
sequence
verification result
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211103552.6A
Other languages
Chinese (zh)
Inventor
李昱江
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Vivo Mobile Communication Co Ltd
Original Assignee
Vivo Mobile Communication Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Vivo Mobile Communication Co Ltd filed Critical Vivo Mobile Communication Co Ltd
Priority to CN202211103552.6A priority Critical patent/CN115631131A/en
Publication of CN115631131A publication Critical patent/CN115631131A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30168—Image quality inspection

Landscapes

  • Engineering & Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

The application discloses an image processing method, an image processing device, electronic equipment and a readable storage medium, and belongs to the technical field of data processing. The image processing method comprises the following steps: acquiring a first image sequence and a second image sequence, wherein the second image sequence is an image sequence obtained after the first image sequence is processed; verifying the second image frames in the second image sequence according to the identification information of each first image frame in the first image sequence and the identification information of each second image frame in the second image sequence to obtain a verification result; under the condition that the verification result represents that the second image sequence comprises error frames, generating a third image sequence based on the second image sequence and the verification result; and under the condition that the verification result indicates that the second image sequence does not comprise error frames, determining the second image sequence as the third image sequence.

Description

Image processing method, image processing device, electronic equipment and readable storage medium
Technical Field
The present application relates to the field of data processing technologies, and in particular, to an image processing method and apparatus, an electronic device, and a readable storage medium.
Background
With the development of image processing technology, people have higher and higher requirements on images displayed by electronic equipment. In order to ensure that an electronic device can output a high-quality image meeting the needs of a user, various chips for processing the image are generally arranged in the electronic device in the prior art, however, when an object to be processed is an image sequence, after the image sequence is processed by the image processing chip, a problem of frame loss is likely to occur, and further, a problem that the image does not correspond to a special effect such as a subtitle is likely to occur. Therefore, the existing image processing method has the problem of poor image processing effect.
Disclosure of Invention
The application provides an image processing method, an image processing device, an electronic device and a readable storage medium, which can solve the problem of poor image processing effect in the existing image processing method.
In a first aspect, an embodiment of the present application provides an image processing method, including:
acquiring a first image sequence and a second image sequence, wherein the second image sequence is an image sequence obtained after the first image sequence is processed;
verifying the second image frames in the second image sequence according to the identification information of each first image frame in the first image sequence and the identification information of each second image frame in the second image sequence to obtain a verification result;
under the condition that the verification result represents that the second image sequence comprises error frames, generating a third image sequence based on the second image sequence and the verification result;
and determining the second image sequence as the third image sequence under the condition that the verification result indicates that the second image sequence does not include error frames.
In a second aspect, an embodiment of the present application provides an image processing apparatus, including:
the device comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring a first image sequence and a second image sequence, and the second image sequence is an image sequence obtained after the first image sequence is processed;
the verification module is used for verifying the second image frames in the second image sequence according to the identification information of each first image frame in the first image sequence and the identification information of each second image frame in the second image sequence to obtain a verification result;
a generating module, configured to generate a third image sequence based on the second image sequence and the verification result when the verification result indicates that the second image sequence includes an error frame;
and the determining module is used for determining the second image sequence as the third image sequence under the condition that the verification result indicates that the second image sequence does not comprise error frames.
In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, where the memory stores a program or instructions executable on the processor, and the program or instructions, when executed by the processor, implement the steps of the image processing method according to the first aspect.
In a fourth aspect, the present application provides a readable storage medium, where a program or an instruction is stored, and the program or the instruction when executed by a processor implements the steps of the image processing method according to the first aspect.
In the embodiment of the present application, after a first image sequence is processed to obtain a second image sequence, a second image frame in the second image sequence is checked by using identification information in the first image sequence and identification information of each second image frame in the second image sequence, so as to determine whether an error frame exists in the second image sequence. And generating a third image sequence based on the second image sequence and the checking result under the condition that the second image sequence comprises error frames. According to the method and the device, after the image sequence is processed, whether the error frame exists in the processing result is further verified, and the processing result is further processed under the condition that the error frame exists, so that the problem that the subsequent image display is abnormal due to the error frame existing in the processed image sequence is favorably solved, and the image processing effect is improved.
Drawings
Fig. 1 is a schematic flowchart of an image processing method according to an embodiment of the present application;
FIG. 2 is a diagram illustrating a buffer queue inside a main control chip according to an embodiment of the present disclosure;
FIG. 3 is a second schematic diagram of a buffer queue inside the main control chip according to the embodiment of the present application;
FIG. 4 is a schematic diagram illustrating an initialization process of a main control chip and an image processing chip according to an embodiment of the present disclosure;
fig. 5 is a schematic diagram of interpolating a first image sequence to obtain a second image sequence in an embodiment of the present application;
fig. 6 is a second flowchart of an image processing method according to an embodiment of the present application;
fig. 7 is a third schematic flowchart of an image processing method according to an embodiment of the present application;
FIG. 8 is a schematic diagram of a process of combining a shooting time stamp with an initial image frame in an embodiment of the present application;
FIG. 9 is a diagram illustrating the result of an image processing apparatus according to an embodiment of the present application;
fig. 10 is a schematic structural diagram of another electronic device provided in an embodiment of the present application;
fig. 11 is a schematic hardware structure diagram of an electronic device according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The terms first, second and the like in the description and in the claims of the present application are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that embodiments of the application may be practiced in sequences other than those illustrated or described herein, and that the terms "first," "second," and the like are generally used herein in a generic sense and do not limit the number of terms, e.g., the first term can be one or more than one. In addition, "and/or" in the specification and claims means at least one of connected objects, a character "/" generally means that a preceding and succeeding related objects are in an "or" relationship.
The image processing method, the image processing apparatus, the electronic device, and the readable storage medium according to the embodiments of the present application are described in detail below with reference to the accompanying drawings.
Referring to fig. 1, fig. 1 is a schematic flowchart of an image processing method according to an embodiment of the present application, where the image processing method includes:
step 101, acquiring a first image sequence and a second image sequence, wherein the second image sequence is an image sequence obtained after the first image sequence is processed;
102, verifying a second image frame in the second image sequence according to identification information of each first image frame in the first image sequence and identification information of each second image frame in the second image sequence to obtain a verification result;
103, generating a third image sequence based on the second image sequence and the verification result under the condition that the verification result represents that the second image sequence comprises error frames;
and 104, determining the second image sequence as the third image sequence under the condition that the verification result represents that the second image sequence does not include error frames.
The first image sequence may be an image sequence of multimedia content to be played by an electronic device, wherein to improve the picture quality of the multimedia content played by the electronic device, the electronic device may perform processing such as drying, sharpening, frame interpolation and the like on the first image sequence based on an internal chip of the electronic device to obtain the second image sequence.
However, in the process of processing the first image sequence by the electronic device based on the chip, an abnormality such as frame loss or disordered frame sequence may occur. Based on this, in the embodiment of the present application, before the second image sequence is displayed, the second image sequence is checked to determine whether the second image sequence includes an error frame.
The identification information may be an identification of each image frame in the image sequence, for example, a frame number of the image frame, and the identification information may also be an identification artificially added to the image frame. It is understood that the identification information of the same image frame does not change before and after the image processing, and the identification information of the same image frame is the same in the first image sequence and the second image sequence.
Based on this, the specific checking process for checking the second image frame in the second image sequence according to the identification information of each first image frame in the first image sequence and the identification information of each second image frame in the second image sequence may be: checking whether a second image frame corresponding to a first image frame in the first image sequence exists in the second image sequence, wherein the second image frame corresponding to the first image frame may refer to: a second image frame having the same identification information as the first image frame. In this way, in the case where there is a second image frame in the second image sequence that corresponds one-to-one to the first image frame in the first image sequence, it is determined that the second image frame includes an error frame. Accordingly, when at least one second image frame corresponding to a first image frame in the first image sequence is missing in the second image sequence, it is determined that an erroneous frame is included in the second image sequence.
In the foregoing, when the verification result indicates that the second image sequence includes an error frame, generating a third image sequence based on the second image sequence and the verification result may refer to: and marking error frames in the second image sequence according to the checking result to obtain the third image sequence. Or, when the error frame indicates that the second image sequence has frame loss abnormality, the frame insertion can be performed at the frame loss position of the second image sequence according to the position indicated by the error frame, so that the frame sequence of the second image sequence can be prevented from being disturbed due to the error frame, and the display effect of the third image sequence can be improved.
In this embodiment, after the first image sequence is processed to obtain the second image sequence, the second image frames in the second image sequence are checked by using the identification information in the first image sequence and the identification information of each second image frame in the second image sequence to determine whether there is an error frame in the second image sequence. And generating a third image sequence based on the second image sequence and the verification result under the condition that the second image sequence comprises error frames. According to the method and the device, after the image sequence is processed, whether the error frame exists in the processing result is further verified, and the processing result is further processed under the condition that the error frame exists, so that the problem that the subsequent image display is abnormal due to the error frame existing in the processed image sequence is favorably solved, and the image processing effect is improved.
Optionally, the verifying the second image frame in the second image sequence according to the identifier information of each first image frame in the first image sequence and the identifier information of each second image frame in the second image sequence to obtain a verification result includes:
under the condition that the time stamp of each first image frame in the first image sequence is in one-to-one correspondence with the time stamp of each second image frame in the second image sequence, obtaining a first time stamp checking result, wherein the first time stamp checking result represents that the second image sequence does not include error frames;
and under the condition that the time stamp of each first image frame in the first image sequence and the time stamp of each second image frame in the second image sequence have non-corresponding image frames, obtaining a second time stamp checking result, wherein the second time stamp checking result represents that the second image sequence comprises error frames.
The time stamp information may be time stamp information of the image frame when being generated, for example, the first image sequence may be an image sequence of a video that is captured by a user based on the electronic device, and the time stamp information may include a capture time stamp of the corresponding image frame. Since each first image frame in the first image sequence includes a unique capture timestamp, it is determined whether an erroneous frame is contained in the second image sequence by checking whether the timestamp of each first image frame in the first image sequence corresponds to the timestamp of each second image frame in the second image sequence.
Specifically, it may be checked whether a second image frame corresponding to a first image frame in the first image sequence exists in the second image sequence, where the second image frame corresponding to the first image frame may refer to: a second image frame having the same timestamp as the first image frame. When second image frames corresponding to the first image frames in the first image sequence in a one-to-one mode exist in the second image sequence, determining that the time stamp of each first image frame in the first image sequence corresponds to the time stamp of each second image frame in the second image sequence in a one-to-one mode. Accordingly, when at least one second image frame corresponding to a first image frame in the first image sequence is missing in the second image sequence, it is determined that a non-corresponding image frame exists between the time stamp of each first image frame in the first image sequence and the time stamp of each second image frame in the second image sequence.
In this embodiment, since the time stamp of each first image frame in the first image sequence has uniqueness, the time stamps of the image frames before and after the first image sequence is processed to obtain the second image sequence do not change. Therefore, whether the image frames in the first image sequence and the second image sequence have the time stamps in one-to-one correspondence is checked, so that the time stamp checking process of the second image sequence is realized.
Optionally, the verifying the second image frame in the second image sequence according to the identification information of each first image frame in the first image sequence and the identification information of each second image frame in the second image sequence to obtain a verification result includes:
obtaining a first ID checking result under the condition that the image Identification (ID) of each first image frame in the first image sequence is the same as the image Identification (ID) of a second image frame in the second image sequence, wherein the first ID checking result represents that the second image sequence does not comprise error frames;
and under the condition that the image identification ID of each first image frame in the first image sequence is different from the image identification ID of a second image frame in the second image sequence, obtaining a second ID check result, wherein the second ID check result represents that the second image sequence comprises error frames.
Wherein the image identification ID may be an identity of a preconfigured image, based on which the position of the image frame in an image sequence may be determined, the image identification ID of different image frames being different in an image sequence. And in the process of processing the first image sequence, the image identification IDs of the image frames before and after image processing do not change, so whether the second image sequence contains an error frame or not can be determined by checking whether the image identification ID of each first image frame in the first image sequence corresponds to the image identification ID of each second image frame in the second image sequence.
Specifically, it may be checked whether a second image frame corresponding to a first image frame in the first image sequence exists in the second image sequence, where the second image frame corresponding to the first image frame may refer to: a second image frame having the same image identification ID as the first image frame. When second image frames which correspond to the first image frames in the first image sequence in a one-to-one manner exist in the second image sequence, determining that the image identification ID of each first image frame in the first image sequence is the same as the image identification ID of the second image frame in the second image sequence. Accordingly, when at least one second image frame corresponding to a first image frame in the first image sequence is missing in the second image sequence, it is determined that the image identification ID of each first image frame in the first image sequence is different from the image identification ID of a second image frame in the second image sequence.
In this embodiment, since the image identifier ID of each first image frame in the first image sequence has uniqueness, the image identifier IDs of the image frames before and after the processing of the first image sequence to obtain the second image sequence do not change. Therefore, whether the image frames in the first image sequence and the second image sequence have image identification IDs in one-to-one correspondence or not is verified, so that the image identification ID verification process of the second image sequence is realized.
Optionally, in a case that the verification result indicates that an error frame is included in the second image sequence, generating a third image sequence based on the second image sequence and the verification result includes:
and under the condition that the verification result represents that the second image sequence loses a first target image frame, marking the first target image frame in the second image sequence as an error frame to obtain a third image sequence.
The first target image frame may refer to any image frame in the first image sequence, and it may be understood that the verification result indicating that the second image sequence loses the first target image frame may refer to: the second image sequence misses a first target image frame; or, the step of the verification result representing that the second image sequence loses the first target image frame may also refer to: the second image sequence misses at least two different first target image frames.
Specifically, after the second image sequence is obtained, the second image sequence may be stored in a pre-configured buffer queue, where the buffer queue includes a buffer space corresponding to each first image frame in the first image sequence. When the second image sequence loses the second target image frame, a loss flag may be set for a buffer space in the buffer queue corresponding to the second target image frame. In this way, the subsequent electronic device may determine that the first target image frame is an error frame based on the loss flag when displaying the third image sequence.
In this way, when the electronic device recognizes that the first target image frame is a displaced frame, the first target image frame may be processed according to the preconfigured processing policy, so as to avoid a problem that the electronic device displays a special effect such as an image and a subtitle in the third image sequence, where the special effect is not corresponding to the first target image frame.
In this embodiment, in a case that the verification result indicates that the second image sequence loses the first target image frame, the first target image frame in the second image sequence is marked as an error frame, so as to obtain a third image sequence, so that when a subsequent electronic device displays the third image sequence, the error frame in the third image sequence may be determined.
Optionally, in a case that the verification result indicates that the second image sequence includes an error frame, generating a third image sequence based on the second image sequence and the verification result includes:
and under the condition that the verification result represents that the identification information of a second target image frame in the second image sequence is not matched with the second target image frame and the identification information of the second target image frame is matched with a third target image frame in the second image sequence, updating the identification information of the third target image frame into the identification information of the second target image frame, and marking the second target image frame as an error frame to obtain a third image sequence.
Wherein the identification information may refer to an image identification ID, which may be used to indicate a position of the image frame in the first image sequence.
Specifically, since an error may occur in the frame sequence from the frame loss position when the frame loss occurs during the processing of the first image sequence, for example, when the first image sequence includes 20 image frame changes, if the 10 th image frame is lost, the frame numbers of the 11 th image frame to the 20 th image frame are changed to the 10 th frame to the 19 th frame, that is, the positions of the 11 th image frame to the 20 th image frame in the second image sequence are all changed. And the image identifier ID may be used to indicate the position of the image frame in the first image sequence, and the current frame number of the image frame characterizes the position of the image frame in the second image sequence, so that by comparing the image identifier ID with the current frame number of the image frame, it may be determined whether the current image frame is an error frame.
For example, referring to fig. 2, in an embodiment of the present application, a main control chip of an electronic device stores a first image sequence, a second image sequence, and a buffer queue for buffering the second image sequence, where the buffer queue is used for buffering the second image sequence. Wherein, T represents the shooting time stamp of the image frame, and BufID represents the image identification ID. After the main control chip acquires the second image sequence, the image frames in the second image sequence may be stored in a buffer space corresponding to a buffer queue.
Referring to fig. 3, in an embodiment of the present application, when image frames of BufID =21 in the second image sequence need to be stored in the buffer queue, if BufID =20 in the determined buffer space, the identification information of the image frame representing BufID =21 does not match the image frame of BufID =21, in this case, the second target image frame is an image frame of BufID =20, and the third target image frame is an image frame of BufID = 21. Therefore, the image frame with BufID =20 in the buffer queue may be marked as a misplaced frame, and at the same time, the identification information of the image frame with BufID =21 in the second image sequence may be stored in the image frame with BufID =21 in the buffer queue, so as to ensure that the identification information of the image frame with BufID =21 in the buffer queue matches with the image frame with BufID = 21. In this way, it may be avoided that the identification information of the image frames following the image frame with BufID =21 in the buffer queue matches with the corresponding image frame, and after the buffering of all the second image frames in the second image sequence is completed, the obtained buffer queue may be determined as the third image sequence.
In this embodiment, when the verification result indicates that the identification information of the second target image frame in the second image sequence is not matched with the second target image frame and the identification information of the second target image frame is matched with the third target image frame in the second image sequence, the identification information of the third target image frame is updated to the identification information of the second target image frame and the second target image frame is marked as an error frame to obtain the third image sequence.
Optionally, the error frame is used to instruct the electronic device to skip the error frame when displaying the third image sequence.
In this embodiment, the electronic device skips displaying the error frame when displaying the third image sequence, which is beneficial to improving the display effect of the third image sequence.
Optionally, the method is applied to an electronic device, where the electronic device includes a main control chip and an image processing chip, and the acquiring a first image sequence and a second image sequence includes:
the main control chip acquires a first image sequence;
the main control chip sends the first image sequence to an image processing chip;
the image processing chip carries out frame interpolation processing on the first image sequence to obtain a second image sequence;
and the image processing chip sends the second image sequence to the main control chip.
The main control Chip may be a Chip with an image processing function, which is commonly found in electronic devices, and may be, for example, a System On Chip (SOC). Accordingly, the image processing chip may be a plug-in chip independent of the image signal processing chip. In order to improve the image quality of the output amount of the electronic equipment, the image data can be processed simultaneously based on the main control chip and the image processing chip.
Specifically, the first image sequence may be an image sequence of a video recorded by an electronic device, or may also be an image sequence of a video to be played by the electronic device, and in order to improve a display effect of the first image sequence, in this embodiment of the application, the image processing chip may be used to perform frame interpolation processing on the first image sequence to obtain a second image sequence.
Referring to fig. 4, before processing the first image sequence, the main control chip and the image processing chip may be initialized based on initialization configuration information based on SoftWare (SoftWare, SW). Specifically, the initialization configuration may be performed on a relevant module in the main control chip, and then the initialization configuration is performed on a Camera module (Camera) and a Display Processing Unit (DPU) in the image Processing chip, so as to complete the initialization process of the image Processing chip.
Specifically, the image processing chip performs frame interpolation on the first image sequence to obtain the second image sequence, and may specifically insert an image frame between any two adjacent frames of the first image sequence based on a motion smoothing means, and the image identifier ID of the inserted image frame may be an average value of the image identifier IDs of the two adjacent image frames. Meanwhile, the time stamp of the inserted image frame may be an average of time stamps of its adjacent two image frames. For example, referring to fig. 5, in an embodiment of the present application, the first image sequence may include two image frames, namely an image frame 1 and an image frame 2, the main control chip transfers the first image sequence to the image processing chip based on a Display Serial Interface (DSI), and the image processing chip inserts the image frame 1.5 between the image frame 1 and the image frame 2 after acquiring the first image sequence, so as to obtain a second image sequence. And the image processing chip transmits the second image sequence back to the main control chip through a Camera Serial Interface (CSI), so that the main control chip can further process the second image sequence.
In the embodiment, because the probability of frame loss is high in the process of frame insertion processing of the image sequence, based on this, the second image sequence after frame insertion is checked by using the identification information, so that whether the frame loss and other problems exist in the frame insertion process can be determined.
Optionally, the sending, by the main control chip, the first image sequence to an image processing chip includes:
and under the condition that the main control chip detects that the brightness value of a first image frame in the first image sequence is lower than a preset brightness value, the main control chip sends the first image sequence to the image processing chip.
Specifically, the first image sequence may be a sequence of recorded images recorded by the electronic device. When the brightness value of the first image frame is lower than the preset brightness value, it can be regarded that the current shooting environment is dark, and at this time, the display fluency when the first image sequence obtained by shooting is displayed is generally poor, so that the first image sequence can be sent to the image processing chip, the image processing chip can conveniently perform frame insertion processing on the first image sequence, and the display fluency of the processed image sequence can be further improved.
It can be understood that, when the brightness value of the first image frame is not lower than the preset brightness value, it represents that the current shooting environment is normal, and at this time, the main control chip may directly send and display the first image sequence without sending to the image processing chip for processing, so as to reduce the time delay of image processing.
In the embodiment, when the current shooting environment is determined to be darker based on the brightness value of the first image frame in the first image sequence, the shot first image sequence is sent to the image processing chip for frame insertion processing, so that the quality of the image output by the electronic device is improved.
Optionally, the acquiring, by the main control chip, a first image sequence includes:
the main control chip receives an initial image sequence and a time stamp sequence which are shot by an image sensor, wherein the initial image sequence comprises initial image frames which are in one-to-one correspondence with first image frames in the first image sequence, and the time stamp sequence comprises shooting time stamps of each initial image frame in the initial image sequence;
and synthesizing the initial image sequence and the time stamp sequence to obtain the first image sequence, wherein the first image frame is obtained by synthesizing the corresponding initial image frame and the shooting time stamp of the initial image frame.
Referring to fig. 6-7, in the embodiment of the present application, the main control chip has a Display frame and a Display Processing Unit (DPU). The display framework sequentially comprises a Hardware layer (Hardware), a driving layer (Driver), a middle layer (MiddleWare) and an Application layer (APP) so as to realize the display of the image sequence. The DPU may be configured to synthesize the initial sequence of images and the sequence of time stamps.
Specifically, when the display frame of the main control chip acquires the initial image sequence and the timestamp sequence, it may be determined whether a brightness value of an image frame in the initial image sequence is lower than a preset brightness value, and if not, the display is directly sent. And if so, respectively transmitting the initial image sequence and the time stamp sequence to the DPU, synthesizing each initial image frame in the initial image sequence and the shooting time stamp corresponding to the initial image frame by the DPU after acquiring the initial image sequence and the time stamp sequence to obtain a first image frame, and obtaining the first image sequence after completing the synthesis of all the initial image frames in the initial image sequence. And transmitting the first image sequence to the image processing chip for frame interpolation to obtain a second image sequence. And the image processing chip transmits the second image sequence to a display frame of the main control chip, and the display frame verifies the second image sequence based on the identification information and sends and displays a third image sequence under the condition of obtaining the third image sequence.
Specifically, after the third image sequence is obtained by the display framework, the third image sequence may be transmitted to the application layer based on a display path, where please refer to fig. 7, the display path includes a Hardware layer (Hardware), a Driver layer (Driver), an intermediate layer (MiddleWare), and an application layer (APP) that are connected in sequence. And then, transmitting the third image sequence to a display screen by the application layer for displaying. The application layer may determine that an error image frame exists in a third image sequence according to an error frame identified in the third image sequence, and skip the error frame when sequentially transmitting the image frames in the third image sequence to a display screen for display, that is, the application layer may not transmit the error frame to the display screen, so as to avoid a problem of abnormal display such as stuck when the display screen displays the third image sequence.
Referring to fig. 8, when the initial image frame and the corresponding shooting timestamp are combined, the shooting timestamp and the initial image frame may be respectively located at different positions of a first image frame after the combination, for example, the first 6 lines (lines) of the first image frame are the shooting timestamp, and the last 1920 lines are the initial image frame.
Referring to fig. 9, a schematic structural diagram of an image processing apparatus 900 according to an embodiment of the present disclosure is shown, where the image processing apparatus 900 includes:
an obtaining module 901, configured to obtain a first image sequence and a second image sequence, where the second image sequence is an image sequence obtained after processing the first image sequence;
a checking module 902, configured to check a second image frame in the second image sequence according to identification information of each first image frame in the first image sequence and identification information of each second image frame in the second image sequence to obtain a checking result;
a generating module 903, configured to generate a third image sequence based on the second image sequence and the verification result when the verification result indicates that the second image sequence includes an error frame;
a determining module 904, configured to determine the second image sequence as the third image sequence if the check result indicates that no error frame is included in the second image sequence.
Optionally, the identification information includes timestamp information of an image frame, the verification result includes a timestamp verification result, and the verification module 902 is specifically configured to obtain a first timestamp verification result when a timestamp of each first image frame in the first image sequence corresponds to a timestamp of each second image frame in the second image sequence one to one, where the first timestamp verification result indicates that the second image sequence does not include an error frame;
the checking module 902 is further configured to obtain a second timestamp check result when the timestamp of each first image frame in the first image sequence and the timestamp of each second image frame in the second image sequence have non-corresponding image frames, where the second timestamp check result indicates that the second image sequence includes an error frame.
Optionally, the identification information includes an image identifier ID of the image frame, the verification result includes an ID verification result, and the verification module 902 is further configured to obtain a first ID verification result when the image identifier ID of each first image frame in the first image sequence is the same as the image identifier ID of a second image frame in the second image sequence, where the first ID verification result indicates that the second image sequence does not include an error frame;
the checking module 902 is further configured to obtain a second ID checking result when the image identifier ID of each first image frame in the first image sequence is different from the image identifier ID of the second image frame in the second image sequence, where the second ID checking result indicates that the second image sequence includes an error frame.
Optionally, the generating module 903 is specifically configured to, when the check result indicates that the second image sequence loses a first target image frame, mark the first target image frame in the second image sequence as an error frame, and obtain a third image sequence.
Optionally, the generating module 903 is specifically configured to, when the verification result represents that the identification information of a second target image frame in the second image sequence is not matched with the second target image frame, and the identification information of the second target image frame is matched with a third target image frame in the second image sequence, update the identification information of the third target image frame to the identification information of the second target image frame, and mark the second target image frame as an error frame, so as to obtain a third image sequence.
Optionally, the error frame is used to instruct the electronic device to skip the error frame when displaying the third image sequence.
Optionally, the apparatus includes a main control chip and an image processing chip, and the obtaining module 901 includes:
the acquisition submodule is used for acquiring a first image sequence based on the main control chip;
the sending submodule is used for sending the first image sequence to an image processing chip by the main control chip;
the frame interpolation submodule is used for performing frame interpolation processing on the first image sequence based on the image processing chip to obtain a second image sequence;
the sending submodule is further configured to send the second image sequence to the main control chip based on the image processing chip.
Optionally, the sending sub-module is specifically configured to, when the main control chip detects that a brightness value of a first image frame in the first image sequence is lower than a preset brightness value, send the first image sequence to the image processing chip by the main control chip.
Optionally, the obtaining sub-module includes:
the image processing device comprises a receiving unit, a processing unit and a processing unit, wherein the receiving unit is used for receiving an initial image sequence and a time stamp sequence which are shot by an image sensor based on the main control chip, the initial image sequence comprises initial image frames which are in one-to-one correspondence with first image frames in the first image sequence, and the time stamp sequence comprises shooting time stamps of each initial image frame in the initial image sequence;
and the synthesizing unit is used for synthesizing the initial image sequence and the time stamp sequence to obtain the first image sequence, wherein the first image frame is obtained by synthesizing the corresponding initial image frame and the shooting time stamp of the initial image frame.
In this embodiment, after a first image sequence is processed to obtain a second image sequence, a second image frame in the second image sequence is checked by using identification information in the first image sequence and identification information of each second image frame in the second image sequence to determine whether an error frame exists in the second image sequence. And generating a third image sequence based on the second image sequence and the checking result under the condition that the second image sequence comprises error frames. According to the method and the device, after the image sequence is processed, whether the error frame exists in the processing result is further checked, and the processing result is further processed under the condition that the error frame exists, so that the problem that the subsequent image display is abnormal due to the fact that the error frame exists in the processed image sequence is solved, and the image processing effect is improved.
Optionally, as shown in fig. 10, another electronic device 1000 is further provided in the embodiment of the present application, and includes an image processing circuit, a processor 1001, and a memory 1002, where the image processing circuit is the image processing circuit described in the foregoing embodiment, and a program or an instruction stored in the memory 1002 and capable of running on the processor 1001 is implemented on the memory 1002, and when the program or the instruction is executed by the processor 1001, the processes of the embodiment of the image processing method are implemented, and the same technical effects can be achieved, and are not described herein again to avoid repetition.
It should be noted that the electronic device in the embodiment of the present application includes the mobile electronic device and the non-mobile electronic device described above.
Fig. 11 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.
The electronic device 1100 includes, but is not limited to: radio frequency unit 1101, network module 1102, audio output unit 1103, input unit 1104, sensor 1105, display unit 1106, user input unit 1107, interface unit 1108, memory 1109, and processor 1110.
The processor 1110 is configured to acquire a first image sequence and a second image sequence, where the second image sequence is an image sequence obtained after the first image sequence is processed;
the processor 1110 is configured to check a second image frame in the second image sequence according to identification information of each first image frame in the first image sequence and identification information of each second image frame in the second image sequence, so as to obtain a check result;
the processor 1110 is configured to generate a third image sequence based on the second image sequence and the verification result if the verification result indicates that the second image sequence includes an error frame;
the processor 1110 is configured to determine the second image sequence as the third image sequence if the verification result indicates that no error frame is included in the second image sequence.
Optionally, the processor 1110 is configured to obtain a first timestamp check result in a case that a timestamp of each first image frame in the first image sequence corresponds to a timestamp of each second image frame in the second image sequence in a one-to-one manner, where the first timestamp check result indicates that the second image sequence does not include an error frame;
the processor 1110 is configured to obtain a second timestamp check result when a timestamp of each first image frame in the first image sequence and a timestamp of each second image frame in the second image sequence have non-corresponding image frames, where the second timestamp check result indicates that the second image sequence includes an error frame.
Optionally, the processor 1110 is configured to obtain a first ID check result in the case that an image identification ID of each first image frame in the first image sequence is the same as an image identification ID of a second image frame in the second image sequence, where the first ID check result indicates that the second image sequence does not include an error frame;
the processor 1110 is configured to obtain a second ID check result when the image identifier ID of each first image frame in the first image sequence is different from the image identifier ID of a second image frame in the second image sequence, where the second ID check result indicates that the second image sequence includes an error frame.
Optionally, the processor 1110 is configured to mark a first target image frame in the second image sequence as an error frame to obtain a third image sequence, where the check result indicates that the second image sequence loses the first target image frame.
Optionally, the processor 1110 is configured to, when the verification result indicates that the identification information of a second target image frame in the second image sequence does not match the second target image frame, and the identification information of the second target image frame matches a third target image frame in the second image sequence, update the identification information of the third target image frame to the identification information of the second target image frame, and mark the second target image frame as an error frame, so as to obtain a third image sequence.
Optionally, the processor 1110 is configured to obtain a first image sequence based on the main control chip;
the processor 1110 is configured to send the first image sequence to an image processing chip based on the main control chip;
the processor 1110 is configured to perform frame interpolation on the first image sequence based on the image processing chip to obtain a second image sequence;
the processor 1110 is configured to send the second image sequence to the main control chip based on the image processing chip.
Optionally, the processor 1110 is configured to send the first image sequence to the image processing chip based on the main control chip when the main control chip detects that a brightness value of a first image frame in the first image sequence is lower than a preset brightness value.
Optionally, the processor 1110 is configured to receive, based on the main control chip, an initial image sequence captured by an image sensor and a timestamp sequence, where the initial image sequence includes initial image frames in one-to-one correspondence with first image frames in the first image sequence, and the timestamp sequence includes a capture timestamp of each initial image frame in the initial image sequence;
the processor 1110 is configured to synthesize the initial image sequence and the time stamp sequence to obtain the first image sequence, where the first image frame is obtained by synthesizing the corresponding initial image frame and the shooting time stamp of the initial image frame.
Those skilled in the art will appreciate that the electronic device 1100 may further include a power source (e.g., a battery) for supplying power to the various components, and the power source may be logically connected to the processor 1110 via a power management system, so as to manage charging, discharging, and power consumption management functions via the power management system. The electronic device structure shown in fig. 11 does not constitute a limitation of the electronic device, and the electronic device may include more or less components than those shown, or combine some components, or arrange different components, and thus, the description is not repeated here.
It should be understood that, in the embodiment of the present application, the input Unit 1104 may include a Graphics Processing Unit (GPU) 11041 and a microphone 11042, and the Graphics processor 11041 processes image data of still pictures or video obtained by an image capturing device (such as a camera) in a video capturing mode or an image capturing mode. The display unit 1106 may include a display panel 11061, and the display panel 11061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 1107 includes a touch panel 11071 and other input devices 11072. A touch panel 11071, also called a touch screen. The touch panel 11071 may include two portions of a touch detection device and a touch controller. Other input devices 11072 may include, but are not limited to, a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which are not described in detail herein. The memory 1109 may be used to store software programs and various data including, but not limited to, application programs and an operating system. Processor 1110 may integrate an application processor that handles primarily operating systems, user interfaces, applications, etc. and a modem processor that handles primarily wireless communications. It will be appreciated that the modem processor described above may not be integrated into processor 1110.
The embodiments of the present application further provide a readable storage medium, where a program or an instruction is stored, and when the program or the instruction is executed by a processor, the program or the instruction implements the processes of the embodiment of the image processing method, and can achieve the same technical effect, and in order to avoid repetition, details are not repeated here.
The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and so on.
The embodiment of the present application further provides a chip, where the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to execute a program or an instruction to implement each process of the above-mentioned embodiment of the image processing method, and can achieve the same technical effect, and is not described here again to avoid repetition.
It should be understood that the chips mentioned in the embodiments of the present application may also be referred to as system-on-chip, system-on-chip or system-on-chip, etc.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one of 8230, and" comprising 8230does not exclude the presence of additional like elements in a process, method, article, or apparatus comprising the element. Further, it should be noted that the scope of the methods and apparatus of the embodiments of the present application is not limited to performing the functions in the order illustrated or discussed, but may include performing the functions in a substantially simultaneous manner or in a reverse order based on the functions involved, e.g., the methods described may be performed in an order different than that described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.
Through the description of the foregoing embodiments, it is clear to those skilled in the art that the method of the foregoing embodiments may be implemented by software plus a necessary general hardware platform, and certainly may also be implemented by hardware, but in many cases, the former is a better implementation. Based on such understanding, the technical solutions of the present application or portions thereof that contribute to the prior art may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (which may be a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present application.
While the present embodiments have been described with reference to the accompanying drawings, it is to be understood that the present embodiments are not limited to those precise embodiments, which are intended to be illustrative rather than restrictive, and that various changes and modifications may be effected therein by one skilled in the art without departing from the scope of the appended claims.

Claims (14)

1. An image processing method, comprising:
acquiring a first image sequence and a second image sequence, wherein the second image sequence is an image sequence obtained after the first image sequence is processed;
verifying a second image frame in the second image sequence according to the identification information of each first image frame in the first image sequence and the identification information of each second image frame in the second image sequence to obtain a verification result;
under the condition that the verification result indicates that the second image sequence comprises error frames, generating a third image sequence based on the second image sequence and the verification result;
and under the condition that the verification result indicates that the second image sequence does not comprise error frames, determining the second image sequence as the third image sequence.
2. The method according to claim 1, wherein the identification information includes time stamp information of image frames, the verification result includes a time stamp verification result, and the verifying a second image frame in the second image sequence according to the identification information of each first image frame in the first image sequence and the identification information of each second image frame in the second image sequence to obtain the verification result includes:
under the condition that the time stamp of each first image frame in the first image sequence is in one-to-one correspondence with the time stamp of each second image frame in the second image sequence, obtaining a first time stamp checking result, wherein the first time stamp checking result represents that the second image sequence does not include error frames;
and under the condition that the time stamp of each first image frame in the first image sequence and the time stamp of each second image frame in the second image sequence have non-corresponding image frames, obtaining a second time stamp checking result, wherein the second time stamp checking result represents that the second image sequence comprises error frames.
3. The method according to claim 1, wherein the identification information includes image identification IDs of the image frames, the verification result includes an ID verification result, and the verifying a second image frame in the second image sequence according to the identification information of each first image frame in the first image sequence and the identification information of each second image frame in the second image sequence to obtain the verification result includes:
under the condition that the image identification ID of each first image frame in the first image sequence is the same as the image identification ID of a second image frame in the second image sequence, obtaining a first ID check result, wherein the first ID check result indicates that the second image sequence does not comprise error frames;
and under the condition that the image identification ID of each first image frame in the first image sequence is different from the image identification ID of a second image frame in the second image sequence, obtaining a second ID check result, wherein the second ID check result represents that the second image sequence comprises error frames.
4. The method according to claim 1, wherein in a case that the verification result indicates that the second image sequence includes an error frame, generating a third image sequence based on the second image sequence and the verification result comprises:
and under the condition that the verification result represents that the second image sequence loses a first target image frame, marking the first target image frame in the second image sequence as an error frame to obtain a third image sequence.
5. The method according to claim 1, wherein in a case that the verification result indicates that the second image sequence includes an error frame, generating a third image sequence based on the second image sequence and the verification result comprises:
and under the condition that the verification result indicates that the identification information of a second target image frame in the second image sequence is not matched with the second target image frame and the identification information of the second target image frame is matched with a third target image frame in the second image sequence, updating the identification information of the third target image frame into the identification information of the second target image frame, and marking the second target image frame as an error frame to obtain a third image sequence.
6. The method according to claim 4 or 5, wherein the error frame is used to instruct an electronic device to skip the error frame when displaying the third image sequence.
7. The method according to any one of claims 1 to 4, applied to an electronic device, wherein the electronic device comprises a main control chip and an image processing chip, and the acquiring the first image sequence and the second image sequence comprises:
the main control chip acquires a first image sequence;
the main control chip sends the first image sequence to an image processing chip;
the image processing chip carries out frame interpolation processing on the first image sequence to obtain a second image sequence;
and the image processing chip sends the second image sequence to the main control chip.
8. The method of claim 7, wherein the master chip sends the first sequence of images to an image processing chip, comprising:
and under the condition that the main control chip detects that the brightness value of a first image frame in the first image sequence is lower than a preset brightness value, the main control chip sends the first image sequence to the image processing chip.
9. The method of claim 7, wherein the master control chip acquiring the first sequence of images comprises:
the main control chip receives an initial image sequence and a time stamp sequence which are shot by an image sensor, wherein the initial image sequence comprises initial image frames which are in one-to-one correspondence with first image frames in the first image sequence, and the time stamp sequence comprises shooting time stamps of each initial image frame in the initial image sequence;
and synthesizing the initial image sequence and the time stamp sequence to obtain the first image sequence, wherein the first image frame is obtained by synthesizing the corresponding initial image frame and the shooting time stamp of the initial image frame.
10. An image processing apparatus characterized by comprising:
the device comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring a first image sequence and a second image sequence, and the second image sequence is an image sequence obtained after the first image sequence is processed;
the verification module is used for verifying the second image frames in the second image sequence according to the identification information of each first image frame in the first image sequence and the identification information of each second image frame in the second image sequence to obtain a verification result;
a generating module, configured to generate a third image sequence based on the second image sequence and the verification result when the verification result indicates that the second image sequence includes an error frame;
and the determining module is used for determining the second image sequence as the third image sequence under the condition that the verification result indicates that the second image sequence does not comprise error frames.
11. The apparatus according to claim 10, wherein the identification information includes timestamp information of the image frames, the verification result includes a timestamp verification result, and the verification module is specifically configured to obtain a first timestamp verification result if a timestamp of each first image frame in the first image sequence corresponds to a timestamp of each second image frame in the second image sequence one to one, where the first timestamp verification result indicates that the second image sequence does not include an error frame;
the checking module is further configured to obtain a second timestamp checking result when a timestamp of each first image frame in the first image sequence and a timestamp of each second image frame in the second image sequence have non-corresponding image frames, where the second timestamp checking result indicates that the second image sequence includes an error frame.
12. The apparatus of claim 10, wherein the apparatus comprises a main control chip and an image processing chip, and the obtaining module comprises:
the acquisition submodule is used for acquiring a first image sequence based on the main control chip;
the sending submodule is used for sending the first image sequence to an image processing chip by the main control chip;
the frame interpolation submodule is used for performing frame interpolation processing on the first image sequence based on the image processing chip to obtain a second image sequence;
the sending submodule is further configured to send the second image sequence to the main control chip based on the image processing chip.
13. An electronic device comprising a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions when executed by the processor implementing the steps of the image processing method of any of claims 1 to 9.
14. A readable storage medium, characterized in that it stores thereon a program or instructions which, when executed by a processor, implement the steps of the image processing method according to any one of claims 1 to 9.
CN202211103552.6A 2022-09-09 2022-09-09 Image processing method, image processing device, electronic equipment and readable storage medium Pending CN115631131A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211103552.6A CN115631131A (en) 2022-09-09 2022-09-09 Image processing method, image processing device, electronic equipment and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211103552.6A CN115631131A (en) 2022-09-09 2022-09-09 Image processing method, image processing device, electronic equipment and readable storage medium

Publications (1)

Publication Number Publication Date
CN115631131A true CN115631131A (en) 2023-01-20

Family

ID=84902541

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211103552.6A Pending CN115631131A (en) 2022-09-09 2022-09-09 Image processing method, image processing device, electronic equipment and readable storage medium

Country Status (1)

Country Link
CN (1) CN115631131A (en)

Similar Documents

Publication Publication Date Title
CN109783178B (en) Color adjusting method, device, equipment and medium for interface component
US11778263B2 (en) Live streaming video interaction method and apparatus, and computer device
CN106648391B (en) Handwriting synchronization method, device and system
CN106791915B (en) Method and device for displaying video image
CN115665342B (en) Image processing method, image processing circuit, electronic device, and readable storage medium
CN112019907A (en) Live broadcast picture distribution method, computer equipment and readable storage medium
CN114302092A (en) One-display frame insertion circuit, method, device, chip, electronic device and medium
CN113891168B (en) Subtitle processing method, subtitle processing device, electronic equipment and storage medium
CN110187858B (en) Image display method and system
CN116419049A (en) Image processing method, image processing system, device and electronic equipment
US20120300127A1 (en) System for managing detection of advertisements in an electronic device, for example in a digital tv decoder
CN112019906A (en) Live broadcast method, computer equipment and readable storage medium
CN115631131A (en) Image processing method, image processing device, electronic equipment and readable storage medium
CN107995538B (en) Video annotation method and system
CN112492331B (en) Live broadcast method, device, system and storage medium
CN115426514A (en) Cross-device audio and video synchronization method, device, equipment and medium
CN115080099A (en) System upgrade method, apparatus, storage medium, and program product
CN114302209A (en) Video processing method, video processing device, electronic equipment and medium
CN110189388B (en) Animation detection method, readable storage medium, and computer device
CN112887781A (en) Subtitle processing method and device
CN112541858A (en) Video image enhancement method, device, equipment, chip and storage medium
CN112765185B (en) Data updating method, device, equipment and storage medium for vehicle-mounted equipment
CN114338953B (en) Video processing circuit, video processing method and electronic equipment
CN115589450B (en) Video recording method and device
CN115604540B (en) Video acquisition method, electronic equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination