WO2024104439A1

WO2024104439A1 - Image frame interpolation method and apparatus, device, and computer readable storage medium

Info

Publication number: WO2024104439A1
Application number: PCT/CN2023/132099
Authority: WO
Inventors: 邱绪东
Original assignee: 歌尔科技有限公司
Priority date: 2022-11-17
Filing date: 2023-11-16
Publication date: 2024-05-23
Also published as: CN115835035A

Abstract

The present application relates to the technical field of image frame interpolation and discloses an image frame interpolation method and apparatus, a device, and a computer readable storage medium. The image frame interpolation method comprises: obtaining a current image frame and inter-frame pixel point movement information, wherein the inter-frame pixel point movement information is pixel point movement information between the current image frame and a next image frame; generating at least one interpolated image frame according to the current image frame and the inter-frame pixel point movement information; and sequentially outputting the current image frame and the interpolated image frame to a preset display screen. The present application increases output image frame rate, and the MR experience of a user is ensured.

Description

Image interpolation method, device, equipment and computer-readable storage medium

This application claims priority to the Chinese patent application filed with the China Patent Office on November 17, 2022, with application number 202211441273.0 and application name “Image interpolation method, device, equipment and computer-readable storage medium”, the entire contents of which are incorporated by reference in this application.

Technical Field

The present application relates to the field of image interpolation technology, and in particular to an image interpolation method, device, equipment and computer-readable storage medium.

Background technique

With the explosion of VR (Virtual Reality) and AR (Augmented Reality) industries in the past two years, their value in industry, medical care, entertainment, office, and social interaction has gradually been revealed. Currently, major manufacturers are also exploring and deploying in the MR (Mixed Reality) field.

The main technical route for realizing MR at present is based on VST (Video See-Through) technology. In VST technology, a real-time view of the real world is captured by a camera, and then the real-time view is combined with a virtual view of the digital world and presented on a display screen to send the combined image to the user's eyes. Based on VST technology, visual integration can be fully controlled, allowing complete occlusion between virtual and real objects, and even higher-level modifications to real objects.

However, the cameras that use VST technology to shoot real-world scenes usually use CIS (CMOS Image Sensor) for imaging, and CIS is based on charge, which needs to be integrated before output. This limits the sampling frequency of CIS, resulting in a low frame rate of the output image (usually dozens to hundreds of frames per second), affecting the user's MR experience.

Summary of the invention

The main purpose of the present application is to provide an image interpolation method, device, equipment and computer-readable storage medium, aiming to solve the technical problem of low image frame rate output by a camera using CIS for imaging.

To achieve the above objectives, in a first aspect, the present application provides an image interpolation method, the image interpolation method comprising:

Acquire the current frame image and inter-frame pixel point movement information, wherein the inter-frame pixel point movement information is the pixel point movement information between the current frame image and the next frame image;

Generate at least one inserted frame image according to the current frame image and inter-frame pixel movement information;

The current frame image and the inserted frame image are outputted sequentially to a preset display screen.

Based on the above technical solution, by obtaining the current frame image and inter-frame pixel point movement information, wherein the inter-frame pixel point movement information is the pixel point movement information between the current frame image and the next frame image; generating at least one frame of inserted frame image according to the current frame image and the inter-frame pixel point movement information; and outputting the current frame image and the inserted frame image to the preset display screen in sequence. Thus, this embodiment performs image compensation on the current frame image through the inter-frame pixel point movement information, generates an inserted frame image after the current frame image, and after outputting the current frame image and the inserted frame image to the preset display screen in sequence, the frame rate of the output image can be improved, thereby ensuring the user's MR experience.

According to the first aspect, the moving pixels in the current frame image and the moving trajectory of the moving pixels are determined according to the inter-frame pixel movement information;

The moving pixels in the current frame image are moved according to the moving trajectory to generate at least one inserted frame image.

Based on the above technical solution, by determining the moving pixels in the current frame image and the moving trajectory of the moving pixels according to the inter-frame pixel movement information; moving the moving pixels in the current frame image according to the moving trajectory to generate at least one inserted frame image. Compared with the method of generating an inserted frame image through motion vectors and a neural network model, this embodiment generates an inserted frame image with higher accuracy.

According to the first aspect, or any implementation of the first aspect above, before the step of determining the moving pixels in the current frame image and the moving trajectory of the moving pixels according to the inter-frame pixel movement information, the step includes:

Acquire an inter-frame period and a target number of inserted frames between a first shooting moment of the current frame image and a second shooting moment of the next frame image;

According to the inter-frame period and the target number of inserted frames, a corresponding inter-frame time point in the inter-frame period is determined.

Based on the above technical solution, the interframe period and the target number of inserted frames between the first shooting moment of the current frame image and the second shooting moment of the next frame image are obtained; according to the interframe period and the target number of inserted frames, the corresponding interframe time in the interframe period is determined. Thus, the interframe time that needs to be inserted in the interframe period between the current frame image and the next frame image can be determined.

According to the first aspect, or any implementation of the first aspect above, the time differences between adjacent interpolation moments are equal.

Based on the above technical solution, by making the time difference between adjacent interpolation frame moments equal, the time intervals of the output interpolation frame images are consistent, thereby ensuring the smoothness of the output image.

According to the first aspect, or any implementation of the first aspect above, the movement trajectory includes a sub-movement trajectory from the first shooting moment to each of the interpolation frame moments, and the step of moving the moving pixel points in the current frame image according to the movement trajectory to generate at least one interpolation frame image includes:

Determine, according to the sub-movement trajectory, a target position of the moving pixel point at the interpolation time corresponding to the sub-movement trajectory;

The moving pixel point is moved to the target position to generate an insertion frame image corresponding to the insertion frame moment.

Based on the above technical solution, the corresponding inserted frame image is generated through the sub-movement trajectory from the first shooting moment of the standard camera to each of the inserted frame moments. Compared with the method of generating the inserted frame image through the motion vector and the neural network model, the accuracy of generating the inserted frame image in this embodiment is higher.

According to the first aspect, or any implementation of the first aspect above, before the step of obtaining the current frame image and inter-frame pixel point movement information, wherein the inter-frame pixel point movement information is the pixel point movement information between the current frame image and the next frame image, the step includes:

The target scene is photographed by a standard camera to obtain the current frame image;

The event-driven camera is used to detect pixel points in the imaging picture of the standard camera to obtain the pixel point movement information between frames.

Based on the above technical solution, after the target scene is photographed by a standard camera and the current frame image is obtained, the inter-frame pixel movement information in the inter-frame period between the first shooting moment of the current frame image and the second shooting moment of the next frame image is collected by an event-driven camera, thereby determining the movement of the pixels in the inter-frame period, so that the corresponding inserted frame image can be generated subsequently according to the inter-frame pixel movement information.

According to the first aspect, or any implementation of the first aspect above, the step of outputting the current frame image and the inserted frame image sequentially to a preset display screen includes:

Outputting the current frame image to a preset display screen, and obtaining the insertion frame time of the inserted frame image;

According to the sequence of the frame insertion moments, the inserted frame images are output to the preset display screen in sequence.

Based on the above technical solution, after obtaining the current frame image, this embodiment can output the current frame image to a preset display screen. Then, the insertion frame time of the inserted frame image is obtained, and according to the time sequence of the insertion frame time, the inserted frame images are sequentially output to the preset display screen to ensure the accuracy of the image output sequence.

In a second aspect, the present application provides an image interpolation device, the image interpolation device comprising:

An acquisition module, used to acquire the current frame image and inter-frame pixel point movement information, wherein the inter-frame pixel point movement information is the pixel point movement information between the current frame image and the next frame image;

A generating module, used for generating at least one inserted frame image according to the current frame image and the inter-frame pixel point movement information;

The output module is used to output the current frame image and the inserted frame image to a preset display screen in sequence.

According to the second aspect, the generating module is further used for:

Determine the moving pixels in the current frame image and the moving trajectory of the moving pixels according to the inter-frame pixel movement information;

According to the second aspect, or any implementation of the second aspect above, the generating module is further used to:

According to the second aspect, or any implementation of the second aspect above, the time differences between adjacent interpolation moments are equal.

According to the second aspect, or any implementation of the second aspect, the acquisition module is further used to:

The target scene is photographed by a standard camera to obtain a current frame image;

According to the second aspect, or any implementation of the second aspect above, the output module is further used to:

In a third aspect, the present application provides an image interpolation device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the image interpolation method as described above.

The third aspect and any implementation of the third aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the third aspect and any implementation of the third aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

In a fourth aspect, the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the processor executes an image interpolation method as described in any one of the above-mentioned first aspect or possible implementations of the first aspect.

The fourth aspect and any implementation of the fourth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fourth aspect and any implementation of the fourth aspect can refer to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, which will not be repeated here.

In a fifth aspect, an embodiment of the present application provides a computer program, which includes instructions for executing the image interpolation method in the first aspect and any possible implementation of the first aspect.

The fifth aspect and any implementation of the fifth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fifth aspect and any implementation of the fifth aspect can refer to the technical effects corresponding to the first aspect and any implementation of the first aspect, which will not be repeated here.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of a video perspective technology involved in an embodiment of the present application;

FIG2 is a schematic diagram of the structure of an image interpolation device in a hardware operating environment according to an embodiment of the present application;

FIG3 is a schematic diagram of a flow chart of a first embodiment of an image interpolation method of the present application;

FIG4 is a schematic diagram of a flow chart of a second embodiment of the image interpolation method of the present application;

FIG. 5 is a schematic diagram of the structure of an image interpolation device according to an embodiment of the present application.

The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings.

Detailed ways

The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

The term "and/or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and/or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

The terms "first" and "second" in the description and claims of the embodiments of the present application are used to distinguish different objects rather than to describe a specific order of objects. For example, a first target object and a second target object are used to distinguish different target objects rather than to describe a specific order of target objects.

In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

In the description of the embodiments of the present application, unless otherwise specified, the meaning of "multiple" refers to two or more than two. For example, multiple processing units refer to two or more processing units; multiple systems refer to two or more systems.

In order to make the description of the following embodiments clear and concise, a brief introduction to an implementation scheme of an image interpolation method is first given:

The main technical route for realizing MR at present is based on VST (Video See-Through) technology. Referring to Figure 1, Figure 1 is a schematic diagram of the video see-through technology involved in the embodiment of the present application. In VST technology, a real-time view of the real world is captured by a camera 10, and then the real-time view is combined with a virtual view of the digital world, and presented on a display screen 20 to send the combined image to the user's eyes. Based on VST technology, visual integration can be fully controlled, allowing complete occlusion between virtual and real objects, and even higher-level modifications to real objects.

The present application designs an image interpolation method for performing image compensation on the current frame image through pixel movement information, thereby improving the frame rate of the output image and ensuring the user's MR experience.

In some embodiments, by obtaining the current frame image and inter-frame pixel point movement information, wherein the inter-frame pixel point movement information is the pixel point movement information between the current frame image and the next frame image; generating at least one frame of inserted frame image according to the current frame image and the inter-frame pixel point movement information; and outputting the current frame image and the inserted frame image to a preset display screen in sequence. Thus, in this embodiment, image compensation is performed on the current frame image through the inter-frame pixel point movement information, and an inserted frame image after the current frame image is generated. After the current frame image and the inserted frame image are output to the preset display screen in sequence, the frame rate of the output image can be increased, thereby ensuring the user's MR experience.

Refer to Figure 2, which is a schematic diagram of the structure of an image interpolation device in the hardware operating environment involved in the embodiment of the present application.

Specifically, the image interpolation device can be an MR device, a PC (Personal Computer), a tablet computer, a portable computer or a server.

As shown in FIG. 2 , the image interpolation device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

Those skilled in the art will appreciate that the structure shown in FIG. 2 does not constitute a limitation on the image interpolation device, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

As shown in FIG. 2 , the memory 1005 as a storage medium may include an operating system, a data storage module, a network communication module, a user interface module, and a computer program.

In the image interpolation device shown in FIG2 , the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the image interpolation device of the present application can be set in the image interpolation device, and the image interpolation device calls the computer program stored in the memory 1005 through the processor 1001, and executes the image interpolation method provided in the embodiment of the present application.

It should be understood that the above description is merely an example listed for a better understanding of the technical solution of this embodiment, and is not intended to be the sole limitation to this embodiment.

The image interpolation method will be described in detail below with reference to the flowchart of the first embodiment of the image interpolation method shown in FIG. 3 .

Referring to FIG. 3 , an image interpolation method according to an embodiment of the present application is provided. The image interpolation method includes:

Step S100, obtaining the current frame image and inter-frame pixel point movement information, wherein the inter-frame pixel point movement information is the pixel point movement information between the current frame image and the next frame image;

In this embodiment, it can be understood that the current frame image is an image obtained by photographing the target scene with a standard camera, wherein the standard camera is a camera equipped with a conventional image sensor such as CIS (CMOS Image Sensor). The pixel point movement information at least includes the coordinate position of the pixel point in the imaging picture of the standard camera and the corresponding time point of the coordinate position, etc., collected by EVS (Event-based Vision Sensor) during the inter-frame period from the current frame image to the next frame image. It can be understood that an event-driven camera equipped with EVS can be used to collect the pixel point movement information.

Exemplarily, a standard camera using CIS and an event-driven camera using EVS may be provided in the MR device, so that when the standard camera shoots a target scene, the event-driven camera may collect inter-frame pixel movement information between adjacent frames.

Before the step S100 of acquiring the current frame image and the inter-frame pixel movement information, the following steps are included:

Step S110, photographing the target scene with a standard camera to obtain a current frame image;

Step S120: Detecting pixels in the imaging picture of the standard camera by using an event-driven camera to obtain inter-frame pixel movement information.

Specifically, the target scene is the scene that the user expects to shoot. In this embodiment, the target scene is photographed by a standard camera, and the current frame image corresponding to the target scene can be obtained. Since EVS is based on PD (Photo-Diode, photodiode) current, the current signal of the PD is monitored to see if it has changed. The transformation exceeds a given threshold, taking the 2-bit signal output as an example, that is, if the current signal becomes stronger, 01 can be output, and if the current signal becomes weaker, 10 can be output. If the current signal does not change beyond the threshold, the output is 00. At the same time, the calculation process is that all pixels are simultaneously converted to analog, which is a parallel process. And because the conversion is simple, this 2-bit digital-to-analog conversion is very fast, thereby realizing a very high-speed conversion and output of the entire pixel array, so the sampling frequency of the event-driven camera is much higher than that of the standard camera. Therefore, after obtaining the current frame image captured by the standard camera, the pixel points in the imaging screen of the standard camera can be detected by the event-driven camera, and the inter-frame pixel point movement information of the standard camera in the inter-frame period after the current frame image is captured to the next frame image can be collected.

In this embodiment, after photographing the target scene with a standard camera and obtaining the current frame image, the event-driven camera collects inter-frame pixel movement information in the inter-frame period between the first shooting moment of the current frame image and the second shooting moment of the next frame image, thereby determining the movement of the pixels in the inter-frame period, so that the corresponding inserted frame image can be generated subsequently according to the inter-frame pixel movement information.

Step S200, generating at least one inserted frame image according to the current frame image and the inter-frame pixel movement information;

After obtaining the current frame image and inter-frame pixel point movement information, the moving pixel points that have moved after the first shooting moment of the current frame image and the movement trajectory of the moving pixel points in the inter-frame period between the first shooting moment of the current frame image and the second shooting moment of the next frame image can be determined according to the inter-frame pixel point movement information. Then, the corresponding moving pixel points in the current frame image can be controlled to move according to the movement trajectory to generate at least one frame of inserted frame image for insertion between the current frame image and the next frame image to improve the frame rate of the output image.

Wherein, step S200 generates at least one inserted frame image according to the current frame image and the inter-frame pixel movement information, including:

Step S210, determining the moving pixels in the current frame image and the moving trajectory of the moving pixels according to the inter-frame pixel movement information;

Step S220 , moving pixels in the current frame image according to the moving trajectory to generate at least one inserted frame image.

It can be understood that EVS detects the changes of each pixel in each imaging screen in an asynchronous manner. Therefore, according to the inter-frame pixel movement information, it can determine the moving pixel points that move in the inter-frame period between the first shooting moment of the current frame image and the second shooting moment of the next frame image. And determine the coordinate position and corresponding time point of the moving pixel point in the inter-frame period, thereby generating a moving trajectory of the moving pixel point. Then, at least one frame of inserted frame image can be generated by moving the moving pixel points in the current frame image according to the moving trajectory. Exemplarily, if it is necessary to insert an inserted frame image between the current frame image and the next frame image, the coordinate position corresponding to a certain time point of the moving pixel point in the inter-frame period can be selected according to the moving trajectory, and the moving pixel point in the current frame image can be moved to the coordinate position, so that an inserted frame image can be generated.

In this embodiment, according to the inter-frame pixel movement information, the moving pixels in the current frame image and the moving trajectory of the moving pixels are determined; and according to the moving trajectory, the moving pixels in the current frame image are moved to generate at least one inserted frame image. Compared with the method of generating an inserted frame image by using motion vectors and a neural network model, the accuracy of generating an inserted frame image in this embodiment is higher.

Step S300: output the current frame image and the inserted frame image to a preset display screen in sequence.

After obtaining the current frame image and the inserted frame image, the current frame image and the inserted frame image are output to a preset display screen in sequence according to the corresponding moments of the current frame image and the inserted frame image, so as to increase the image frame rate output to the preset display screen.

Furthermore, step S300 of outputting the current frame image and the inserted frame image to a preset display screen in sequence includes:

Step S310, outputting the current frame image to a preset display screen, and obtaining the insertion frame time of the inserted frame image;

Step S320: output the inserted frame images in sequence to the preset display screen according to the sequence of the inserted frame moments.

After obtaining the current frame image, this embodiment can output the current frame image to a preset display screen, and then obtain the insertion frame time of the inserted frame image, and output the inserted frame images to the preset display screen in sequence according to the time sequence of the insertion frame time, so as to ensure the accuracy of the image output sequence.

In the first embodiment of the present application, by obtaining the current frame image and inter-frame pixel point movement information, wherein the inter-frame pixel point movement information is the pixel point movement information between the current frame image and the next frame image; generating at least one frame of inserted frame image according to the current frame image and the inter-frame pixel point movement information; and outputting the current frame image and the inserted frame image to a preset display screen in sequence. Thus, this embodiment performs image compensation on the current frame image through the inter-frame pixel point movement information, generates an inserted frame image after the current frame image, and after outputting the current frame image and the inserted frame image to the preset display screen in sequence, the frame rate of the output image can be increased, thereby ensuring the user's MR experience.

The image interpolation method will be described in detail below with reference to the flowchart diagram of the second embodiment of the image interpolation method shown in FIG. 4 .

Referring to FIG. 4 , in another embodiment of the image interpolation method of the present application, before the step of determining the moving pixels in the current frame image and the moving trajectory of the moving pixels according to the inter-frame pixel movement information, step S200 includes:

Step A10, obtaining an inter-frame period and a target number of inserted frames between a first shooting moment of the current frame image and a second shooting moment of the next frame image;

Step A20, determining the corresponding frame insertion time in the inter-frame period according to the inter-frame period and the target number of inserted frames.

Specifically, the target insertion frame number is the number of frames that need to be inserted between two adjacent frames of images. The target insertion frame number can be the insertion frame number input by the user, or the insertion frame number calculated according to the preset output frame number. For example, assuming that the image output frame rate of the standard camera is 60 frames per second (i.e., 60 images are output per second), and the preset output frame rate is 60 frames per second. The number of frames is 240 frames/second. Then it can be calculated that (240-60)/60=3 frames, that is, the target number of inserted frames to be inserted between two adjacent frames is 3 frames. The inter-frame period is the time difference between the first shooting moment of the current frame image and the second shooting moment of the next frame image, and the inter-frame period can be obtained according to the image output frame rate of the standard camera. Exemplarily, assuming that the image output frame rate of the standard camera is 60 frames/second, the inter-frame period is 1/60=16.6ms. Therefore, the corresponding interpolation moment in the inter-frame period can be determined according to the inter-frame period and the target number of inserted frames. Among them, the time difference between adjacent interpolation moments is equal, and the time difference between adjacent interpolation moments may also be unequal. Of course, it can be understood that the interpolation moment can also be set by the user according to needs.

In this embodiment, the interframe period and the target number of inserted frames between the first shooting moment of the current frame image and the second shooting moment of the next frame image are obtained; and the corresponding interframe insertion time in the interframe period is determined according to the interframe period and the target number of inserted frames. Thus, the interframe insertion time that needs to be inserted in the interframe period between the current frame image and the next frame image can be determined.

The time differences between adjacent interpolation frame moments are equal.

In this embodiment, by making the time differences between adjacent interpolation frame moments equal, the time intervals of the output interpolation frame images are consistent, thereby ensuring the smoothness of the output image.

The movement trajectory includes sub-movement trajectories from the first shooting moment to each of the interpolation frame moments, and step S220 moves the moving pixel points in the current frame image according to the movement trajectory to generate at least one interpolation frame image, including:

Step B10, determining the target position of the moving pixel point at the interpolation time corresponding to the sub-movement trajectory according to the sub-movement trajectory;

Step B20, moving the moving pixel point to the target position, and generating an insertion frame image corresponding to the insertion frame moment.

Specifically, the movement trajectory includes a sub-movement trajectory from the first shooting moment to each of the interpolation moments. In this embodiment, the target position of the moving pixel point at the interpolation moment corresponding to the sub-movement trajectory is determined according to the sub-movement trajectory, wherein the target position is the position of the moving pixel point on the sub-movement trajectory at the interpolation moment. Then, the interpolation frame image corresponding to the interpolation moment is generated by moving the moving pixel point to the target position.

In this embodiment, the corresponding inserted frame image is generated through the sub-movement trajectory from the first shooting moment of the standard camera to each of the inserted frame moments. Compared with the method of generating the inserted frame image through the motion vector and the neural network model, the accuracy of generating the inserted frame image in this embodiment is higher.

Refer to FIG. 5 , which is a schematic diagram of the structure of an image interpolation device involved in an embodiment of the present application.

Referring to FIG. 5 , the present application provides an image interpolation device, the image interpolation device comprising:

An acquisition module 10 is used to acquire the current frame image and inter-frame pixel point movement information, wherein the inter-frame pixel point movement information is the pixel point movement information between the current frame image and the next frame image;

A generating module 20, configured to generate at least one inserted frame image according to the current frame image and the inter-frame pixel point movement information;

The output module 30 is used to output the current frame image and the inserted frame image to a preset display screen in sequence.

Optionally, the generating module 20 is further configured to:

Optionally, or in any implementation of the second aspect above, the time differences between adjacent interpolation moments are equal.

Optionally, the generating module 20 is further configured to:

Optionally, the acquisition module 10 is further used for:

Optionally, the output module 30 is further used for:

It can be understood that the image interpolation device implements the operations in the image interpolation method provided in the above embodiment. The specific implementation steps can refer to the contents recorded in the above embodiment, and will not be elaborated here.

In addition, an embodiment of the present application further proposes a computer storage medium, on which a computer program is stored. When the computer program is executed by a processor, the operations in the image interpolation method provided in the above embodiment are implemented, and the specific steps are not repeated here.

It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

The serial numbers of the embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

An image interpolation method, characterized in that the image interpolation method comprises:

Acquire the current frame image and inter-frame pixel point movement information, wherein the inter-frame pixel point movement information is the pixel point movement information between the current frame image and the next frame image;

Generate at least one inserted frame image according to the current frame image and inter-frame pixel movement information;

The current frame image and the inserted frame image are outputted sequentially to a preset display screen.
The image interpolation method according to claim 1, wherein the step of generating at least one interpolation frame image according to the current frame image and the inter-frame pixel movement information comprises:

Determine the moving pixels in the current frame image and the moving trajectory of the moving pixels according to the inter-frame pixel movement information;

The moving pixels in the current frame image are moved according to the moving trajectory to generate at least one inserted frame image.
The image interpolation method according to claim 2, characterized in that before the step of determining the moving pixels in the current frame image and the moving trajectory of the moving pixels according to the inter-frame pixel movement information, it comprises:

Acquire an inter-frame period and a target number of inserted frames between a first shooting moment of the current frame image and a second shooting moment of the next frame image;

According to the inter-frame period and the target number of inserted frames, a corresponding inter-frame time point in the inter-frame period is determined.
The image interpolation method according to claim 3, wherein the time differences between adjacent interpolation moments are equal.
The image interpolation method according to claim 3, characterized in that the movement trajectory includes a sub-movement trajectory from the first shooting moment to each of the interpolation moments, and the step of moving the moving pixel points in the current frame image according to the movement trajectory to generate at least one interpolation frame image comprises:

Determine, according to the sub-movement trajectory, a target position of the moving pixel point at the interpolation time corresponding to the sub-movement trajectory;

The moving pixel point is moved to the target position to generate an insertion frame image corresponding to the insertion frame moment.
The image interpolation method according to claim 1, characterized in that before the step of obtaining the current frame image and inter-frame pixel point movement information, wherein the inter-frame pixel point movement information is the pixel point movement information between the current frame image and the next frame image, it comprises:

The target scene is photographed by a standard camera to obtain a current frame image;

The event-driven camera is used to detect pixel points in the imaging picture of the standard camera to obtain the pixel point movement information between frames.
The image interpolation method according to any one of claims 1 to 6, characterized in that the step of sequentially outputting the current frame image and the inserted frame image to a preset display screen comprises:

Outputting the current frame image to a preset display screen, and obtaining the insertion frame time of the inserted frame image;

According to the sequence of the frame insertion moments, the inserted frame images are output to the preset display screen in sequence.
An image interpolation device, characterized in that the image interpolation device comprises:

An acquisition module, used to acquire the current frame image and inter-frame pixel point movement information, wherein the inter-frame pixel point movement information is the pixel point movement information between the current frame image and the next frame image;

A generating module, used for generating at least one inserted frame image according to the current frame image and the inter-frame pixel point movement information;

The output module is used to output the current frame image and the inserted frame image to a preset display screen in sequence.
An image interpolation device, characterized in that the image interpolation device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the image interpolation method as described in any one of claims 1 to 7.
A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the image interpolation method according to any one of claims 1 to 7 are implemented.