WO2023123357A1

WO2023123357A1 - Image processing method and apparatus

Info

Publication number: WO2023123357A1
Application number: PCT/CN2021/143687
Authority: WO
Inventors: 杨铀; 蒋小广; 刘琼
Original assignee: Oppo广东移动通信有限公司
Priority date: 2021-12-31
Filing date: 2021-12-31
Publication date: 2023-07-06

Abstract

An image processing method and apparatus, which are applied to a decoder. The image processing method comprises: determining first position information of at least two planar regions of a multiplane image in a reference coordinate system; determining second position information of at least one light ray in the reference coordinate system; determining the coordinates, in the reference coordinate system, of at least one intersection point of the light ray and the at least two planar regions on the basis of the first position information and the second position information; and determining image information of at least one pixel point according to the coordinates of the at least one intersection point. The method can simplify the operation process of determining the coordinates of at least one intersection point of a light ray, which corresponds to a pixel point, and at least two planar regions, such that the complexity of a rendering process of the pixel point can be reduced.

Description

Method and device for image processing

technical field

The embodiments of the present application relate to the technical field of computer vision, and more specifically, to an image processing method and device.

Background technique

Multiplane image (MPI) is a common scene representation, that is, for a given reference viewpoint as the spatial coordinate system of the coordinate origin, MPI can decompose the scene into a series of plane layers or spherical layers. Take for example an MPI composed of planar layers that are positively parallel and located at different depths with respect to the reference viewpoint. In order to improve the sampling efficiency of MPI, the scene representation of depth adaptive change can be performed through Patch Multiplane Image (PMPI). PMPI introduces the depth information of the scene on the basis of MPI, increases the adaptive ability for the depth of the scene, and places more sampling points in the effective position of the scene. PMPI divides the plane into multiple regions, and each region can set a depth range according to the corresponding scene depth. In this way, the depth values of multi-planar images in different regions may be different.

For MPI, rendering can be performed according to reference camera parameters and target camera parameters to obtain a new view. For example, standard inverse homography (Standard inverse homography) and alpha synthesis can be used for rendering. However, for PMPI, since PMPI has multiple plane regions with different depth ranges compared to MPI, if the MPI rendering method is still used, the rendering process will be more complex.

Contents of the invention

Embodiments of the present application provide an image processing method and device, which can reduce the complexity of the pixel rendering process.

In the first aspect, a method for image processing is provided, the method is applied to a decoder, including:

Determine first position information of at least two plane regions of the multi-plane image in the reference coordinate system;

determining second position information of at least one ray in the reference coordinate system;

determining coordinates of at least one intersection point of the at least one ray with the at least two planar regions based on the first position information and the second position information;

Image information of at least one pixel point is determined according to the coordinates of the at least one intersection point.

In a second aspect, an image processing device is provided, which is applied to a decoder, including:

A first processing unit, configured to determine first position information of at least two plane regions of the multi-plane image in a reference coordinate system;

The first processing unit is further configured to determine second position information of at least one ray in the reference coordinate system;

The first processing unit is further configured to determine coordinates of at least one intersection point of the at least one ray and the at least two planar regions based on the first position information and the second position information;

The second processing unit is configured to determine image information of at least one pixel point according to the coordinates of the at least one intersection point.

In a third aspect, an electronic device is provided, including a processor and a memory. The memory is used to store a computer program, and the processor is used to invoke and run the computer program stored in the memory to execute the method in the first aspect above.

In a fourth aspect, a chip is provided, including: a processor, configured to call and run a computer program from a memory, so that a device installed with the chip executes the method in the first aspect above.

A fifth aspect provides a computer-readable storage medium for storing a computer program, and the computer program causes a computer to execute the method in the first aspect above.

In a sixth aspect, a computer program product is provided, including computer program instructions, the computer program instructions cause a computer to execute the method in the first aspect above.

In a seventh aspect, a computer program is provided, which, when running on a computer, causes the computer to execute the method of the first aspect above.

The embodiment of the present application realizes the simplified expression of at least two planar areas of the multi-plane image in the reference coordinate system through the first position information, and realizes the light corresponding to the pixel point in the first view through the second position information. Referring to the simplified expression in the coordinate system, during the rendering process of the pixel point, according to the first position information and the second position information, the calculation process of the coordinates of the light rays and at least one intersection point of the at least two planar regions can be simplified, Further, the complexity of the rendering process of the pixel is reduced.

Description of drawings

FIG. 1 is a schematic diagram of an encoding process of an encoder provided by an embodiment of the present application.

Fig. 2 is a schematic diagram of a decoding process of a decoder provided by an embodiment of the present application.

Figure 3 is an example of an MPI structure consisting of 4 planar layers.

Fig. 4 is a schematic diagram of standard inverse homography transformation.

Figure 5 shows several examples of MPI plane layers.

Fig. 6 is a schematic diagram of a scenario where an embodiment of the present application is applied.

Figure 7 is an example of PMPI divided by two grids.

FIG. 8 is a schematic diagram of the PMPI in FIG. 7 at a reference viewing angle.

Fig. 9 is a specific example of determining the initial depth value of the depth map.

Fig. 10 is a schematic flowchart of an image processing method provided by an embodiment of the present application.

Fig. 11 is an example of the intersection of the ray and the patch of PMPI in the embodiment of the present application.

Fig. 12 is a schematic block diagram of an image processing device provided by an embodiment of the present application.

Fig. 13 is a schematic block diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, but not all of the embodiments. With regard to the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts all belong to the scope of protection of this application.

First, the basic flow of video encoding and decoding under the applicable encoding and decoding framework of the embodiment of the present application will be described with reference to FIG. 1 and FIG. 2 .

FIG. 1 is a schematic diagram of an encoding process of an encoder provided by an embodiment of the present application. The encoder may be an MPI encoder of a test model of immersive video (Test model of immersive video, TMIV). Each plane layer in MPI can be divided into two parts: a color frame and a transparency frame. The color map contains the appearance texture of the scene at the position of the plane layer, and the transparency map contains the transparency information of the scene at the position of the plane layer. MPI's view parameters (view parameters), color map and transparency map can be used as the input of TMIV's MPI encoder after compression.

As shown in Figure 1, the input of the encoder can be source MPI, which includes view parameters (view parameters), texture attribute component (texture attribute component), and transparency attribute component (transparency attribute component). For the source MPI, a part of the video stream parameters can be directly obtained, such as a view parameter list (view parameters list) and a parameter set (parameters set), and steps 101 to 106 can also be performed to obtain another part of the encoded video stream parameters, such as the Set data (atlas data), texture attribute video data (texture attribute video data) and transparency attribute video data (transparency attribute video data).

Steps 101 to 106 are described below. Wherein,

steps

101 and 102 are mask processes (mask processes), steps 103 to 105 are atlas processes (atlas processes), and step 106 is video processes (video processes).

Step 101, create a mask from MPI (create mask from MPI). Masks can be used to cull invalid regions of the colormap and transparency map.

Step 102, aggregate MPI masks (aggregate MPI masks). Exemplarily, step 102 may aggregate MPI masks over a period of time.

Step 103, clustering effective pixels (cluster active pixels). Exemplarily, the mask can be divided into 8-connected regions through step 103 .

Step 104, split the cluster to reduce invalid pixels in the cluster and compress the data volume.

Step 105, pack patches (pack patches). In the package patch, atlas data can be obtained. Exemplarily, in step 105, the segmentation result obtained in step 104 may be packed into an atlas.

Step 106, generate video data (generate video data). Exemplarily, the atlas obtained in step 105 may be used to generate video data. Among the video data, texture attribute video data and transparency attribute video data can be obtained.

Fig. 2 is a schematic diagram of a decoding process of a decoder provided by an embodiment of the present application. The decoder may be an MPI decoder of TMIV. As shown in FIG. 2 , the decoder may include a decoded access unit (decoded access unit) 210 and a renderer 220 . Wherein, the decoding and accessing unit 210 can obtain parameter sets, view parameters, and atlases by decoding the acquired coded video stream. Exemplarily, the atlas may include an atlas parameter list, a patch parameter list, a block to patch map, texture video data and transparency Video data (transparency video data), etc.

The renderer 220 may take parameters in the decoding access unit 210 and viewport parameters as input. Exemplarily, the viewing angle parameter may be a viewing angle parameter of a playback device. The renderer 220 may execute steps 201 to 204 to obtain the viewing angle image under the viewing angle parameter, so as to obtain the playing video stream.

Steps 201 to 204 are described below.

Step 201, layer depth values decoding (layer depth values decoding) to reconstruct the MPI to obtain the layer depth values of each plane layer of the MPI.

Step 202, view synthesis. For example, rendering may be performed according to the MPI and the viewing angle parameter to obtain a new view, that is, a viewing angle image under the viewing angle parameter.

After the new view is obtained through

steps

201 and 202, the perspective image may be post-processed to enhance the image through the following steps 203 and 204, so as to obtain the final perspective image.

Step 203, inpainting.

Step 204, viewing space handling

It should be noted that the basic process of video encoding and decoding under the TMIV-based MPI encoding and decoding framework is described above with reference to FIG. 1 or FIG. For example, with the development of technology, some modules or steps of the framework or process may be optimized. During specific implementation, the technical solutions provided by the embodiments of the present application may be flexibly applied according to actual needs.

To facilitate a better understanding of the embodiments of the present application, the MPI involved in the embodiments of the present application will be described.

In a space coordinate system with a given reference viewpoint as the coordinate origin, MPI decomposes the scene into a series of plane layers or spherical layers. Take for example an MPI composed of planar layers that are positively parallel and located at different depths with respect to the reference viewpoint. FIG. 3 shows an example of an MPI structure composed of 4 plane layers, where the depth range [d _min ,d _max ] of the plane layer can be set in advance according to the scene depth data.

Each plane layer in MPI can be divided into two parts: color frame and transparency frame. The color frame contains the appearance texture of the scene at the position of the plane layer. For example, red green blue (red green) can be used blue, RGB) values; the transparency map contains the transparency information of the scene at the position of the plane layer, for example, it can be represented by α.

In order to better understand the embodiment of the present application, the rendering process of the decoding end involved in the embodiment of the present application will be described.

At the decoding end, rendering may be performed by MPI, reference camera parameters and target camera parameters to obtain a new view, for example, it may correspond to the processing procedure of step 202 in FIG. 2 . A current rendering method can obtain a new view through two steps of standard inverse homography and alpha synthesis. As an example, the standard inverse homography transformation can be shown in the following formula (1):

Among them, R and t are the rotation matrix and translation vector from the reference camera coordinate system to the target camera coordinate system in the world coordinate system; a is the opposite number of the plane depth value in MPI. n=(0,0,1) is the unit normal vector of the MPI plane in the reference camera coordinate system; k _s and k _t are the internal parameters of the reference camera and the target camera. (u _t ,v _t ,1) are the homogeneous coordinates of the pixels in the image in the target camera coordinate system; (u _s ,v _s ,1) are the homogeneous coordinates of the MPI point in the corresponding plane under the reference viewing angle.

For each pixel (u _t ,v _t ,1) in the target view, there is a corresponding point (u _si ,v _si , 1). Assuming that the number of planes of the MPI is D, each pixel point (u _t ,v _t ,1) has D corresponding pixel points in the MPI. FIG. 4 shows a schematic diagram of standard inverse homography transformation, where D=3. As shown in Figure 4, the pixel point (u _t , v _t ) has a corresponding point (u s1 , v s1 ) in the first plane from far to near in MPI, and a corresponding point (u _s1 , v _s1 ) in the second plane _s2 ,v _s2 ), there is a corresponding point (u _s3 ,v _s3 ) on the third plane.

After obtaining a series of MPI points (u _si ,v _si ,1) corresponding to the pixel point (u _t ,v _t ,1) according to the standard inverse homography transformation, the RGB value C at each MPI point can be obtained _i and the transparency α value α _i are calculated according to the following formula (2) to obtain the RGB value C _t of the pixel. This process is called alpha synthesis:

To facilitate a better understanding of the embodiments of the present application, the PMPI involved in the embodiments of the present application will be described.

In the codec architecture of TMIV, MPI is a non-redundant scene representation method. However, in real scenes, there are no visible surfaces in most spatial regions. The intuitive reflection is that most regions of the color map and transparency map in the MPI plane layer have invalid values, that is, they do not contain visible information. Referring to Fig. 5, several examples of MPI plane layers are shown, wherein (a) to (f) are respectively the 40th plane layer to the 45th plane layer, the first row of each plane layer is a color map, the second Behavioral transparency map, black is the invalid area.

As shown in Figure 5, if the MPI is regarded as a sampling of the scene, most of the sampling points in the MPI are located in invalid positions in the scene, and a very small part are located in the valid area of the scene. However, from the perspective of immersive experience, it is the effective area in the scene that plays a decisive role. Therefore, in the encoding architecture shown in Figure 1, when TMIV encodes MPI, it will remove the invalid areas of the color map and transparency map through the mask. This means that most of the sampling results of MPI are wasted, the sampling efficiency is low, and the final immersive video resolution is also low.

In order to improve the sampling efficiency of MPI, a scene representation with depth-adaptive change characteristics - Patch Multiplane Image (PMPI) is proposed. PMPI introduces the depth information of the scene on the basis of MPI, and aims to increase the adaptive ability for the depth of the scene, and place more sampling points in the effective position of the scene.

PMPI can be seen as an extension of MPI. The basic unit of MPI is the plane layer, and PMPI divides the plane into multiple regions, that is, the plane layer in MPI is regarded as a collection of multiple regions. The depth range [d _min , d _max ] of the MPI plane layer is set by the global depth of the scene (the depth range is enough to cover most of the effective information of the scene). Taking the simple scene in Figure 6 as an example, MPI must use a large depth range to represent the main information of the scene (four geometric bodies), and the resulting plane layer is relatively sparse. Apparently, for the three objects located in the foreground area, effective information will only appear on the two planes with larger depths.

In order to alleviate this situation, PMPI can be used, that is, the grid is used to divide the plane layer into an uncertain number (such as two or more) areas, and each area sets the depth range according to the corresponding scene depth. Fig. 7 shows an example of PMPI divided by two grids. In this example, the number of regions A is set to 2, each region contains 4 depth layers, the end depths of the 4 depth layers are set to be the same, and the starting depth is set by the scene depth of this region and its adjacent regions . Exemplarily, the black line in each region in FIG. 7 represents the position of the depth layer in the region. Fig. 8 shows a schematic diagram of the PMPI in Fig. 7 at a reference viewing angle, wherein each of the two regions of the PMPI includes 4 sub-planes (patch), and a patch can be a square sub-plane, not done limited.

The above is just an example to show the difference between PMPI and MPI. In actual use, the number of PMPI areas can be set to M×N, that is, the scene is divided into M×N grids, where M can represent the number of areas divided into plane layers, and N can represent the depth layer in each area quantity. Among them, M and N can be set according to factors such as the complexity of the scene. After M and N are determined, the initial depth value of each area in PMPI is determined according to the depth map under the reference viewpoint. The specific process is as follows:

a). Divide the depth map into M×N regions using an M×N grid;

b). For each region of the depth map, its depth value is replaced by the minimum value in the K×K neighborhood of the region to reduce the error of the depth value of the region, where K is a positive odd number. For example, K×K minimum pooling in units of grid regions can be used. In order to make the grid number of the pooled depth map still M×N, the original depth map grid can be copied and filled, and the pooling step size is set to 1;

c). The depth value of each region obtained in steps a) and b) is used as the initial depth value of the PMPI corresponding region.

FIG. 9 shows a specific example of determining the initial depth value of the depth map, where the pooling size is 5×5. As shown in Figure 9, the grid number of the original depth map is 6×6. First, the grid of the original depth map can be copied and filled, and the grid number of the filled depth map is 10×10, and then the pooling size is 5×5, and the step size is 1 for minimum pooling, and the pooled depth map is obtained, and the number of grids is the same as that of the original depth map. At this time, the depth value of each region of the pooled depth map may be used as the depth value of each region. It should be understood that the grid number and pooling size in FIG. 9 are for example only, and do not limit the embodiment of the present application.

After the above processing, the initial depth values of the M×N regions of the PMPI can be determined. The depth range end value dmax of each region in PMPI is the same as MPI and can be given in advance. Optionally, in each region, the number of depths is the same, and can be distributed according to the same law, such as equidistant distribution or equal parallax distribution, which is not limited.

In order to better understand the embodiments of the present application, the technical problems to be solved in the embodiments of the present application are described.

As can be seen from the above description, there is a big difference in structure between PMPI and MPI. Compared with MPI, PMPI has multiple plane areas with different depth ranges, which means that the rendering method of MPI is no longer applicable to the rendering of PMPI. That is to say, if the rendering method of MPI is used for rendering of PMPI, that is, the standard inverse homography transformation and alpha synthesis are used to render PMPI, which will lead to a high complexity of the rendering process.

Specifically, in the process of the standard reverse homography transformation described by formula (1), it is necessary to calculate once for each sub-plane (patch) in each region of the PMPI. Taking PMPI with the number of regions as M×N as an example, compared with MPI with the same depth number, the complexity of PMPI calculation is M×N times that of MPI. In actual use, the rendering complexity of PMPI is hundreds or thousands of times that of MPI rendering.

In view of this, the embodiments of the present application provide an image processing method and device, and when the method is used for rendering a multi-plane image in a decoder, the complexity of the rendering process can be reduced.

Specifically, in this embodiment of the present application, by determining the first position information of at least two plane regions of the multi-plane image under the reference coordinate system, and the second position information of at least one ray under the reference coordinate system, and then according to the first position information a position information and a second position information, determine the coordinates of at least one intersection point of the at least one ray and the at least two planar areas in the reference coordinate system, and finally determine according to the coordinates of the at least one intersection point in the reference coordinate system Image information of at least one pixel.

Exemplarily, for a PMPI having a plurality of planar regions with different depth ranges, through the solution of the embodiment of the present application, the light corresponding to each patch in the PMPI and the pixel point can be determined through simple addition and subtraction operations and judgment Intersect, instead of using the standard inverse homography transformation to determine whether the ray intersects the patch, which can greatly simplify the calculation process of determining the coordinates of the intersection point of the ray corresponding to the pixel and the patch in PMPI, thereby reducing the rendering of the pixel the complexity of the process.

The image processing method provided by the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

FIG. 10 shows a schematic flowchart of an image processing method 400 provided by an embodiment of the present application. The method 400 may be applied to a decoder, such as the decoder in FIG. 2 , or a decoding process involved in the decoder. Further, the method 400 may be used for view synthesis of the renderer 220 in the decoder in FIG. 2 . As shown in FIG. 10 , method 400 includes steps 410 to 440 .

410. Determine first position information of at least two plane regions of a multi-plane image in a reference coordinate system.

As an example, the multi-plane image may be a PMPI, which corresponds to at least two plane areas in each area after the at least two plane areas are divided into at least two areas. For example, for the PMPI in FIG. 8 , the at least two plane areas include each sub-plane (patch) in 2 areas, ie a total of 8 plane areas.

In some optional embodiments, when the multi-plane image is PMPI, before step 410, the multi-plane layer may also be divided into at least two regions, wherein the scene depth ranges of the at least two regions are different; then According to the scene depth ranges of the at least two areas, planar areas are respectively acquired in the at least two areas as the at least two planar areas.

Exemplarily, according to the specific scene, the plane layer can be divided into at least two regions with a grid, and the depth range of each region can be set according to the corresponding scene, for example, the starting depth can be determined by the region or the adjacent region of the region Depth of scene setting. Then, in each area, at least two sub-planes can be obtained according to the scene depth range of the area, and the sub-planes in all areas can form the above-mentioned at least two plane areas. As a specific example, reference may be made to the process of obtaining multiple patches of PMPI in FIG. 7 to FIG. 9 above, which will not be repeated here.

As another example, the multi-plane image may be an MPI, and the corresponding at least two plane areas may be each plane area of the MPI. For example, for the MPI in FIG. 3 , the at least two plane areas include a plane area corresponding to each plane layer in the four plane layers.

In some embodiments, the first position information of the at least two plane regions of the multi-plane image under the reference coordinate system may also be described as the first position of the at least two plane regions of the multi-plane image under the reference viewing angle information, without limitation.

In some embodiments, the reference coordinate system may also be referred to as the reference camera coordinate system, and the two have the same or similar meanings.

In some optional embodiments, the first position information includes coordinates of the geometric center of the planar area in the reference coordinate system and size information of the planar area.

Exemplarily, for each patch in the PMPI, that is, each sub-plane in each region, the first position information of the patch may include the coordinates of the geometric center of the patch in the reference coordinate system and each side of the patch side length. As an example, the sub-plane may be a square, a rectangle, or other shapes, which are not limited.

For example, when the sub-plane is a square, the first position information of the patch can be expressed as a vector (x _p , y _p , d _p , r _p ), where (x _p , y _p , d _p ) represents the patch's The coordinates of the geometric center in the reference coordinate system, x _p , y _p , and d _p are the coordinates of the x, y, and z axes respectively, d _p can represent the depth value of the patch, and r _p can represent the patch in the reference coordinate system half the length of the lower side. Here, the unit normal vector of the patch is fixed to (0,0,1).

It can be understood that, in the embodiment of the present application, the first position information, such as the position coordinates and size information of the geometric center of the plane area, can realize the simplified expression of at least two plane areas of the multi-plane image in the reference coordinate system.

420. Determine second position information of at least one ray in the reference coordinate system.

Exemplarily, the at least one ray may include a ray corresponding to at least one pixel in at least one view (such as a first view, or a target view), and the first view may be the target view. Wherein, one pixel point in each first view may correspond to one ray.

Exemplarily, the at least one ray includes a ray whose starting point is the target camera and passes through the at least one pixel. In the following, a pixel in a first view corresponds to a ray as an example. The ray corresponding to the pixel may be a ray whose starting point is the target camera and passes through the pixel. In some embodiments, the target coordinate system may also be referred to as the target camera coordinate system, and both have the same or similar meanings.

In some optional embodiments, the second position information includes the coordinates and direction vector of the starting point of the ray (under the reference coordinate system) corresponding to the pixel point, where the coordinates of the starting point are the coordinates of the target camera at the reference coordinates Coordinates under the system.

Exemplarily, for each pixel in the first view, its corresponding ray can be expressed as a vector (x ₀ , y ₀ , z ₀ , x _d , y _d ), where (x ₀ , y ₀ , z ₀ ) represents the coordinates of the starting point of the ray, that is, the coordinates of the target camera in the reference coordinate system, and (x _d , y _d , 1) represents the direction vector of the ray in the reference coordinate system.

As an example, the direction vector of the ray in the reference coordinate system can be obtained according to the direction vector of the ray in the target coordinate system and the mapping relationship between the target coordinate system and the reference coordinate system, which is not limited in the present application.

It can be understood that, the embodiment of the present application uses the second position information, such as the starting point and direction information of the ray corresponding to the pixel, to realize the simplified expression of the ray corresponding to the pixel in the view in the reference coordinate system.

430. Based on the first position information and the second position information, determine coordinates in the reference coordinate system of at least one intersection point of the at least one ray and the at least two planar regions.

In the following, a pixel corresponding to a ray in a first view is taken as an example for description.

In some optional embodiments, the intersection point of the ray corresponding to the pixel point and the plane where at least two plane regions of the multi-plane image are located can be determined according to the first position information and the second position information, and then it is judged that the ray and each Whether the intersection point of the plane where the plane area is located is on the plane area. When the intersection point is on the corresponding plane area, it can be determined that the ray intersects the plane area, and then the coordinates of the intersection point in the reference coordinate system can be obtained; when the intersection point is not on the corresponding plane area, it can be determined that the ray does not intersect the plane area intersect.

Exemplarily, the process of judging whether a ray intersects a plane area here may be referred to as an algorithm for judging whether a ray intersects a plane area, which is not limited in this application.

As a possible implementation, according to the above first position information and the second position information, the first intersection point of the first plane where the first plane area of the at least two plane areas is located and the above ray can be calculated in the reference coordinate system Next, according to the first coordinate and the boundary position of the first plane area, it is determined that the first intersection point is on the first plane area, and then the first coordinate can be determined as the ray and the first plane area The coordinates of the intersection point in the reference coordinate system.

Exemplarily, when the reference coordinate system is the XOY coordinate system, according to the first coordinate and the boundary position of the first plane area, determining that the first intersection point is on the first plane area may be achieved in the following manner:

Determine that the X coordinate in the first coordinate is greater than or equal to the left boundary of the first plane area on the X axis;

Determine that the X coordinate in the first coordinate is smaller than the right boundary of the first plane area on the X axis;

Determine that the Y coordinate in the first coordinate is smaller than the upper boundary of the first plane area on the Y axis;

It is determined that the Y coordinate in the first coordinate is greater than or equal to the lower boundary of the first plane area on the Y axis.

When the above conditions are met, that is, the X coordinate in the first coordinate is between the left boundary of the first plane area on the X axis and the border, and the Y coordinate is between the upper boundary and the lower boundary of the Y axis of the first plane area, Then it can be determined that the first intersection point is on the first plane area.

As a specific example, the x-coordinate range of a point in the first plane area is [x1, x2), where x1 represents the left boundary of the first plane area on the x-axis, x2 represents the right boundary of the first plane area on the x-axis, The y-coordinate range of points in the first plane area is [y1, y2), where y1 represents the lower boundary of the first plane area on the y-axis, and y2 represents the upper boundary of the first plane area on the y-axis, where x1, x2, y1 and y2 are real numbers, and x1 is smaller than x2, and y1 is smaller than y2. When the first coordinates (x, y) satisfy x1≤x<x2 and y1≤y<y2, it is determined that the first intersection point is on the first plane area.

As another possible implementation, the reference coordinates of the second intersection point of the second plane where the second plane area of the at least two plane areas is located and the light ray can be calculated according to the first position information and the second position information. Then, according to the second coordinates and the boundary position of the second plane area, it is determined that the second intersection point is not on the first plane area, that is, it can be determined that the ray does not intersect the second plane area.

Exemplarily, when the reference coordinate system is the XOY coordinate system, according to the second coordinates and the boundary position of the second plane area, it is determined that the second intersection point is not on the first plane area, which may be implemented in the following manner:

Determining that the X coordinate in the second coordinate is smaller than the left boundary of the second planar area on the X axis; and/or

determining that the X coordinate in the second coordinate is greater than or equal to the right boundary of the second planar area on the X axis; and/or

Determining that the Y coordinate in the second coordinate is greater than or equal to the upper boundary of the second planar area on the Y axis; and/or

It is determined that the Y coordinate in the second coordinate is smaller than the lower boundary of the second plane area on the Y axis.

When one of the above conditions is met, that is, the X coordinate in the second coordinate is not between the left boundary of the X axis and the border of the second plane area, or the Y coordinate is between the upper boundary and the lower boundary of the Y axis of the second plane area , it can be determined that the second intersection point is not on the second plane area.

As a specific example, the x-coordinate range of the point in the second plane area is [x3, x4), where x3 represents the left boundary of the second plane area on the x-axis, x4 represents the right boundary of the second plane area on the x-axis, The y-coordinate range of the points in the second plane area is [y3, y4), where y3 represents the lower boundary of the second plane area on the y-axis, and y4 represents the upper boundary of the second plane area on the y-axis, where x3, x4, y3 and y4 are real numbers, and x3 is smaller than x4, and y3 is smaller than y4. When the second coordinate (x', y') satisfies at least one item of x'<x3, or x'≥x4, or y'<y3, or y'≥y4, it is determined that the second intersection point is not in the second plane area superior.

As a specific example, the coordinates of the intersection point of the pixel point and at least one plane area in the reference coordinate system may be determined according to the following steps a) to g).

a). Calculate the X coordinate x _cross of the intersection point of the light corresponding to the pixel point and the plane where the first patch is located:

x _cross ＝x ₀ +(d _p -z ₀ )×x _d

b). Calculate the left boundary x _left =x _p -r _p of the first patch on the X axis, and when x _cross ≥ x _left , proceed to the subsequent steps. Otherwise, when x _cross < x _left , it is determined that the ray does not intersect with the first patch.

c). Calculate the right boundary of the first patch on the X axis x _right =x _p +r _p , and when x _cross < x _right , proceed to the subsequent steps. Otherwise, when x _cross ≥ x _left , it is determined that the ray does not intersect the first patch.

d). Calculate the Y coordinate y _cross of the intersection point between the ray and the plane where the first patch is located:

y _cross ＝y ₀ +(d _p -z ₀ )×y _d

e). Calculate the upper boundary of the first patch on the Y axis y _up =y _p +r _p , and when y _cross <y _up , proceed to the subsequent steps. Otherwise, when y _cross ≥ y _up , it is determined that the ray does not intersect with the first patch.

f). Calculate the lower boundary of the first patch on the Y axis y _down =y _p -r _p , and when y _cross >y _down , proceed to the subsequent steps. Otherwise, when y _cross < y _up , it is determined that the ray does not intersect with the first patch.

g). Store the coordinates (x _cross , y _cross , d _p ) of the intersection point of the ray and the first patch in the reference coordinate system.

Therefore, in the embodiment of the present application, based on the above-mentioned first position information and second position information, only simple addition and subtraction operations and judgments are required to determine that the light corresponding to the pixel point intersects with at least one intersection point of the at least two planar regions within the reference The coordinates in the coordinate system, that is, the embodiment of the present application can simplify the calculation process of the coordinates of at least one intersection point of the ray corresponding to the pixel point and the at least two planar regions.

For example, in steps a) to g), no more than 4 times of addition and subtraction operations and judgments are required to determine whether the first patch intersects the light corresponding to the pixel. However, if the standard inverse homography transformation described above is used to determine whether the ray intersects the plane area, then it is necessary to run the formula (1) once and several judgment statements (to determine whether the intersection point is within the boundary of the patch), the formula (1) It is a complex matrix operation that takes a long time to calculate.

Fig. 11 shows an example where the ray corresponding to the pixel intersects with the three patches of PMPI. As shown in Figure 11, the PMPI includes 8 patches. If the standard inverse homography is used to judge whether the ray intersects with the plane area, then the formula (1) needs to be calculated 8 times to find 3 intersection points (black dots in the figure) , and if the scheme provided by the embodiment of the present application is adopted, the above steps a) to g) are run 8 times (each operation only needs no more than 4 addition and subtraction operations and judgment statements), and these 3 can be found intersection. Therefore, the embodiment of the present application can greatly simplify the calculation process of determining the coordinates of the intersection of the ray corresponding to the pixel point and the patch in the PMPI.

In some optional embodiments, the at least two plane areas may be arranged according to the depth values of the at least two plane areas, and then based on the first position information and the second position information, the pixel points corresponding to Coordinates of at least one intersection point of the ray and the at least two planar areas in the reference coordinate system, so as to obtain coordinates of at least one intersection point arranged in sequence according to depth values.

Exemplarily, the at least two plane regions may be arranged in ascending or descending order according to the depth values of the at least two plane regions, which is not limited in the present application.

440. Determine image information of at least one pixel point according to the coordinates of the at least one intersection point.

Exemplarily, exemplary, at least one pixel point may be a pixel point in the above view (for example, the first view or the target view). For example, image information of pixels (for example, all pixels) in the first view is rendered to the first view to obtain a new view.

Therefore, the embodiment of the present application realizes the simplified expression of at least two plane regions of the multi-plane image in the reference coordinate system through the first position information, and realizes the light corresponding to the pixel point in the first view through the second position information Simplified expression in the reference coordinate system, so that in the rendering process of the pixel point, according to the first position information and the second position information, the calculation of the coordinates of the light rays and at least one intersection point of the at least two planar regions can be simplified process, thereby reducing the complexity of the rendering process of the pixel.

As a possible implementation, it is possible to sample the plane image where the at least one intersection point is located to obtain at least one image information of the plane image where the pixel is located at the at least one intersection point, and then according to the at least one image information and the The transparency information of the plane image where the at least one intersection point is located determines the image information of the pixel point.

Exemplarily, the image information of each intersection point in the plane image of the at least one intersection point may include RGB values, and the transparency information of each intersection point in the plane image is, for example, the α value mentioned above. That is to say, after obtaining a series of intersection points (x _cross , y _cross , d _p ) corresponding to the pixel points through the above step 430, the RGB value and transparency value α of the series of intersection points at each plane image can be obtained, Therefore, alpha synthesis can be performed according to the above formula (2), and the RBG value of the pixel can be obtained as the image information of the pixel. Optionally, when alpha compositing is performed, the series of intersection points are arranged according to their depth values at each plane image.

In some optional embodiments, some or all of the above steps 410 to 440 may be deployed on a GPU for parallel computing, so as to utilize the parallel acceleration feature of the GPU to greatly improve the rendering efficiency of pixels. Exemplarily, the above steps a) to g) and step 440 may be deployed on the GPU, which is not limited in this application.

Exemplarily, the embodiment of the present application measures the process of rendering 2000 pixels in a PMPI with a region number of 33×40 and a depth number of 40. Under the same hardware conditions, standard inverse homography transformation and alpha synthesis are used. The time spent on rendering is 383.480 milliseconds, and the time spent on rendering is 36.022 milliseconds by using the algorithm for judging whether a ray intersects a plane area and alpha compositing provided by the embodiment of the present application. It can be seen that the method provided by the embodiment of the present application can increase the speed of the pixel-marking rendering process by more than 10 times.

Therefore, through the scheme of the embodiment of the present application, through simple addition and subtraction and judgment, it can be determined whether each patch in the PMPI intersects with the light corresponding to the pixel point, without using standard reverse homography transformation to judge the light Whether it intersects with the patch can greatly simplify the calculation process of determining the coordinates of the intersection point of the ray corresponding to the pixel and the patch in the PMPI, thereby reducing the complexity of the rendering process of the pixel.

The specific implementation of the application has been described in detail above in conjunction with the accompanying drawings. However, the application is not limited to the specific details in the above-mentioned implementation. Within the scope of the technical concept of the application, various simple modifications can be made to the technical solution of the application. These simple modifications all belong to the protection scope of the present application. For example, the various specific technical features described in the above specific implementation manners can be combined in any suitable manner if there is no contradiction. Separately. As another example, any combination of various implementations of the present application can also be made, as long as they do not violate the idea of the present application, they should also be regarded as the content disclosed in the present application.

It should also be understood that, in various method embodiments of the present application, the sequence numbers of the above-mentioned processes do not mean the order of execution, and the order of execution of the processes should be determined by their functions and internal logic, and should not be used in this application. The implementation of the examples constitutes no limitation. It is to be understood that these ordinal numbers may be interchanged under appropriate circumstances such that the described embodiments of the application can be practiced in sequences other than those illustrated or described.

The method embodiment of the present application is described in detail above, and the device embodiment of the present application is described in detail below in conjunction with FIG. 12 to FIG. 13 .

FIG. 12 is a schematic block diagram of an image processing apparatus 700 according to an embodiment of the present application. The apparatus 700 is, for example, the decoder in FIG. 2 . As shown in FIG. 12 , the device 700 may include a first processing unit 710 and a second processing unit 720 .

The first processing unit 710 is configured to determine first position information of at least two plane regions of the multi-plane image in the reference coordinate system;

The first processing unit 710 is further configured to determine second position information of at least one ray in the reference coordinate system;

The first processing unit 710 is further configured to determine coordinates of at least one intersection point of the at least one ray and the at least two planar regions based on the first position information and the second position information;

The second processing unit 720 is configured to determine image information of at least one pixel point according to the coordinates of the at least one intersection point.

Optionally, the second processing unit 720 is specifically configured to:

Sampling the plane image where the at least one intersection point is located respectively, to obtain at least one image information of the plane image where the at least one pixel point is located in the at least one intersection point;

The image information of the at least one pixel point is determined according to the at least one image information and the transparency information of the plane image where the at least one intersection point is located.

Optionally, the first processing unit 710 is specifically configured to:

According to the first position information and the second position information, calculate the first coordinates of the first intersection point of the first plane where the first plane area of the at least two plane areas is located and the at least one ray;

determining that the first intersection point is on the first plane area according to the first coordinates and the boundary position of the first plane area;

The first coordinate is determined as a coordinate of an intersection point of the at least one ray and the first plane area.

Optionally, the first processing unit 710 is specifically used for:

determining that the X coordinate in the first coordinate is greater than or equal to the left boundary of the first plane area on the X axis;

determining that the X coordinate in the first coordinate is smaller than the right boundary of the first plane area on the X axis;

determining that the Y coordinate in the first coordinate is smaller than the upper boundary of the first plane area on the Y axis;

Optionally, the first processing unit 710 is specifically configured to:

calculating second coordinates of a second intersection point of a second plane where a second plane area of the at least two plane areas is located and the at least one ray according to the first position information and the second position information;

determining that the second intersection point is not on the first plane area according to the second coordinates and the boundary position of the second plane area;

It is determined that the at least one ray does not intersect the second planar region.

Optionally, the first processing unit 710 is specifically configured to:

determining that the X coordinate in the second coordinates is greater than or equal to the right boundary of the second planar region on the X axis; and/or

Optionally, the first processing unit 710 is specifically configured to:

arranging the at least two planar regions according to the depth values of the at least two planar regions;

Based on the first position information and the second position information, sequentially determine the coordinates of at least one intersection point of the at least one ray and the at least two planar regions.

Optionally, the first processing unit 710 is also used for:

Dividing the multi-plane layer corresponding to the multi-plane image into at least two regions, wherein the scene depth ranges of the at least two regions are different;

According to scene depth ranges of the at least two areas, planar areas are respectively acquired in the at least two areas as the at least two planar areas.

Optionally, the first position information includes coordinates of the geometric center of the planar area and size information of the planar area.

Optionally, the at least one ray includes a ray corresponding to at least one pixel in at least one view, and the at least one ray includes a ray whose starting point is the target camera and passes through the at least one pixel.

Optionally, the second position information includes the coordinates and direction vector of the starting point of the ray.

It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, details are not repeated here. Specifically, in this embodiment, the device 700 may correspond to the corresponding subject that executes the method 400 of the embodiment of the present application, and the aforementioned and other operations and/or functions of the modules in the device 700 are for realizing each method in FIG. 4 , or the corresponding process in the method in FIG. 4 , for the sake of brevity, details are not repeated here.

The device and system of the embodiments of the present application are described above from the perspective of functional modules with reference to the accompanying drawings. It should be understood that the functional modules may be implemented in the form of hardware, may also be implemented by instructions in the form of software, and may also be implemented by a combination of hardware and software modules. Specifically, each step of the method embodiment in the embodiment of the present application can be completed by an integrated logic circuit of the hardware in the processor and/or instructions in the form of software, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as hardware The execution of the decoding processor is completed, or the combination of hardware and software modules in the decoding processor is used to complete the execution. Optionally, the software module may be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, and registers. The storage medium is located in the memory, and the processor reads the information in the memory, and completes the steps in the above method embodiments in combination with its hardware.

FIG. 13 is a schematic block diagram of an electronic device 800 provided by an embodiment of the present application.

As shown in Figure 13, the electronic device 800 may include:

A memory 810 and a processor 820 , the memory 810 is used to store computer programs and transmit the program codes to the processor 820 . In other words, the processor 820 can invoke and run a computer program from the memory 810, so as to implement the method in the embodiment of the present application.

For example, the processor 820 may be configured to execute the steps in the above-mentioned method 400 according to instructions in the computer program.

In some embodiments of the present application, the processor 820 may include but not limited to:

General-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), field programmable gate arrays (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates Or transistor logic devices, discrete hardware components, and so on.

In some embodiments of the present application, the memory 810 includes but is not limited to:

volatile memory and/or non-volatile memory. Among them, the non-volatile memory can be read-only memory (Read-Only Memory, ROM), programmable read-only memory (Programmable ROM, PROM), erasable programmable read-only memory (Erasable PROM, EPROM), electronically programmable Erase Programmable Read-Only Memory (Electrically EPROM, EEPROM) or Flash. The volatile memory can be Random Access Memory (RAM), which acts as external cache memory. By way of illustration and not limitation, many forms of RAM are available, such as Static Random Access Memory (Static RAM, SRAM), Dynamic Random Access Memory (Dynamic RAM, DRAM), Synchronous Dynamic Random Access Memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous connection dynamic random access memory (synch link DRAM, SLDRAM) and Direct Memory Bus Random Access Memory (Direct Rambus RAM, DR RAM).

In some embodiments of the present application, the computer program can be divided into one or more modules, and the one or more modules are stored in the memory 810 and executed by the processor 820 to complete the method. The one or more modules may be a series of computer program instruction segments capable of accomplishing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device 800 .

Optionally, as shown in FIG. 8, the electronic device 800 may further include:

Transceiver 830 , the transceiver 830 can be connected to the processor 820 or the memory 810 .

Wherein, the processor 820 can control the transceiver 830 to communicate with other devices, specifically, can send information or data to other devices, or receive information or data sent by other devices. Transceiver 830 may include a transmitter and a receiver. The transceiver 830 may further include an antenna, and the number of antennas may be one or more.

It should be understood that various components in the electronic device 800 are connected through a bus system, wherein the bus system includes a power bus, a control bus and a status signal bus in addition to a data bus.

According to one aspect of the present application, an electronic device is provided, including a processor and a memory, the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory, so that the electronic device executes The method of the above method embodiment.

According to one aspect of the present application, a computer storage medium is provided, on which a computer program is stored, and when the computer program is executed by a computer, the computer can execute the methods of the above method embodiments. In other words, the embodiments of the present application further provide a computer program product including instructions, and when the instructions are executed by a computer, the computer executes the methods of the foregoing method embodiments.

According to another aspect of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method of the above method embodiment.

In other words, when implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable device. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, e.g. (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or a data center integrated with one or more available media. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, or a magnetic tape), an optical medium (such as a digital video disc (digital video disc, DVD)), or a semiconductor medium (such as a solid state disk (solid state disk, SSD)), etc.

It should be understood that in this embodiment of the present application, "B corresponding to A" means that B is associated with A. In one implementation, B may be determined from A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B may also be determined according to A and/or other information.

In the description of the present application, unless otherwise specified, "at least one" means one or more, and "plurality" means two or more than two. In addition, "and/or" describes the association relationship of associated objects, indicating that there may be three types of relationships, for example, A and/or B, which may indicate: A exists alone, A and B exist simultaneously, and B exists alone, among them A and B can be singular or plural. The character "/" generally indicates that the contextual objects are an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one item (piece) of a, b, or c can represent: a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, c can be single or multiple .

It should also be understood that the first, second, etc. descriptions appearing in the embodiments of the present application are only used to illustrate and distinguish the description objects, and there is no order, nor does it represent a special limitation on the number of devices in the embodiments of the present application. It cannot be construed as any limitation to the embodiments of the present application.

It should also be understood that a particular feature, structure, or characteristic described in the specification in relation to an embodiment is included in at least one embodiment of the present application. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner in one or more embodiments.

Furthermore, the terms "comprising" and "having", as well as any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or server comprising a series of steps or elements is not necessarily limited to the expressly listed instead, may include other steps or elements not explicitly listed or inherent to the process, method, product or apparatus.

Those skilled in the art can appreciate that the modules and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Skilled artisans may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present application.

In the several embodiments provided in this application, it should be understood that the disclosed devices, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or components can be combined or can be Integrate into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or modules may be in electrical, mechanical or other forms.

A module described as a separate component may or may not be physically separated, and a component displayed as a module may or may not be a physical module, that is, it may be located in one place, or may also be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. For example, each functional module in each embodiment of the present application may be integrated into one processing module, each module may exist separately physically, or two or more modules may be integrated into one module.

The above is only the specific implementation of the application, but the scope of protection of the application is not limited thereto. Anyone familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the application, and should be covered Within the protection scope of this application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

A method for image processing, applied to a decoder, is characterized in that, comprising:

Determine first position information of at least two plane regions of the multi-plane image in the reference coordinate system;

determining second position information of at least one ray in the reference coordinate system;

determining coordinates of at least one intersection point of the at least one ray with the at least two planar regions based on the first position information and the second position information;

Image information of at least one pixel point is determined according to the coordinates of the at least one intersection point.
The method according to claim 1, wherein the determining the image information of the at least one pixel point according to the coordinates of the at least one intersection point comprises:

Sampling the plane image where the at least one intersection point is located respectively, to obtain at least one image information of the plane image where the at least one pixel point is located in the at least one intersection point;

The image information of the at least one pixel point is determined according to the at least one image information and the transparency information of the plane image where the at least one intersection point is located.
The method according to claim 1 or 2, wherein, based on the first position information and the second position information, at least one intersection point between the at least one ray and the at least two planar regions is determined coordinates, including:

According to the first position information and the second position information, calculate the first coordinates of the first intersection point of the first plane where the first plane area of the at least two plane areas is located and the at least one ray;

determining that the first intersection point is on the first plane area according to the first coordinates and the boundary position of the first plane area;

The first coordinate is determined as a coordinate of an intersection point of the at least one ray and the first plane area.
The method according to claim 3, wherein the determining that the first intersection point is on the first plane area according to the first coordinates and the boundary position of the first plane area comprises:

determining that the X coordinate in the first coordinate is greater than or equal to the left boundary of the first plane area on the X axis;

determining that the X coordinate in the first coordinate is smaller than the right boundary of the first plane area on the X axis;

determining that the Y coordinate in the first coordinate is smaller than the upper boundary of the first plane area on the Y axis;

It is determined that the Y coordinate in the first coordinate is greater than or equal to the lower boundary of the first plane area on the Y axis.
The method according to claim 3, further comprising:

calculating second coordinates of a second intersection point of a second plane where a second plane area of the at least two plane areas is located and the at least one ray according to the first position information and the second position information;

determining that the second intersection point is not on the first plane area according to the second coordinates and the boundary position of the second plane area;

It is determined that the at least one ray does not intersect the second planar region.
The method according to claim 5, wherein the determining that the second intersection point is not on the first plane area according to the second coordinates and the boundary position of the second plane area comprises:

determining that the X coordinate in the second coordinate is smaller than the left boundary of the second planar area on the X axis; and/or

determining that the X coordinate in the second coordinates is greater than or equal to the right boundary of the second planar region on the X axis; and/or

determining that the Y coordinate in the second coordinate is greater than or equal to the upper boundary of the second planar area on the Y axis; and/or

It is determined that the Y coordinate in the second coordinate is smaller than the lower boundary of the second plane area on the Y axis.
The method according to any one of claims 1-6, wherein, based on the first position information and the second position information, determining the distance between the at least one ray and the at least two planar regions Coordinates of at least one point of intersection, including:

arranging the at least two planar regions according to the depth values of the at least two planar regions;

Based on the first position information and the second position information, sequentially determine the coordinates of at least one intersection point of the at least one ray and the at least two planar regions.
The method according to any one of claims 1-7, further comprising:

Dividing the multi-plane layer corresponding to the multi-plane image into at least two regions, wherein the scene depth ranges of the at least two regions are different;

According to scene depth ranges of the at least two areas, planar areas are respectively acquired in the at least two areas as the at least two planar areas.
The method according to any one of claims 1-8, wherein the first position information includes coordinates of a geometric center of the plane area and size information of the plane area.
The method according to any one of claims 1-9, wherein the at least one ray includes a ray corresponding to at least one pixel point in at least one view, and the at least one ray includes a starting point that is the target camera and passes through Light rays of the at least one pixel point.
The method according to any one of claims 1-10, wherein the second position information includes coordinates and a direction vector of a starting point of the ray.
A device for image processing, applied to a decoder, is characterized in that it comprises:

A first processing unit, configured to determine first position information of at least two plane regions of the multi-plane image in a reference coordinate system;

The first processing unit is further configured to determine second position information of at least one ray in the reference coordinate system;

The first processing unit is further configured to determine coordinates of at least one intersection point of the at least one ray and the at least two planar regions based on the first position information and the second position information;

The second processing unit is configured to determine the image information of the at least one pixel point according to the coordinates of the at least one intersection point.
The device according to claim 12, wherein the second processing unit is specifically configured to:

Sampling the plane image where the at least one intersection point is located respectively, to obtain at least one image information of the plane image where the at least one pixel point is located in the at least one intersection point;

The image information of the at least one pixel point is determined according to the at least one image information and the transparency information of the plane image where the at least one intersection point is located.
The device according to claim 12 or 13, wherein the first processing unit is specifically used for:

According to the first position information and the second position information, calculate the first coordinates of the first intersection point of the first plane where the first plane area of the at least two plane areas is located and the at least one ray;

determining that the first intersection point is on the first plane area according to the first coordinates and the boundary position of the first plane area;

The first coordinate is determined as a coordinate of an intersection point of the at least one ray and the first plane area.
The device according to claim 14, wherein the first processing unit is specifically used for:

determining that the X coordinate in the first coordinate is greater than or equal to the left boundary of the first plane area on the X axis;

determining that the X coordinate in the first coordinate is smaller than the right boundary of the first plane area on the X axis;

determining that the Y coordinate in the first coordinate is smaller than the upper boundary of the first plane area on the Y axis;

It is determined that the Y coordinate in the first coordinate is greater than or equal to the lower boundary of the first plane area on the Y axis.
The device according to claim 14, wherein the first processing unit is specifically configured to:

calculating second coordinates of a second intersection point of a second plane where a second plane area of the at least two plane areas is located and the at least one ray according to the first position information and the second position information;

determining that the second intersection point is not on the first plane area according to the second coordinates and the boundary position of the second plane area;

It is determined that the at least one ray does not intersect the second planar region.
The device according to claim 16, wherein the first processing unit is specifically configured to:

determining that the X coordinate in the second coordinate is smaller than the left boundary of the second planar area on the X axis; and/or

determining that the X coordinate in the second coordinates is greater than or equal to the right boundary of the second planar region on the X axis; and/or

determining that the Y coordinate in the second coordinate is greater than or equal to the upper boundary of the second planar area on the Y axis; and/or

It is determined that the Y coordinate in the second coordinate is smaller than the lower boundary of the second plane area on the Y axis.
The device according to any one of claims 12-17, wherein the first processing unit is specifically configured to:

arranging the at least two planar regions according to the depth values of the at least two planar regions;

Based on the first position information and the second position information, sequentially determine the coordinates of at least one intersection point of the at least one ray and the at least two planar regions.
The device according to any one of claims 12-18, wherein the first processing unit is further configured to:

Dividing the multi-plane layer corresponding to the multi-plane image into at least two regions, wherein the scene depth ranges of the at least two regions are different;

According to scene depth ranges of the at least two areas, planar areas are respectively acquired in the at least two areas as the at least two planar areas.
The device according to any one of claims 12-19, wherein the first position information includes coordinates of a geometric center of the plane area and size information of the plane area.
The device according to any one of claims 12-20, wherein the at least one ray includes a ray corresponding to at least one pixel point in at least one view, and the at least one ray includes a starting point that is the target camera and passes through Light rays of the at least one pixel point.
The device according to any one of claims 12-21, wherein the second position information includes coordinates and a direction vector of a starting point of the ray.
An electronic device, characterized in that it includes a processor and a memory, and instructions are stored in the memory, and when the processor executes the instructions, the processor executes the method described in any one of claims 1-12. method.
A computer storage medium, characterized in that it is used to store a computer program, and the computer program includes a method for executing the method according to any one of claims 1-12.
A computer program product, characterized by comprising computer program code, when the computer program code is executed, the method according to any one of claims 1-11 is executed.