CN118014828A - Image stitching method, device and system for array camera - Google Patents

Abstract

The present application relates to an image stitching method, apparatus, hundred million-level pixel computing imaging system, computer device, storage medium and computer program product for an array camera. The method comprises the following steps: establishing an initial image stitching model based on the local sample image shot by the array camera; adopting initial image transformation parameters and current clipping parameters of the local camera, and carrying out image transformation and clipping on blank contrast images corresponding to the local camera to obtain current images to be stitched; judging whether an invalid pixel area exists in each current image to be stitched, determining the current image to be stitched with the invalid pixel area as a target image to be stitched, and updating current cutting parameters according to global position information of a target inscribed rectangular area of the valid pixel area in the target image to be stitched; and obtaining a target image splicing model based on the updated clipping parameters. By adopting the method, the imaging quality of the fusion video can be improved, and development resources can be saved.

Description

Image stitching method, device and system for array camera

Technical Field

The present application relates to the field of image processing technology, and in particular to an image stitching method and device for array cameras, a billion-pixel computational imaging system, a computer device, a storage medium, and a computer program product.

Background technique

Video surveillance has been widely used in the security field due to its advantages of high reliability, strong timeliness, and easy viewing. When the monitoring range is large, in order to achieve ultra-high-definition video monitoring of large scenes, a billion-pixel computational imaging system based on array cameras has emerged. The array camera contains multiple local cameras with long focal lengths and narrow fields of view to shoot multiple local high-definition videos of the monitoring scene, and then the local videos are spliced and fused to obtain ultra-high-definition fused videos with large or wide fields of view up to billion pixels. It is suitable for video shooting and security monitoring of large scenes such as airports, highways, parks, sports stadiums, borders, and sea surfaces.

In the related art, in order to solve the arc imaging problem caused by factors such as lens distortion when shooting straight targets with a wide field of view (such as airport runways), a structurally optimized array camera has emerged. The image sensor of each local camera in the array camera is rotated around the axis by a certain angle, and the larger the horizontal angle between the axis of the local camera and the global center axis of the array camera, the larger the corresponding rotation angle, thereby improving the arc imaging problem of straight targets and optimizing the imaging quality of straight shooting targets.

However, since the image sensors of each local camera are rotated by a certain angle, the effective picture taken is changed. If the image stitching model construction method used for the array camera with the rotating structure is used to construct the image stitching model, invalid pixel areas are likely to appear in the fused video image obtained by stitching, such as the black triangular area at the boundary or the stitching (usually the invalid pixel area uses the pixel value 0, so it is black). That is, the obtained fused video image is not a complete and effective picture, resulting in poor imaging quality of the fused video.

Summary of the invention

Based on this, it is necessary to provide an image stitching method, device, billion-pixel computational imaging system, computer equipment, storage medium and computer program product for array cameras to address the above-mentioned technical problems, which can improve the imaging quality of fused video, save development resources and improve development efficiency, and is particularly suitable for array cameras with rotating structures.

In a first aspect, the present application provides an image stitching method for an array camera. The method comprises:

Acquire local sample images taken by each local camera of the array camera, and establish an initial image stitching model based on each local sample image; the initial image stitching model includes initial image transformation parameters and initial cropping parameters corresponding to each local camera;

Using the initial cropping parameters as current cropping parameters, for each of the local cameras, using the initial image transformation parameters of the local camera and the current cropping parameters, performing image transformation and cropping on the blank control image corresponding to the local camera, to obtain a current image to be stitched; wherein the blank control image has the same size as the local sample image, and the pixel value of each pixel in the blank control image is different from the pixel value of the invalid pixel;

According to the pixel value of each pixel point in each of the current images to be stitched, it is judged whether there is an invalid pixel area in each of the current images to be stitched, the current images to be stitched with the invalid pixel area are determined as the target images to be stitched, and the target inscribed rectangular area of the valid pixel area in the target images to be stitched is determined;

According to the global position information of the target inscribed rectangular area, the current cropping parameters of the target local camera corresponding to the target image to be stitched and the associated local camera of the target local camera are updated;

A target image stitching model is obtained based on the updated cropping parameters, and the target image stitching model is used to perform stitching and fusion processing on each local video image captured by the array camera.

In one embodiment, the associated local camera includes a local camera in the same line; the current cropping parameter includes a current upper boundary cropping parameter and a current lower boundary cropping parameter; the global position information of the target inscribed rectangular area includes the upper boundary ordinate of the area and the lower boundary ordinate of the area; and updating the current cropping parameters of the target local camera corresponding to the target image to be stitched and the associated local camera of the target local camera according to the global position information of the target inscribed rectangular area includes:

According to the upper boundary ordinate and the lower boundary ordinate of each target inscribed rectangular area in the row where the target local camera is located, the maximum upper boundary ordinate and the minimum lower boundary ordinate of the row are determined, and the current upper boundary cropping parameters and the current lower boundary cropping parameters of each local camera in the row are updated to the maximum upper boundary ordinate and the minimum lower boundary ordinate, respectively.

In one embodiment, the associated local camera includes a left neighboring local camera and a right neighboring local camera; the current cropping parameters include current left boundary cropping parameters and current right boundary cropping parameters; the global position information of the target inscribed rectangular area includes the horizontal coordinate of the left boundary of the area and the horizontal coordinate of the right boundary of the area; and updating the current cropping parameters of the target local camera corresponding to the target image to be stitched and the associated local camera of the target local camera according to the global position information of the target inscribed rectangular area includes:

For each of the target local cameras, the difference between the horizontal coordinate of the left boundary of the region of the target local camera and the current left boundary clipping parameter is calculated to obtain a current left boundary difference, and the difference between the horizontal coordinate of the right boundary of the region of the target local camera and the current right boundary clipping parameter is calculated to obtain a current right boundary difference;

When the current left boundary difference is greater than a preset threshold, the current left boundary clipping parameter of the target local camera and the current right boundary clipping parameter of the left neighboring local camera of the target local camera are updated to the horizontal coordinate of the left boundary of the region of the target local camera;

When the current right boundary difference is greater than a preset threshold, the current right boundary cropping parameter of the target local camera and the current left boundary cropping parameter of the local camera to the right of the target local camera are updated to the horizontal coordinate of the right boundary of the area of the target local camera.

In one embodiment, obtaining a target image stitching model based on the updated cropping parameters includes:

Using the updated cropping parameters as new current cropping parameters, returning to execute, for each of the local cameras, the steps of image transformation and cropping the blank control image corresponding to the local camera using the initial image transformation parameters and the current cropping parameters of the local camera;

After determining whether there is an invalid pixel area in each of the current images to be stitched according to the pixel value of each pixel point in each of the current images to be stitched, the method further includes:

In the case that there is no invalid pixel area in each of the current images to be stitched, the current cropping parameters are used as target cropping parameters to obtain a target image stitching model.

In one embodiment, after updating the current cropping parameters of the target local camera corresponding to the target image to be stitched and the local camera associated with the target local camera, the method further includes:

Record the current update times;

After determining whether there is an invalid pixel area in each of the current images to be stitched according to the pixel value of each pixel point in each of the current images to be stitched, the method further includes:

In the case that at least one of the current images to be stitched has an invalid pixel region and the current update number is less than a preset value, performing a step of determining the current image to be stitched having the invalid pixel region as a target image to be stitched;

When at least one of the current images to be stitched has an invalid pixel region and the current update number is greater than or equal to a preset value, the current image to be stitched having the invalid pixel region is determined as the image to be corrected, and the local camera corresponding to the image is the local camera to be corrected;

According to the global position information of the invalid pixel points contained in the image to be corrected, the current upper boundary cropping parameters and/or the current lower boundary cropping parameters of the local camera to be corrected and the local camera in the same row as the local camera to be corrected are corrected, and the corrected cropping parameters are used as target cropping parameters to obtain a target image stitching model.

In one of the embodiments, the correcting of the current upper boundary cropping parameters and/or the current lower boundary cropping parameters of the local camera to be corrected and the local camera in the same row of the local camera to be corrected according to the global position information of the invalid pixel points contained in the image to be corrected includes:

Traverse the first column of pixels and the last column of pixels of the image to be corrected from top to bottom respectively. If the current pixel is an invalid pixel, search for the next pixel until the current pixel is a valid pixel, stop traversal, and use the ordinate of the current pixel as a candidate upper boundary clipping parameter;

Traverse the first column of pixels and the last column of pixels of the image to be corrected from bottom to top respectively, if the current pixel is an invalid pixel, search for the next pixel until the current pixel is a valid pixel, stop traversal, and use the ordinate of the current pixel as a candidate lower boundary cropping parameter;

According to the maximum value of each candidate upper boundary cropping parameter and the minimum value of each candidate lower boundary cropping parameter, the current upper boundary cropping parameter and the current lower boundary cropping parameter of the local camera to be corrected and the local camera in the same row as the local camera to be corrected are corrected.

In a second aspect, the present application also provides an image stitching device for an array camera. The device comprises:

Establishing a module for acquiring local sample images taken by each local camera of the array camera, and establishing an initial image stitching model based on each local sample image; the initial image stitching model includes initial image transformation parameters and initial cropping parameters corresponding to each local camera;

A cropping module, configured to use the initial cropping parameters as current cropping parameters, and for each of the local cameras, use the initial image transformation parameters of the local camera and the current cropping parameters to perform image transformation and cropping on the blank control image corresponding to the local camera, so as to obtain a current image to be stitched; wherein the blank control image has the same size as the local sample image, and the pixel value of each pixel in the blank control image is different from the pixel value of the invalid pixel;

A first determination module is used to determine whether there is an invalid pixel area in each of the current images to be stitched according to the pixel value of each pixel point in each of the current images to be stitched, determine the current image to be stitched with the invalid pixel area as the target image to be stitched, and determine a target inscribed rectangular area of a valid pixel area in the target image to be stitched;

An updating module is used to update the current cropping parameters of the target local camera corresponding to the target image to be stitched and the associated local camera of the target local camera according to the global position information of the inscribed rectangular area of the target, and obtain a target image stitching model based on the updated cropping parameters, and the target image stitching model is used to perform stitching and fusion processing on the local video images taken by the array camera.

In a third aspect, the present application also provides a billion-pixel computational imaging system. The system includes an array camera and a server, wherein:

The array camera is used to capture local sample images through each local camera, and send each local sample image to the server;

The server is used to establish an initial image stitching model based on each of the local sample images; the initial image stitching model includes initial image transformation parameters and initial cropping parameters corresponding to each of the local cameras; and the initial cropping parameters are used as current cropping parameters;

The server is also used for performing image transformation and cropping on the blank control image corresponding to each local camera using the initial image transformation parameters and the current cropping parameters of the local camera to obtain a current image to be stitched; wherein the blank control image has the same size as the local sample image, and the pixel value of each pixel in the blank control image is different from the pixel value of the invalid pixel; judging whether there is an invalid pixel area in each current image to be stitched according to the pixel value of each pixel in each current image to be stitched, determining the current image to be stitched with the invalid pixel area as the target image to be stitched, and determining the target inscribed rectangular area of the valid pixel area in the target image to be stitched; updating the current cropping parameters of the target local camera corresponding to the target image to be stitched and the associated local camera of the target local camera according to the global position information of the target inscribed rectangular area; obtaining a target image stitching model based on the updated cropping parameters;

The array camera is also used to capture local video images and send them to the server;

The server is also used to perform splicing and fusion processing on each of the local video images based on the target image splicing model to obtain a fused video image, and send the fused video image to a display terminal.

In a fourth aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method described in the first aspect are implemented.

In a fifth aspect, the present application further provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

In a sixth aspect, the present application further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

The above-mentioned image stitching method, device, billion-pixel computational imaging system, computer equipment, storage medium and computer program product for array cameras adopt the existing image stitching model construction method to establish the initial image stitching model of the array camera, and then perform image transformation and cropping on the blank reference image through the initial image transformation parameters and initial cropping parameters of each local camera to obtain the image to be stitched. Since the pixel value of each pixel in the blank reference image is different from the pixel value of the invalid pixel, the valid pixel area and the invalid pixel area in the image to be stitched can be quickly and accurately determined according to the pixel value of each pixel in the image to be stitched, and then the cropping parameters of the target local camera and its associated local camera are updated according to the global position information of the target inscribed rectangle of the valid pixel area in the target image to be stitched where the invalid pixel area exists. Among them, the pixels in the target inscribed rectangle area are all valid pixels, and the cropping parameters are updated based on the inscribed rectangle area, which can reduce or eliminate the invalid pixel area in the fused video image and improve the picture integrity and imaging quality of the fused video image. Moreover, this method only needs to simply update the parameters of the image stitching model to improve the adaptability of the target image stitching model. There is no need to make complex modifications to the image stitching model construction algorithm, which improves the reusability of the existing image stitching model construction algorithm and effectively saves development resources.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of a billion-pixel computational imaging system in an example;

FIG2 is a schematic diagram of a flow chart of an image stitching method in one embodiment;

FIG3 a is a front view of an array camera in one example;

FIG3b is a top view of the main internal structure of the array camera shown in FIG3a;

FIG3c is a schematic diagram of the positions of the array camera shown in FIG3a for capturing images;

FIG4a is a schematic diagram of a transformed image corresponding to a blank control image in an example;

FIG4 b is a schematic diagram of images to be stitched in an example;

FIG4c is a schematic diagram of an image to be stitched and a target inscribed rectangular area in an example;

FIG5 is a structural block diagram of an image stitching device in one embodiment;

FIG6 is a diagram showing the internal structure of a computer device in one embodiment;

FIG7 a is a fused image obtained based on the initial image stitching model in an example;

FIG. 7 b is a fused image obtained based on a target image stitching model in an example.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

First, before specifically introducing the technical solution of the embodiment of the present application, the technical background or technical evolution context on which the embodiment of the present application is based is introduced. In order to achieve ultra-high-definition video surveillance with large scenes and wide fields of view, a billion-pixel computational imaging system based on array cameras has emerged. The array camera usually includes multiple local cameras with different shooting angles, and the local cameras shoot local high-definition video images with rich detail information of the target scene, and then stitch and fuse the local high-definition video images to obtain a fused video image with a large field of view or a wide field of view. In order to improve the generation efficiency of the fused video, usually before shooting the video of the target scene, an image stitching model can be constructed based on the local sample images shot by the local camera, so that when shooting the video of the target scene, the image stitching model can be used to stitch and fuse the multiple local high-definition videos shot by each local camera, and output ultra-high-definition fused videos with a large field of view or a wide field of view in real time, which can reach the billion-pixel level, and are suitable for video shooting and security monitoring of large scenes such as airports, highways, parks, sports stadiums, borders, and sea surfaces.

Among them, the construction process of the image stitching model can be to extract feature points from the overlapping areas of each local video image, and then perform image registration based on the feature points to determine the image transformation parameters corresponding to each local camera (such as homography transformation parameters, rotation/translation/scaling/affine transformation parameters, etc.), so as to transform each local video image to the same plane (unified perspective, unified coordinates). For example, the plane where one of the local video images is located can be designated as the reference plane, and based on the image transformation parameters corresponding to each local camera, each local video image is transformed to the reference plane. Then, the stitching line position and the cropping parameters of each local camera can be determined. The cropping parameters can specify the position to be cropped in the transformed image of each local video image, and each cropped transformed image (which can be called the image to be stitched) is spliced, fused or stitched to obtain a fused image.

The algorithms involved in the above-mentioned process of constructing the image stitching model, such as feature point pair extraction, image registration, and cropping parameter determination, need to be designed and debugged for the array camera so that the constructed image stitching model has a high degree of match with the array camera and can be used to stitch together a fused video image with high imaging quality. If the structure of the array camera is optimized or improved, the previously designed image stitching model construction algorithm may no longer be applicable, resulting in poor image stitching results.

For example, in order to solve the arc imaging problem caused by factors such as lens distortion when shooting a straight target with a wide field of view (such as an airport runway), an array camera with improved structure (hereinafter referred to as an array camera with a rotating structure) has been developed. The image sensor of each local camera in the array camera is rotated around the axis by a certain angle, and the larger the horizontal angle between the axis of the local camera and the global center axis of the array camera, the larger the corresponding rotation angle, thereby improving the arc imaging problem of the straight target and optimizing the imaging quality of the straight target. Since the image sensor of each local camera is rotated by a certain angle, the effective picture taken is changed. Based on the image stitching model construction algorithm designed for the non-rotated array camera (traditional array camera), the fitness of the constructed image stitching model is low, and invalid areas are likely to appear in the obtained fused video image. As shown in FIG7a, a black border area appears at the boundary of the fused image and a black triangle area appears at the splicing (usually the area of the non-effective pixel point adopts the pixel value 0, so it is displayed in black), that is, the obtained fused video image is not a complete and effective picture, resulting in poor imaging quality of the fused video.

However, it is relatively difficult to design or update the image stitching model building algorithm. If each structurally optimized array camera is to be redesigned or updated, more development resources will be consumed, and the more versions of the model building algorithm there are, the more likely it is to cause configuration errors.

Based on this background, the applicant has proposed the image stitching method for array cameras of this application through long-term research and development and experimental verification. By performing simple parameter updates on the basis of the initial image stitching model obtained by the existing image stitching model construction method, the adaptability of the image stitching model can be improved, thereby improving the imaging quality of the fused video. This method is particularly suitable for image stitching of array cameras with rotating structures. This method does not require complex modifications to the construction algorithm of the image stitching model, thereby improving the reusability of the stitching model construction algorithm and saving development resources. In addition, it should be noted that the applicant has made a lot of creative efforts in discovering the technical problem of this application and the technical solutions introduced in the following embodiments.

The image stitching method provided in the embodiment of the present application can be applied to the billion-pixel computational imaging system as shown in FIG1. The array camera 102 and the display terminal 106 communicate with the server 104 through the network respectively. The display terminal 106 can be, but is not limited to, various electronic devices with display units such as personal computers, laptops, smart phones, tablet computers, smart displays, Internet of Things devices, portable wearable devices, etc. The server 104 can be implemented as an independent server or a server cluster composed of multiple servers.

In one embodiment, as shown in FIG2 , an image stitching method is provided, which can be applied to the server in FIG1 . In this embodiment, the method includes the following steps:

Step 201 : obtaining local sample images taken by each local camera of the array camera, and establishing an initial image stitching model based on each local sample image.

The initial image stitching model includes initial image transformation parameters and initial cropping parameters corresponding to each local camera.

In implementation, the server can use the existing image stitching model construction algorithm to construct an image stitching model, which usually includes feature point extraction of local sample images (overlapping areas), image registration, determination of stitching lines and cropping positions, etc. Feature point extraction can be implemented using methods such as scale invariant feature transform (SIFT), accelerated robust features (SURF), etc. Image registration can be performed through machine learning to find the mapping relationship between adjacent local sample images based on the extracted feature points, usually by finding the optimal image transformation parameters so that the position difference of the same feature points in adjacent local sample images after transformation is minimized. Then, the cropping position is determined in the transformed image to determine the boundary of the cropped image (corresponding to the following image to be stitched). Among them, in the two adjacent cropped images on the left and right, the right boundary of the left image and the left boundary of the right image are connected, and the cropped images are spliced, fused or stitched to obtain a large-format ultra-high-definition fused image.

Each cropped image is usually rectangular, and the cropping parameters can specify the position information of the rectangular area (position information in the transformed unified coordinate system, which can be called global position information), such as the coordinates of the upper left corner pixel and the lower right corner pixel of the rectangular area (u ₀ ,v ₀ ,u ₁ ,v ₁ ). Among them, u ₀ is the horizontal coordinate of the upper left corner pixel (the number of columns where the pixel is located), which can specify the left boundary of the rectangular area, which can be called the left boundary cropping parameter; v ₀ is the vertical coordinate of the upper left corner pixel (the number of rows where the pixel is located), which can specify the upper boundary of the rectangular area, which can be called the upper boundary cropping parameter. Correspondingly, u ₁ and v ₁ are the horizontal coordinate and vertical coordinate of the lower right corner pixel, which can be called the right boundary cropping parameter and the lower boundary cropping parameter respectively.

It should be noted that, in practical applications, when performing image cropping based on the cropping parameters (u ₀ ,v ₀ ,u ₁ ,v ₁ ), the principle of left closing and right opening is followed, so the horizontal coordinate of the last column of pixels in the image actually cropped is (u ₁ -1). The horizontal coordinate of the first column of pixels in the right adjacent cropped image is u ₁ . That is, in the cropping parameters of two adjacent local cameras, the right boundary cropping parameters of the left camera and the left boundary cropping parameters of the right camera are the same. In some other examples, if the rectangular cropping area in the initial image stitching model is represented by the upper left corner pixel of the area and the width (w) and height (h) of the rectangular area, such as (u ₀ ,v ₀ ,w,h), the parameter can be converted to (u ₀ ,v ₀ ,u ₁ ,v ₁ ) as the initial cropping parameter for subsequent processing.

The shooting angles of each local camera of the array camera are different, and each local sample image can be arranged in multiple rows and columns or in a single row and multiple columns. In one example, the array camera can be an array camera of a rotating structure, that is, the image sensor of each local camera is rotated around the axis of the local camera by a certain angle (the image sensor is installed at this rotation angle), and the size of the rotation angle is positively correlated with the horizontal angle of the axis of the local camera. As shown in Figures 3a and 3b, the rotating structure array camera includes 10 local cameras (A1 to A10) with different horizontal shooting angles and the same or similar vertical shooting angles, so the local sample images taken are arranged in a row (as shown in Figure 3c), and there are overlapping areas between adjacent local sample images (for ease of distinction, adjacent local images are represented by solid line frames and dotted line frames, respectively). It can be understood that Figure 3c is only used to show the relative position relationship of each local sample image, and the shape of the actual overlapping area is not the rectangle shown in the figure.

In the array camera, the horizontal angle of the axis of each local camera refers to the angle in the horizontal direction (the angle of the horizontal projection line) between the axis of the local camera (such as the axis A shown in Figure 3b) and the global central axis of the array camera (such as the axis L shown in Figure 3b).

Step 202 , using the initial cropping parameters as current cropping parameters, for each local camera, using the initial image transformation parameters and current cropping parameters of the local camera, performing image transformation and cropping on the blank control image corresponding to the local camera, to obtain the current image to be stitched.

The blank control image has the same size as the local sample image, and the pixel value of each pixel in the blank control image is different from the pixel value of the invalid pixel. The blank control image can be a pure color image (such as a pixel value of 255) or a multi-color image, and the pixel value of each pixel is different from the pixel value of the invalid pixel (such as 0).

Usually, the pixel values of invalid pixels are defined in the image stitching model, that is, in the rectangular transformed image, except for the valid pixels corresponding to the pixels in the local sample image, the other pixels (that is, invalid pixels) are filled with a uniform pixel value, which is generally set to 0 pixel value to avoid empty pixels.

It can be understood that if the image taken by one of the local cameras is used as the reference image when constructing the image stitching model (that is, the other images are transformed to the plane where the reference image is located), then the blank control image corresponding to the local camera is transformed by the corresponding image transformation parameters and is consistent with the original image, or no transformation processing is required, and the original image is directly used as the transformed image, so that the transformed image will not contain invalid pixels.

Step 203, judging whether there is an invalid pixel area in each current image to be stitched according to the pixel value of each pixel point in each current image to be stitched, determining the current image to be stitched with the invalid pixel area as the target image to be stitched, and determining the target inscribed rectangular area of the valid pixel area in the target image to be stitched.

In implementation, since the pixel value of each pixel in the blank control image is different from the pixel value of the invalid pixel, the server can traverse each pixel in the image to be stitched, compare its pixel value with the pixel value of the invalid pixel, or compare it with the pixel value of the valid pixel (i.e., the pixel value of each pixel in the blank control image), and judge whether the pixel is a valid pixel or an invalid pixel. If there are a certain number of invalid pixels in the image to be stitched, that is, the total number of invalid pixels is greater than a preset threshold (the threshold can be 0 or greater than 0), or in the connected domain composed of invalid pixels, the size of at least one connected domain is greater than the preset threshold (such as the number of invalid pixels in the connected domain is greater than the preset threshold), it can be judged that there is an invalid pixel area in the image to be stitched.

If the fitness of the image stitching model is high, after the transformed image is cropped, there will be no or only a small number of invalid pixels in the image to be stitched, so that an image with a complete and valid picture can be stitched. However, for the improved rotating structure array camera, the effective picture taken by it changes. When the initial image stitching model obtained by the existing non-rotating structure array camera is used to transform and crop the image, invalid pixel areas will appear in the obtained part of the image to be stitched.

If there are invalid pixel areas in multiple images to be stitched, the server can determine the target inscribed rectangular area of the valid pixel area in each target image to be stitched. The valid pixel area refers to the area composed of valid pixels, usually an irregular polygon area. The target inscribed rectangular area can be the largest horizontal inscribed rectangular area, or the largest inscribed rectangular area among the horizontal inscribed rectangular areas with the longest vertical sides.

As shown in Figure 4a, the image after the blank control image is transformed using the initial image transformation parameters (in this example, six local cameras A1 to A6 are used as examples for explanation). In order to show the positional relationship of each transformed image, each transformed image is uniformly drawn on a unified coordinate plane, and two adjacent transformed images are respectively marked with dotted line boundaries and dotted line boundaries for easy distinction. The valid pixel area in each transformed image is displayed in white (that is, the pixel value of each pixel in the blank control image is set to 255), and the invalid pixel area is displayed in black (the invalid pixel value is set to 0). Then, the transformed image is cropped using the initial cropping parameters of each local camera, and the image to be stitched is shown in Figure 4b. In this example, there are invalid pixel areas in the current images to be stitched of local cameras A3, A4 and A5.

The maximum inscribed rectangle of the area can be calculated based on the area where the valid pixels in the current image to be stitched of the local cameras A3, A4 and A5 are located. As shown in Figure 4c, _a3 , _b3 , _c3 , and _d3 are the four vertices of the maximum inscribed rectangle area of the local camera A3. The server can determine the global position information (pixel coordinates) of the upper left pixel _a3 and the lower right pixel _d3 , and identify the boundary of the maximum inscribed rectangle area.

Step 204 , based on the global position information of the target inscribed rectangular area, the current cropping parameters of the target local camera corresponding to the target image to be stitched and the local camera associated with the target local camera are updated.

Among them, the associated local cameras refer to other local cameras whose cropping parameters are related to the cropping parameters of the target local camera, such as local cameras in the same row and adjacent local cameras. For local cameras in the same row, the upper boundary cropping parameters and the lower boundary cropping parameters are the same. If the cropping parameters of one of the local cameras are updated, the cropping parameters of the local cameras in the row must also be updated, so they are associated cameras. For two adjacent local cameras, if the left camera updates the right boundary, the left boundary of the right camera is updated accordingly, so they are associated cameras.

Since the target inscribed rectangular area is full of valid pixels, the position of the target inscribed rectangular area can be used as the new cropping position of the target camera to obtain new cropping parameters. Accordingly, the cropping parameters of the local camera associated with the local camera need to be adaptively updated.

Step 205: obtaining a target image stitching model based on the updated cropping parameters.

In implementation, after the initial cropping parameters in the initial image stitching model are updated, the target image stitching model is obtained, which is used to stitch the local video images taken by the array camera, thereby reducing or eliminating the invalid pixel area in the fused video image and improving the picture integrity and imaging quality of the fused video image.

In the above-mentioned image stitching method, an existing image stitching model construction method can be used to establish an initial image stitching model of an array camera, and then the blank reference image is transformed and cropped by the initial image transformation parameters and initial cropping parameters of each local camera to obtain an image to be stitched. Since the pixel value of each pixel in the blank reference image is different from the pixel value of the invalid pixel, the invalid pixel area and the valid pixel area in each image to be stitched can be quickly and accurately determined according to the pixel value of each pixel in the image to be stitched, and then the cropping parameters of the target local camera and its associated local camera are updated according to the global position information of the target inscribed rectangle of the valid pixel area in the target image to be stitched where the invalid pixel area exists. Among them, the pixels in the target inscribed rectangle area are all valid pixels, and the cropping parameters are updated based on the inscribed rectangle area, which can reduce or eliminate the invalid pixel area in the fused video image, and improve the picture integrity and imaging quality of the fused video image. Moreover, this method only needs to simply update the parameters of the image stitching model to improve the adaptability of the image stitching model, without the need for complex modifications to the image stitching model construction algorithm, thereby improving the reusability of the existing image stitching model construction algorithm and effectively saving development resources.

In one embodiment, a specific implementation of the updating process of the upper boundary clipping parameter and the lower boundary clipping parameter is provided. The process of updating the current clipping parameter in step 204 specifically includes the following steps:

Step 204a, based on the upper boundary ordinate and the lower boundary ordinate of each target inscribed rectangular area in the row where the target local camera is located, determine the maximum upper boundary ordinate and the minimum lower boundary ordinate of the row, and update the current upper boundary cropping parameters and the current lower boundary cropping parameters of each local camera in the row to the maximum upper boundary ordinate and the minimum lower boundary ordinate, respectively.

In implementation, if the image captured by the array camera is multiple rows, and there are target local cameras in more than one row, the cropping parameters of each row can be adjusted separately. This example is explained by taking a row of captured images as an example. As shown in Figure 4c, the target local cameras A3, A4 and A5 are located in the same row. The upper boundary ordinates and the lower boundary ordinates of the three target inscribed rectangular areas in the row are compared to find the maximum upper boundary ordinate and the minimum lower boundary ordinate. In this example, the upper boundary ordinate of the local camera A5 (i.e., the ordinate v _a5 of the pixel points a ₅ and b ₅ ) is the largest, and the lower boundary ordinates of the three target local cameras are the same, so the upper boundary cropping parameters of each local camera in the row (A1 to A6, and other local cameras not shown) are updated to v _a5 , and the lower boundary cropping parameters remain unchanged.

In one embodiment, a specific implementation of the updating process of the left border cropping parameter and the right border cropping parameter is provided. The process of updating the current cropping parameter in step 204 specifically includes the following steps:

Step 204b, for each target local camera, calculate the difference between the horizontal coordinate of the left boundary of the area of the target local camera and the current left boundary cropping parameter to obtain the current left boundary difference, and calculate the difference between the horizontal coordinate of the right boundary of the area of the target local camera and the current right boundary cropping parameter to obtain the current right boundary difference.

Step 204c, when the current left boundary difference is greater than a preset threshold, the current left boundary cropping parameter of the target local camera and the current right boundary cropping parameter of the local camera to the left of the target local camera are updated to the horizontal coordinate of the left boundary of the target local camera.

Step 204d: when the current right boundary difference is greater than a preset threshold, the current right boundary cropping parameter of the target local camera and the current left boundary cropping parameter of the local camera to the right of the target local camera are updated to the horizontal coordinate of the right boundary of the target local camera.

For example, the preset threshold value may be set to 0 (or a smaller number greater than 0). As shown in FIG4c , for the target local camera A3, the current left boundary difference is Δx1 (greater than the preset threshold value), and the current right boundary difference is 0 (equal to the preset threshold value). Therefore, the left boundary clipping parameter of the target local camera A3 may be updated to the left boundary abscissa of the inscribed rectangular area (the abscissa u _a3 of the pixel point a ₃ ), and the right boundary clipping parameter of its left neighbor local camera A2 may be updated to u _a3 .

In one embodiment, the process of obtaining the target image stitching model in step 205 specifically includes the following steps: using the updated cropping parameters as new current cropping parameters, returning to execute the image transformation and cropping steps for each local camera, using the initial image transformation parameters and current cropping parameters of the local camera, and using the blank control image corresponding to the local camera. Correspondingly, after judging whether there is an invalid pixel area in each current image to be stitched according to the pixel value of each pixel point in each current image to be stitched, the method also includes the following steps: when there is no invalid pixel area in each current image to be stitched, using the current cropping parameters as the target cropping parameters to obtain the target image stitching model.

In implementation, after the initial cropping parameters are updated based on step 204 (first update), the blank control image can be transformed and cropped based on the updated cropping parameters to obtain a new image to be stitched, and it is determined whether there is an invalid pixel area in the new image to be stitched. If there is no invalid pixel area in each new image to be stitched, the updated cropping parameters are used as the cropping parameters in the target image stitching model. If there is still an invalid pixel area in the new image to be stitched, the current image to be stitched with the invalid pixel area is determined as the target image to be stitched in step 203, and the current cropping parameters are updated according to the global position information of the target inscribed rectangular area, that is, it is updated again (second update).

In one embodiment, after the cropping parameters are updated, the current update number can be recorded. For example, after the first update, the current update number is recorded as 1, and the number is increased by 1 for each update. Based on the cropping parameters after the first update, the blank control image is transformed and cropped to obtain a new image to be stitched. If there is an invalid pixel area in the new image to be stitched, since the update number is less than the preset value (the preset value can be set to 2 or 3) at this time, the step of determining the target image to be stitched in step 203 is executed, and the current cropping parameters are updated again, and the update number is recorded as 2. Then, based on the cropping parameters after the second update, the blank control image is transformed and cropped. If there is still an invalid pixel area in the new image to be stitched, since the current update number is equal to the preset value (if it is set to 2), the image to be stitched with the invalid pixel area is determined as the image to be corrected, and its corresponding local camera is the local camera to be corrected. And according to the global position information of the invalid pixel points contained in the image to be corrected, the current upper boundary cropping parameters and/or the current lower boundary cropping parameters of the local camera to be corrected and the local camera in the same line of the local camera to be corrected are corrected.

In this embodiment, the cropping parameters can be iteratively updated. In some examples, the preset value of the number of updates can be set to 3 times, and the first update can only execute step 204a to update the upper and lower boundary cropping parameters. If a second update is required, starting from the second update, step 204a can be omitted, and only steps 204b to 204d can be executed to update the left and right boundary cropping parameters. All steps from step 204a to step 204d can also be executed each time the update is performed. If the number of updates reaches the preset number of times, it means that adjusting the left and right boundary cropping parameters can no longer completely eliminate the invalid pixel area. In order to improve the imaging quality of the fused image, the upper and lower boundary cropping parameters can be further corrected to reduce or eliminate the invalid pixel area.

In one example, the correction process specifically includes the following steps: the server can traverse the first column of pixels and the last column of pixels of the image to be corrected from top to bottom respectively. If the current pixel is an invalid pixel, the server will search for the next pixel until the current pixel is a valid pixel, stop traversing, and use the ordinate of the current pixel as a candidate upper boundary cropping parameter. Also, the server can traverse the first column of pixels and the last column of pixels of the image to be corrected from bottom to top respectively. If the current pixel is an invalid pixel, the server will search for the next pixel until the current pixel is a valid pixel, stop traversing, and use the ordinate of the current pixel as a candidate lower boundary cropping parameter. Then, the server can correct the current upper boundary cropping parameters and the current lower boundary cropping parameters of the local camera to be corrected and the local camera in the same row as the local camera to be corrected according to the maximum value of each candidate upper boundary cropping parameter and the minimum value of each candidate lower boundary cropping parameter.

Based on the initial image stitching model, each local image captured by the array camera shown in Figure 3a is stitched and fused, and the obtained fused image is shown in Figure 7a. Since the array camera has undergone structural improvements, the image sensor of each local camera has a certain rotation angle (positively correlated with the size of the horizontal angle of the axis), and more invalid pixel areas appear in the fused image, indicating that the adaptability of the initial image stitching model is low. The cropping parameters of the initial image stitching model are updated by the image stitching method provided in the embodiment of the present application, and the fused image obtained based on the updated target image stitching model is shown in Figure 7b. It can be seen that there is no obvious invalid pixel area, and the imaging quality of the fused image is significantly improved.

It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

Based on the same inventive concept, the embodiment of the present application also provides an image stitching device for implementing the above-mentioned image stitching method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations of one or more image stitching device embodiments provided below can refer to the above-mentioned limitations on the image stitching method, and will not be repeated here.

In one embodiment, as shown in FIG. 5 , an image stitching device 500 for an array camera is provided, comprising: a building module 501, a cropping module 502, a first determining module 503 and an updating module 504, wherein:

Establishing module 501, used for obtaining local sample images taken by each local camera of the array camera, and establishing an initial image stitching model based on each local sample image; the initial image stitching model includes initial image transformation parameters and initial cropping parameters corresponding to each local camera.

The cropping module 502 is used to use the initial cropping parameters as current cropping parameters, and for each of the local cameras, use the initial image transformation parameters of the local camera and the current cropping parameters to perform image transformation and cropping on the blank control image corresponding to the local camera to obtain the current image to be stitched; wherein the blank control image has the same size as the local sample image, and the pixel value of each pixel in the blank control image is different from the pixel value of the invalid pixel.

The first determination module 503 is used to determine whether there is an invalid pixel area in each of the current images to be stitched according to the pixel value of each pixel point in each of the current images to be stitched, determine the current image to be stitched with the invalid pixel area as the target image to be stitched, and determine the target inscribed rectangular area of the valid pixel area in the target image to be stitched.

The updating module 504 is used to update the current cropping parameters of the target local camera corresponding to the target image to be stitched and the associated local camera of the target local camera according to the global position information of the inscribed rectangular area of the target, and obtain a target image stitching model based on the updated cropping parameters. The target image stitching model is used to perform stitching and fusion processing on the local video images taken by the array camera.

In one embodiment, the associated local camera includes a local camera in the same row; the current cropping parameters include a current upper boundary cropping parameter and a current lower boundary cropping parameter; the global position information of the target inscribed rectangular area includes the upper boundary ordinate of the area and the lower boundary ordinate of the area. The updating module 504 is specifically used to: determine the maximum upper boundary ordinate of the row and the minimum lower boundary ordinate of the area according to the upper boundary ordinate of the area and the lower boundary ordinate of the area of each target inscribed rectangular area in the row where the target local camera is located, and update the current upper boundary cropping parameter and the current lower boundary cropping parameter of each local camera in the row to the maximum upper boundary ordinate and the minimum lower boundary ordinate, respectively.

In one embodiment, the associated local camera includes a left neighboring local camera and a right neighboring local camera; the current cropping parameters include a current left boundary cropping parameter and a current right boundary cropping parameter; and the global position information of the target inscribed rectangular area includes a region left boundary horizontal coordinate and a region right boundary horizontal coordinate. The updating module 504 is specifically used to: for each of the target local cameras, calculate the difference between the region left boundary horizontal coordinate of the target local camera and the current left boundary cropping parameter to obtain the current left boundary difference, and calculate the difference between the region right boundary horizontal coordinate of the target local camera and the current right boundary cropping parameter to obtain the current right boundary difference; when the current left boundary difference is greater than a preset threshold, update the current left boundary cropping parameter of the target local camera and the current right boundary cropping parameter of the left neighboring local camera of the target local camera to the region left boundary horizontal coordinate of the target local camera; when the current right boundary difference is greater than a preset threshold, update the current right boundary cropping parameter of the target local camera and the current left boundary cropping parameter of the right neighboring local camera of the target local camera to the region right boundary horizontal coordinate of the target local camera.

In one embodiment, the updating module 504 is further configured to use the updated cropping parameters as new current cropping parameters, and return to execute, for each of the local cameras, image transformation and cropping steps for the blank control image corresponding to the local camera using the initial image transformation parameters and the current cropping parameters of the local camera.

Correspondingly, the device further comprises a second determining module, which is used to use the current cropping parameters as target cropping parameters to obtain a target image stitching model when none of the current images to be stitched have invalid pixel regions.

In one embodiment, the updating module 504 is also used to record the current update times. Accordingly, the device further includes a third determining module and a correction module, wherein:

The third determining module is used to perform the step of determining the current image to be stitched with the invalid pixel area as the target image to be stitched when at least one of the current images to be stitched has an invalid pixel area and the current update number is less than a preset value.

The correction module is used to determine the new current image to be stitched with the invalid pixel area as the image to be corrected, and its corresponding local camera is the local camera to be corrected, when at least one of the current images to be stitched has an invalid pixel area and the current update number is greater than or equal to a preset value; according to the global position information of the invalid pixel points contained in the image to be corrected, correct the current upper boundary cropping parameters and/or the current lower boundary cropping parameters of the local camera to be corrected and the local camera in the same row of the local camera to be corrected, and use the corrected cropping parameters as the target cropping parameters to obtain the target image stitching model.

In one of the embodiments, the correction module is further used to: traverse the first column of pixels and the last column of pixels of the image to be corrected from top to bottom respectively, if the current pixel is an invalid pixel, search for the next pixel until the current pixel is a valid pixel, stop traversal, and use the vertical coordinate of the current pixel as a candidate upper boundary cropping parameter; traverse the first column of pixels and the last column of pixels of the image to be corrected from bottom to top respectively, if the current pixel is an invalid pixel, search for the next pixel until the current pixel is a valid pixel, stop traversal, and use the vertical coordinate of the current pixel as a candidate lower boundary cropping parameter; correct the current upper boundary cropping parameter and the current lower boundary cropping parameter of the local camera to be corrected and the local camera in the same row of the local camera to be corrected according to the maximum value of each of the candidate upper boundary cropping parameters and the minimum value of each of the candidate lower boundary cropping parameters.

Each module in the above-mentioned image stitching device for array cameras can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG6. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data required or generated for executing the above-mentioned image stitching method for array cameras. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, an image stitching method for array cameras is implemented.

Those skilled in the art will understand that the structure shown in FIG. 6 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

In one embodiment, a billion-pixel computational imaging system is provided. The system includes an array camera and a server, wherein:

The array camera is used to capture local sample images through each local camera, and send each local sample image to the server;

The server is used to establish an initial image stitching model based on each of the local sample images; the initial image stitching model includes initial image transformation parameters and initial cropping parameters corresponding to each of the local cameras; and the initial cropping parameters are used as current cropping parameters;

The server is also used for performing image transformation and cropping on the blank control image corresponding to each local camera using the initial image transformation parameters and the current cropping parameters of the local camera to obtain a current image to be stitched; wherein the blank control image has the same size as the local sample image, and the pixel value of each pixel in the blank control image is different from the pixel value of the invalid pixel; judging whether there is an invalid pixel area in each current image to be stitched according to the pixel value of each pixel in each current image to be stitched, determining the current image to be stitched with the invalid pixel area as the target image to be stitched, and determining the target inscribed rectangular area of the valid pixel area in the target image to be stitched; updating the current cropping parameters of the target local camera corresponding to the target image to be stitched and the associated local camera of the target local camera according to the global position information of the target inscribed rectangular area; obtaining a target image stitching model based on the updated cropping parameters;

The array camera is also used to capture local video images and send them to the server;

The server is also used to perform splicing and fusion processing on each of the local video images based on the target image splicing model to obtain a fused video image, and send the fused video image to a display terminal.

In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims (10)

1. An image stitching method for an array camera, characterized in that the method comprises: Acquire local sample images taken by each local camera of the array camera, and establish an initial image stitching model based on each local sample image; the initial image stitching model includes initial image transformation parameters and initial cropping parameters corresponding to each local camera; Using the initial cropping parameters as current cropping parameters, for each of the local cameras, using the initial image transformation parameters of the local camera and the current cropping parameters, performing image transformation and cropping on the blank control image corresponding to the local camera, to obtain a current image to be stitched; wherein the blank control image has the same size as the local sample image, and the pixel value of each pixel in the blank control image is different from the pixel value of the invalid pixel; According to the pixel value of each pixel point in each of the current images to be stitched, it is judged whether there is an invalid pixel area in each of the current images to be stitched, the current images to be stitched with the invalid pixel area are determined as the target images to be stitched, and the target inscribed rectangular area of the valid pixel area in the target images to be stitched is determined; According to the global position information of the target inscribed rectangular area, the current cropping parameters of the target local camera corresponding to the target image to be stitched and the associated local camera of the target local camera are updated; A target image stitching model is obtained based on the updated cropping parameters, and the target image stitching model is used to perform stitching and fusion processing on each local video image captured by the array camera. 2. The method according to claim 1, characterized in that the associated local camera includes a local camera in the same industry; the current cropping parameters include a current upper boundary cropping parameter and a current lower boundary cropping parameter; the global position information of the target inscribed rectangular area includes the upper boundary ordinate of the area and the lower boundary ordinate of the area; the current cropping parameters of the target local camera corresponding to the target image to be stitched and the associated local camera of the target local camera are updated according to the global position information of the target inscribed rectangular area, comprising: According to the upper boundary ordinate and the lower boundary ordinate of each target inscribed rectangular area in the row where the target local camera is located, the maximum upper boundary ordinate and the minimum lower boundary ordinate of the row are determined, and the current upper boundary cropping parameters and the current lower boundary cropping parameters of each local camera in the row are updated to the maximum upper boundary ordinate and the minimum lower boundary ordinate, respectively. 3. The method according to claim 1, characterized in that the associated local camera includes a left neighboring local camera and a right neighboring local camera; the current cropping parameters include a current left boundary cropping parameter and a current right boundary cropping parameter; the global position information of the target inscribed rectangular area includes a left boundary horizontal coordinate of the area and a right boundary horizontal coordinate of the area; and the current cropping parameters of the target local camera corresponding to the target image to be stitched and the associated local camera of the target local camera are updated according to the global position information of the target inscribed rectangular area, comprising: For each of the target local cameras, the difference between the horizontal coordinate of the left boundary of the region of the target local camera and the current left boundary clipping parameter is calculated to obtain a current left boundary difference, and the difference between the horizontal coordinate of the right boundary of the region of the target local camera and the current right boundary clipping parameter is calculated to obtain a current right boundary difference; When the current left boundary difference is greater than a preset threshold, the current left boundary clipping parameter of the target local camera and the current right boundary clipping parameter of the left neighboring local camera of the target local camera are updated to the horizontal coordinate of the left boundary of the region of the target local camera; When the current right boundary difference is greater than a preset threshold, the current right boundary cropping parameter of the target local camera and the current left boundary cropping parameter of the local camera to the right of the target local camera are updated to the horizontal coordinate of the right boundary of the area of the target local camera. 4. The method according to claim 1, characterized in that the step of obtaining a target image stitching model based on the updated cropping parameters comprises: Using the updated cropping parameters as new current cropping parameters, returning to execute, for each of the local cameras, the steps of image transformation and cropping the blank control image corresponding to the local camera using the initial image transformation parameters and the current cropping parameters of the local camera; After determining whether there is an invalid pixel area in each of the current images to be stitched according to the pixel value of each pixel point in each of the current images to be stitched, the method further includes: In the case that there is no invalid pixel area in each of the current images to be stitched, the current cropping parameters are used as target cropping parameters to obtain a target image stitching model. 5. The method according to claim 4, characterized in that after the current cropping parameters of the target local camera corresponding to the target image to be stitched and the local camera associated with the target local camera are updated, the method further comprises: Record the current update times; After determining whether there is an invalid pixel area in each of the current images to be stitched according to the pixel value of each pixel point in each of the current images to be stitched, the method further includes: In the case that at least one of the current images to be stitched has an invalid pixel region and the current update number is less than a preset value, performing a step of determining the current image to be stitched having the invalid pixel region as a target image to be stitched; In the case that at least one of the current images to be stitched has an invalid pixel region and the current update number is greater than or equal to a preset value, the current image to be stitched with the invalid pixel region is determined as the image to be corrected, and the local camera corresponding to the image is the local camera to be corrected; According to the global position information of the invalid pixel points contained in the image to be corrected, the current upper boundary cropping parameters and/or the current lower boundary cropping parameters of the local camera to be corrected and the local camera in the same row as the local camera to be corrected are corrected, and the corrected cropping parameters are used as target cropping parameters to obtain a target image stitching model. 6. The method according to claim 5, characterized in that the step of correcting the current upper boundary cropping parameters and/or the current lower boundary cropping parameters of the local camera to be corrected and the local camera in the same row as the local camera to be corrected according to the global position information of the invalid pixel points contained in the image to be corrected comprises: Traverse the first column of pixels and the last column of pixels of the image to be corrected from top to bottom respectively, if the current pixel is an invalid pixel, search for the next pixel until the current pixel is a valid pixel, stop traversal, and use the ordinate of the current pixel as a candidate upper boundary clipping parameter; Traverse the first column of pixels and the last column of pixels of the image to be corrected from bottom to top respectively, if the current pixel is an invalid pixel, search for the next pixel until the current pixel is a valid pixel, stop traversal, and use the ordinate of the current pixel as a candidate lower boundary cropping parameter; According to the maximum value of each candidate upper boundary cropping parameter and the minimum value of each candidate lower boundary cropping parameter, the current upper boundary cropping parameter and the current lower boundary cropping parameter of the local camera to be corrected and the local camera in the same row as the local camera to be corrected are corrected. 7. An image stitching device for an array camera, characterized in that the device comprises: Establishing a module for acquiring local sample images taken by each local camera of the array camera, and establishing an initial image stitching model based on each local sample image; the initial image stitching model includes initial image transformation parameters and initial cropping parameters corresponding to each local camera; A cropping module, configured to use the initial cropping parameters as current cropping parameters, and for each of the local cameras, use the initial image transformation parameters of the local camera and the current cropping parameters to perform image transformation and cropping on the blank control image corresponding to the local camera, so as to obtain a current image to be stitched; wherein the blank control image has the same size as the local sample image, and the pixel value of each pixel in the blank control image is different from the pixel value of the invalid pixel; A first determination module is used to determine whether there is an invalid pixel area in each of the current images to be stitched according to the pixel value of each pixel point in each of the current images to be stitched, determine the current image to be stitched with the invalid pixel area as the target image to be stitched, and determine a target inscribed rectangular area of a valid pixel area in the target image to be stitched; An updating module is used to update the current cropping parameters of the target local camera corresponding to the target image to be stitched and the associated local camera of the target local camera according to the global position information of the inscribed rectangular area of the target, and obtain a target image stitching model based on the updated cropping parameters, and the target image stitching model is used to perform stitching and fusion processing on the local video images taken by the array camera. 8. A billion-pixel computational imaging system, characterized in that the system comprises an array camera and a server, wherein: The array camera is used to capture local sample images through each local camera, and send each local sample image to the server; The server is used to establish an initial image stitching model based on each of the local sample images; the initial image stitching model includes initial image transformation parameters and initial cropping parameters corresponding to each of the local cameras; and the initial cropping parameters are used as current cropping parameters; The server is also used for performing image transformation and cropping on the blank control image corresponding to each local camera using the initial image transformation parameters and the current cropping parameters of the local camera to obtain a current image to be stitched; wherein the blank control image has the same size as the local sample image, and the pixel value of each pixel in the blank control image is different from the pixel value of the invalid pixel; judging whether there is an invalid pixel area in each current image to be stitched according to the pixel value of each pixel in each current image to be stitched, determining the current image to be stitched with the invalid pixel area as the target image to be stitched, and determining the target inscribed rectangular area of the valid pixel area in the target image to be stitched; updating the current cropping parameters of the target local camera corresponding to the target image to be stitched and the associated local camera of the target local camera according to the global position information of the target inscribed rectangular area; obtaining a target image stitching model based on the updated cropping parameters; The array camera is also used to capture local video images and send them to the server; The server is also used to perform splicing and fusion processing on each of the local video images based on the target image splicing model to obtain a fused video image, and send the fused video image to a display terminal. 9. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein the processor implements the steps of the method according to any one of claims 1 to 6 when executing the computer program. 10. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.