WO2023024697A1 - 图像拼接方法和电子设备 - Google Patents

图像拼接方法和电子设备 Download PDF

Info

Publication number
WO2023024697A1
WO2023024697A1 PCT/CN2022/102342 CN2022102342W WO2023024697A1 WO 2023024697 A1 WO2023024697 A1 WO 2023024697A1 CN 2022102342 W CN2022102342 W CN 2022102342W WO 2023024697 A1 WO2023024697 A1 WO 2023024697A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
overlapping area
sub
initial
fusion
Prior art date
Application number
PCT/CN2022/102342
Other languages
English (en)
French (fr)
Inventor
刘伟舟
胡晨
周舒畅
Original Assignee
北京旷视科技有限公司
北京迈格威科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京旷视科技有限公司, 北京迈格威科技有限公司 filed Critical 北京旷视科技有限公司
Publication of WO2023024697A1 publication Critical patent/WO2023024697A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Definitions

  • the present disclosure relates to the technical field of image processing, in particular to an image splicing method and electronic equipment.
  • Image stitching is the process of combining multiple images with overlapping fields of view to produce an image with a larger field of view and higher resolution.
  • image splicing process it is usually necessary to fuse the pictures to be spliced.
  • related technologies it is usually based on a CPU (central processing unit, central processing unit) to carry out fusion-related calculation processing for each pixel in the spliced pictures after the initial splicing. Since the CPU needs a certain processing time to calculate each pixel, it takes a long time to fuse and calculate all the pixels in the stitched image after the initial stitching, which reduces the efficiency of fusion processing, thereby reducing the processing efficiency of image stitching.
  • the present disclosure provides an image mosaic method and electronic equipment to improve the processing efficiency of image mosaic.
  • the present disclosure provides an image stitching method, the method comprising: acquiring a first image and a second image to be stitched; determining an initial stitched image of the first image and the second image; using a first neural network model to The target overlapping area is fused to obtain a fused overlapping area corresponding to the target overlapping area; based on the fused overlapping area and the initial stitched image, the target stitched image corresponding to the first image and the second image is determined.
  • the method before determining the initial stitched image of the first image and the second image, the method further includes: performing illumination compensation on the second image; correspondingly, determining the initial stitched image of the first image and the second image includes: based on the first The first image and the second image after illumination compensation are used to determine an initial stitched image.
  • the step of performing illumination compensation on the second image includes: performing illumination compensation on the second image based on the second neural network model and the first image.
  • performing illumination compensation on the second image based on the second neural network model and the first image includes: determining a projection transformation matrix based on the first image and the second image; determining the first image and the second image based on the projection transformation matrix The initial overlapping area of two images; wherein, the initial overlapping area includes: the first sub-overlapping area corresponding to the first image and the second sub-overlapping area corresponding to the second image; the first sub-overlapping area and the second sub-overlapping area, input To the second neural network model, determine the mapping relationship between the first pixel value of each pixel in the first sub-overlapping area and the second pixel value of the same position pixel in the second sub-overlapping area through the second neural network model; obtain the second neural network The mapping relationship output by the network model; for each color channel, based on the mapping relationship, the pixel value of each pixel point in the second image in the color channel is matched with the pixel value of each pixel point in the first image in the color channel, To perform illumination compensation on the second image.
  • the step of determining the initial overlapping area of the first image and the second image includes: obtaining boundary coordinates of the second image; wherein, the boundary coordinates are used to indicate the image area of the second image; based on the projection transformation The boundary coordinates of the matrix and the second image are used to determine the boundary coordinates after the projection transformation; based on the boundary coordinates after the projection transformation, the second image after the projection transformation is determined; the overlapping image area of the second image after the projection transformation and the first image , determined as the initial overlapping region.
  • the step of determining the projection transformation matrix includes: extracting at least one first feature point in the first image, and at least one second feature point in the second image; based on at least one The first feature point and the at least one second feature point, at least one matching feature point pair is determined; based on the at least one matching feature point pair, a projection transformation matrix is determined.
  • the step of performing illumination compensation on the second image includes: inputting the first image and the second image into the second neural network model, and using the second neural network based on The first image performs illumination compensation on the second image to obtain the second image after illumination compensation.
  • the target overlapping area includes: a third sub-overlapping area corresponding to the first image and a fourth sub-overlapping area corresponding to the second image;
  • the first neural network model includes a splicing model and a fusion model; using the first neural network model
  • the step of performing fusion processing on the target overlapping area in the initial stitching image to obtain the fused overlapping area corresponding to the target overlapping area includes: inputting the third sub-overlapping area and the fourth sub-overlapping area into the stitching model, and using the stitching model to The patchwork between the third sub-overlapping area and the fourth sub-overlapping area is searched to obtain the patchwork area corresponding to the third sub-overlapping area and the fourth sub-overlapping area; based on the patchwork area, for the third sub-overlapping area and the fourth sub-overlapping area
  • the sub-overlapping area is fused to obtain an initial fusion overlapping area; the initial fusion overlapping area, the third sub-overlapping area, and the fourth sub-overlapping area are input to the fusion model, and the initial fusion overlapping area, the
  • the target overlapping area includes: a third sub-overlapping area corresponding to the first image and a fourth sub-overlapping area corresponding to the second image;
  • the first neural network model includes a splicing model; the initial splicing is performed using the first neural network model
  • the fusion model is obtained by training in the following manner: obtaining the first picture; performing translation and/or rotation processing on the first picture to obtain the second picture; performing fusion processing on the first picture and the second picture to obtain the initial fusion picture ; Based on the first picture, the second picture and the initial fusion picture, train the fusion model.
  • the step of determining the target stitched image corresponding to the first image and the second image includes: using the fused overlapping area to replace the target overlapping area in the initial stitched image to obtain the first image and The target stitched image corresponding to the second image.
  • color channels of the first image and the second image are arranged in an RGGB manner.
  • the first image and the second image are RAW domain images
  • the determining the initial spliced image of the first image and the second image includes:
  • the initial spliced image is determined based on the first image in the RGGB arrangement and the second image in the RGGB arrangement.
  • An electronic device provided by the present disclosure includes a processing device and a storage device, the storage device stores a computer program, and the computer program executes the above-mentioned image stitching method when the processed device is run.
  • the present disclosure provides a machine-readable storage medium, where a computer program is stored in the machine-readable storage medium, and the computer program executes the above-mentioned image stitching method when executed by a processing device.
  • a computer program product provided by the present disclosure includes a computer-readable storage medium storing program codes, and the program codes include instructions capable of executing the image stitching method as described above.
  • the image mosaic method and electronic equipment first acquire the first image and the second image to be stitched; determine the initial stitched image of the first image and the second image; then use the first neural network model to The target overlapping area is fused to obtain a fused overlapping area corresponding to the target overlapping area; finally, based on the fused overlapping area and the initial stitched image, the target stitched image corresponding to the first image and the second image is determined.
  • the first neural network model is used to fuse the target overlapping area in the initial spliced image to obtain the corresponding fused overlapping area. Fusion correlation calculation is performed for each pixel in the initial stitching image, which saves the time for fusion calculation of all pixels in the initial stitching image, improves the fusion processing efficiency, and further improves the processing efficiency of image stitching.
  • FIG. 1 is a schematic structural diagram of an electronic system provided by an embodiment of the present disclosure
  • FIG. 2 is a flow chart of an image stitching method provided by an embodiment of the present disclosure
  • FIG. 3 is a flowchart of another image stitching method provided by an embodiment of the present disclosure.
  • FIG. 4 is a flow chart of another image stitching method provided by an embodiment of the present disclosure.
  • FIG. 5 is a flow chart of another image stitching method provided by an embodiment of the present disclosure.
  • FIG. 6 is a flowchart of a method for mosaicing RAW domain images provided by an embodiment of the present disclosure
  • FIG. 7 is a schematic structural diagram of an image stitching device provided by an embodiment of the present disclosure.
  • Artificial Intelligence is an emerging science and technology that studies and develops theories, methods, technologies and application systems for simulating and extending human intelligence.
  • the subject of artificial intelligence is a comprehensive subject that involves many technologies such as chips, big data, cloud computing, Internet of Things, distributed storage, deep learning, machine learning, and neural networks.
  • computer vision is specifically to allow machines to recognize the world.
  • Computer vision technology usually includes face recognition, liveness detection, fingerprint recognition and anti-counterfeiting verification, biometric recognition, face detection, pedestrian detection, target detection, pedestrian detection, etc.
  • Image stitching is the process of combining multiple images with overlapping fields of view to produce a larger field of view and high-resolution images.
  • the field of view of an ordinary person's single eye is about 120°, and the field of view of both eyes is generally about 160° to 220°, while the FOV (Field Of Vision) of an ordinary camera is generally only 40° to 220°. Around 60°, it is difficult to ensure a large imaging field of view while performing high-definition imaging of detailed objects.
  • image stitching technology multiple cameras with small field of view can be combined into a multi-camera with a large field of view. , Sports event director and other fields have important application value.
  • RGB Red, red
  • G Green, green
  • B Blue, blue
  • RAW domain Image implementation for example, through the detection, segmentation or recognition of RAW domain images to solve the scene problems such as dark light and backlight that cannot be solved by RGB domain images.
  • embodiments of the present disclosure provide an image stitching method, device, and electronic equipment, and the technology can be applied to applications that require stitching of images. The embodiments of the present disclosure will be described in detail below.
  • FIG. 1 An example electronic system 100 for implementing the image stitching method, device and electronic device of the embodiments of the present disclosure is described with reference to FIG. 1 .
  • the electronic system 100 may include one or more processing devices 102, one or more storage devices 104, an input device 106, an output device 108, and one or more image acquisition devices 110, these components are interconnected via a bus system 112 and/or other forms of connection mechanisms (not shown). It should be noted that the components and structure of the electronic system 100 shown in FIG. 1 are only exemplary rather than limiting, and the electronic system may also have other components and structures as required.
  • the processing device 102 may be a gateway, or an intelligent terminal, or a device including a central processing unit (CPU) or other forms of processing units with data processing capabilities and/or instruction execution capabilities, which can control the electronic system Data from other components in the electronic system 100 can be processed, and other components in the electronic system 100 can be controlled to perform desired functions.
  • CPU central processing unit
  • the storage device 104 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory.
  • the volatile memory may include, for example, random access memory (RAM) and/or cache memory (cache).
  • the non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, and the like.
  • One or more computer program instructions can be stored on the computer-readable storage medium, and the processing device 102 can execute the program instructions to realize the client functions (implemented by the processing device) in the embodiments of the present disclosure described below and/or other desired functionality.
  • Various application programs and various data such as various data used and/or generated by the application programs, may also be stored in the computer-readable storage medium.
  • the input device 106 may be a device used by a user to input instructions, and may include one or more of a keyboard, a mouse, a microphone, and a touch screen.
  • the output device 108 may output various information (eg, images or sounds) to the outside (eg, a user), and may include one or more of a display, a speaker, and the like.
  • the image capture device 110 can capture preview video frames or image data, and store the captured preview video frames or image data in the storage device 104 for use by other components.
  • the devices in the example electronic system for realizing the image mosaic method, device and electronic device may be integrated or distributed, such as the processing device 102, the storage device 104, the input device 106 and the output device 108 are integrated into one body, and the image capture device 110 is set at a designated position where the target image can be captured.
  • the electronic system can be realized as an intelligent terminal such as a camera, a smart phone, a tablet computer, a computer, a vehicle terminal, or a server.
  • This embodiment provides an image stitching method, which is executed by the processing device in the above-mentioned electronic system; the processing device may be any device or chip capable of data processing. As shown in Figure 2, the method includes the following steps:
  • Step S202 acquiring the first image and the second image to be spliced.
  • the first image can be used as the reference image, and the second image can be used as the image to be registered, or the second image can be used as the reference image, and the first image can be used as the image to be registered. It can be selected according to actual needs; for convenience of explanation, take the first image as the reference image and the second image as the image to be registered as an example, the first image is usually located in the reference image coordinate system, and the first image can be a tar image Representation, etc.; the second image is usually located in the image coordinate system to be registered, and the second image can be represented by a src image; the first image and the second image can be RAW domain images, or RGB domain images, etc., In actual implementation, when images need to be spliced, it is usually necessary to first acquire a first image and a second image to be spliced, and the first image and the second image are usually images with a part of overlapping fields of view.
  • Step S204 determining an initial spliced image of the first image and the second image.
  • the second image can be projected onto the first image for initial stitching to obtain the first An initial stitched image of the stitching of the first image and the second image.
  • Step S206 using the first neural network model to perform fusion processing on the target overlapping areas in the initial spliced image to obtain the fused overlapping areas corresponding to the target overlapping areas.
  • the above-mentioned first neural network model can be implemented through various convolutional neural networks, such as residual network, VGG network, etc.; since the first image and the second image are images with a part of overlapping fields of view, the target overlapping area can be understood as, The overlapping area after the initial stitching process is performed on the first image and the second image; in actual implementation, after the above initial stitching image is determined, the target overlapping of the first image and the second image can be determined from the initial stitching image area, using the first neural network to perform fusion processing on the target overlapping area to obtain a corresponding fused overlapping area.
  • convolutional neural networks such as residual network, VGG network, etc.
  • Step S208 based on the fused overlapping area and the initial stitched image, determine the target stitched image corresponding to the first image and the second image.
  • the target stitched image of the first image and the second image can be obtained; the target stitched image can be two or more images with The overlapping images are assembled into a seamless panorama or high-resolution image; among them, two or several overlapping images may be images obtained at different times, different viewing angles, or different sensors.
  • the above image stitching method first obtains the first image and the second image to be stitched; determines the initial stitched image of the first image and the second image; then uses the first neural network model to perform fusion processing on the target overlapping area in the initial stitched image , to obtain the fused overlapping area corresponding to the target overlapping area; finally, based on the fused overlapping area and the initial stitched image, determine the target stitched image corresponding to the first image and the second image.
  • the first neural network model is used to fuse the target overlapping area in the initial spliced image to obtain the corresponding fused overlapping area. Fusion correlation calculation is performed for each pixel in the initial stitching image, which saves the time for fusion calculation of all pixels in the initial stitching image, improves the fusion processing efficiency, and further improves the processing efficiency of image stitching.
  • the embodiment of the present disclosure also provides another image stitching method, which is implemented on the basis of the method in the above embodiment; as shown in Figure 3, the method may include the following steps:
  • Step S302 acquiring the first image and the second image to be spliced.
  • Step S304 performing illumination compensation on the second image.
  • This step S304 can specifically be realized through the following step 1:
  • Step 1 Perform illumination compensation on the second image based on the second neural network model and the first image.
  • the second neural network model can be realized by a variety of convolutional neural networks, such as residual network, VGG network, etc.; the second neural network model can be a network model different from the first neural network model, or it can be the first neural network model. Network model, except that the process of lighting compensation can be performed by a certain sub-model or sub-module in the first neural network model.
  • the second image taken may have For the color deviation caused by the imbalance, in order to offset the color deviation existing in the second image, illumination compensation can be performed on the second image based on the pre-trained second neural network model and the first image.
  • This step one can specifically be realized through the following steps A to D:
  • Step A based on the first image and the second image, determine a projection transformation matrix.
  • the projection transformation matrix can be determined according to the acquired first image and the second image, where the first image and the second image usually have the same number of color channels, for example, the first image has R, G, There are four color channels of G and B, and the second image also has four color channels of R, G, G, and B; specifically, this step A can be realized through steps a to c:
  • Step a extracting at least one first feature point in the first image and at least one second feature point in the second image.
  • the above-mentioned first feature point can be From the corner points, edge points, bright spots in dark areas or dark spots in bright areas extracted from the first image, the number of the first feature points can be one or more; the above-mentioned second feature points can be obtained from the second For corner points, edge points, bright spots in dark areas or dark points in bright areas extracted from the image, the number of the second feature points may be one or more.
  • each color channel is preset with a corresponding first weight value; among the color channels of the second image, each color channel is preset with a corresponding second weight value.
  • Weight value in actual implementation, the above-mentioned first weight value and second weight value are usually preset fixed values, which can be set according to actual needs, and are not limited here.
  • the first image and the second weight value All images have four color channels of R, G, G, and B.
  • the first weight value corresponding to the R channel is 0.3
  • the first weight value corresponding to the two G channels is 0.2
  • the B channel is 0.2.
  • the corresponding first weight value is 0.3
  • in the second image the second weight value corresponding to the R channel is 0.3, the second weight value corresponding to the two G channels is 0.2, and the second weight value corresponding to the B channel is 0.3.
  • the step of extracting at least one first feature point in the first image may include step a0 to step a3:
  • Step a0 for each pixel in the first image, multiply the component of each color channel in the pixel by the first weight value corresponding to the color channel to obtain a first calculation result for each color channel.
  • the above-mentioned color channel can be understood as a channel for storing image color information.
  • an RGB image has three color channels, which are R channel, G channel and B channel respectively; the components of each color channel above can be understood as the brightness value of each color channel;
  • the first image usually includes multiple pixels, and each pixel usually has multiple color channels.
  • the component of each color channel in the pixel can be multiplied by the corresponding first Weight value, to obtain the first calculation result of the color channel
  • the first image has four color channels of R, G, G, and B, wherein, the first weight value corresponding to the R channel is 0.3, and the two G channels
  • the corresponding first weight value is 0.2
  • the first weight value corresponding to the B channel is 0.3
  • the component corresponding to the R channel of a specified pixel is 150
  • the corresponding components of the two G channels are both 100
  • Step a1 summing up the first calculation results of each color channel in the pixel to obtain the first gray value corresponding to the pixel.
  • Step a2 based on the first grayscale value corresponding to each pixel in the first image, determine the grayscale image of the first image.
  • a grayscale image of the first image can be obtained according to the obtained multiple first grayscale values.
  • Step a3 extracting at least one first feature point from the grayscale image of the first image.
  • one or more first feature points can be extracted from the grayscale image of the obtained first image based on algorithms such as SIFT (Scale-Invariant Feature Transform, scale invariant feature transformation) or SuperPoint,
  • SIFT Scale-Invariant Feature Transform, scale invariant feature transformation
  • SuperPoint a feature point detection and descriptor extraction method based on self-supervised training.
  • the step of extracting at least one second feature point in the second image may include step a4 to step a7:
  • Step a4 For each pixel in the second image, multiply the component of each color channel in the pixel by the second weight value corresponding to the color channel to obtain a second calculation result for each color channel.
  • the second image usually includes multiple pixels, and each pixel usually has multiple color channels.
  • the component of each color channel in the pixel can be multiplied by the corresponding Two weight values to obtain the second calculation result of the color channel, for example, the second image still has four color channels of R, G, G and B, wherein the first weight value corresponding to the R channel is 0.3, and the two The first weight value corresponding to the G channel is 0.2, and the first weight value corresponding to the B channel is 0.3.
  • Step a5 adding up the second calculation results of each color channel in the pixel to obtain the second gray value corresponding to the pixel.
  • the second calculation result of each color channel in each pixel in the second image is obtained through the above step a4, the second calculation result of each color channel in the pixel can be added to obtain the second calculation result of the pixel.
  • Step a6 based on the second grayscale value corresponding to each pixel in the second image, determine the grayscale image of the second image.
  • a grayscale image of the second image can be obtained according to the obtained multiple second grayscale values.
  • Step a7 extracting at least one second feature point from the grayscale image of the second image.
  • one or more second feature points may be extracted from the obtained grayscale image of the second image based on algorithms such as SIFT or SuperPoint.
  • Step b based on at least one first feature point and at least one second feature point, determine at least one matching feature point pair.
  • feature point matching can be performed, specifically, based on at least one first feature point and At least one second feature point, using algorithms such as KNN (K-NearestNeighbor, K-nearest neighbor classification algorithm) or RANSAC (RANdomSAmple Consensus, random sampling consensus algorithm), to obtain one or more matches in the first image and the second image Feature point pairs, the number of matching feature point pairs is generally not less than four pairs; for details, please refer to the process of using KNN or RANSAC to perform feature point matching in related technologies, and will not go into details here; wherein, KNN is each A sample can be represented by its nearest K neighbors.
  • KNN K-NearestNeighbor, K-nearest neighbor classification algorithm
  • RANSAC RandomSAmple Consensus, random sampling consensus algorithm
  • RANSAC random sampling consistent matching, which uses matching points to calculate the homography matrix between two images, and then uses the reprojection error to determine whether a certain match is a correct match.
  • Step c based on at least one matching feature point pair, determine a projection transformation matrix.
  • the projection transformation matrix can be calculated based on the matching information of the matching feature point pair, such as the coordinates of the matching feature point pair.
  • the matrix process will not be repeated here.
  • the second image can be projected onto the first image, and the projection transformation matrix works in such a way that, for each color channel included in each pixel in the second image, the calculated projection transformation matrix Perform projection transformation.
  • the corresponding transformed coordinates may be decimals. In this case, the coordinates usually need to be interpolated according to the values of adjacent pixels.
  • the interpolation processing method for example, a Cubic bicubic interpolation method, etc. may be used to reduce the influence of resolution, wherein the resolution may be understood as the definition of the second image after projection transformation.
  • Step B Determine the initial overlapping area of the first image and the second image based on the projection transformation matrix; wherein, the initial overlapping area includes: a first sub-overlapping area corresponding to the first image and a second sub-overlapping area corresponding to the second image.
  • the second image is projected onto the first image. Since the first image and the second image are images with a part of the overlapping field of view, after the projection transformation is completed, the first image can be obtained An initial overlapping area with the projectively transformed second image, the initial overlapping area includes a first sub-overlapping area of the first image, and a second sub-overlapping area of the projectively transformed second image.
  • This step B can specifically be realized through step h to step k:
  • Step h acquiring boundary coordinates of the second image; where the boundary coordinates are used to indicate the image area of the second image.
  • the above boundary coordinates can be understood as edge coordinates used to indicate the overall shape of the second image, and the image area corresponding to the second image can be obtained through the edge coordinates.
  • edge coordinates used to indicate the overall shape of the second image
  • the image area corresponding to the second image can be obtained through the edge coordinates.
  • the number of the boundary coordinates is usually multiple.
  • Step i based on the projection transformation matrix and the boundary coordinates of the second image, determine the projection-transformed boundary coordinates.
  • the boundary coordinates of the projectively transformed second image can be obtained; for example, if the boundary coordinates of the second image are the coordinates of the four corners, and after the projection transformation is completed based on the projection transformation matrix, the coordinates of the four corners of the second image after the projection transformation can be obtained.
  • Step j based on the boundary coordinates after projection transformation, determine the second image after projection transformation.
  • the projectively transformed second image is determined according to the image area surrounded by the projectively transformed boundary coordinates. For example, still taking the boundary coordinates of the second image as four corner coordinates as an example, the image area enclosed by the four corner coordinates of the second image after projection transformation is the second image after projection transformation.
  • Step k determining the overlapping image area of the projectively transformed second image and the first image as an initial overlapping area.
  • the intersection of the second image after projection transformation and the first image is taken to obtain the above initial overlapping area.
  • Step C input the first sub-overlapping area and the second sub-overlapping area into the second neural network model, and determine the first pixel value and the second sub-overlapping area of each pixel in the first sub-overlapping area through the second neural network model The mapping relationship of the second pixel value of the pixel at the same position in .
  • all the pixel values of the first sub-overlapping area corresponding to the first image and the second sub-overlapping area corresponding to the second image can be extracted to calculate , the first sub-overlapping area corresponds to the pixel value histogram distribution of the image, and in the second image, the second sub-overlapping area corresponds to the pixel value histogram distribution of the image, and the second sub-overlapping area corresponds to the image based on histogram matching
  • the distribution of the pixel value histogram is transformed to match the distribution of the pixel value histogram of the image corresponding to the first sub-overlapping region. Specifically, it can be realized by constructing a pixel value mapping table.
  • the pixel value mapping table can be a LUT (Look-Up-Table , display lookup table) tables, etc., and the number of color channels included in the first image and the second image is the same, and each color channel usually has its corresponding pixel value mapping table; for example, the first image and the second image Both include the four color channels of R, G, G and B, input the first sub-overlapping area and the second sub-overlapping area to the pre-trained second neural network model, and determine the color of each pixel in the first sub-overlapping area
  • the mapping relationship between the first pixel value and the second pixel value of the same position pixel in the second sub-overlapping area can be GT (Ground Truth, which represents the classification accuracy of the training set of supervised learning, and is used to prove or overthrow a hypothesis ) to learn four LUT tables of 0-65535, and finally four LUT tables of 0-65535 can be calculated by the second neural network model, wherein, the four LUT tables of 0-65535 are usually different, and the LUT table The
  • the above-mentioned pixel value mapping table is usually calculated based on the traditional method of the CPU. Since it usually needs to be calculated pixel by pixel, the calculation process takes a long time and the efficiency is low.
  • NN Neurological Network, neural network ) realizes the acceleration of calculation methods in related technologies.
  • the above-mentioned pixel value mapping table can be understood as a mapping table of pixel gray value.
  • the mapping table undergoes a certain transformation of the actual sampled pixel gray value, such as threshold, inversion, binarization, contrast adjustment, linear Transformation, etc., become another corresponding gray value, which can highlight the useful information of the image and enhance the light contrast of the image.
  • Step D obtaining the mapping relationship output by the second neural network model
  • Step E for each color channel, based on the mapping relationship, match the pixel value of each pixel point in the color channel in the second image with the pixel value of each pixel point in the color channel in the first image, so as to match the pixel value of each pixel point in the color channel in the second image
  • the image is light compensated.
  • the pixel value mapping table can be applied to the entire second image, that is, through the pixel value mapping
  • the table can map the value range of each color channel of the second image to a distribution similar to that of the first image, that is, the pixel value of each pixel in the color channel in the second image can be compared with that in the first image Each pixel is matched with the pixel value of the color channel.
  • the first image and the second image both include the four color channels of R, G, G, and B, and the four color channels corresponding to four 0-65535 LUT tables are calculated through the second neural network model For example, for each color channel, through the LUT table corresponding to the color channel, the pixel values of each pixel point of the second image in the color channel can be mapped to a distribution similar to that of the first image, thereby realizing Lighting compensation for the second image.
  • This step one can specifically also be realized through the following steps H:
  • Step H inputting the first image and the second image into the second neural network model, performing illumination compensation on the second image based on the first image through the second neural network, and obtaining an illumination-compensated second image.
  • the first image and the second image can be input into a pre-trained second neural network model, and the second image can be compensated for illumination based on the first image through the second neural network, and the second image can be used
  • the first image, the second image after illumination compensation based on histogram matching is GT to supervise the training of the second neural network model, and finally the second image after illumination compensation is obtained through the calculation of the second neural network model.
  • the illumination compensation method in this embodiment uses a pre-trained neural network model to perform illumination compensation on the second image, and does not need to calculate each pixel based on the CPU, which saves the time for compensating all pixels in the image and improves the performance of the image. Processing efficiency of light compensation.
  • Step S306 based on the first image and the second image after illumination compensation, determine an initial spliced image.
  • the second image after illumination compensation is projected onto the first image and initially stitched to obtain an initial stitched image of the first image and the second image after illumination compensation.
  • Step S308 using the first neural network model to perform fusion processing on the target overlapping areas in the initial spliced image to obtain the fused overlapping areas corresponding to the target overlapping areas.
  • Step S310 based on the fused overlapping area and the initial stitched image, determine the target stitched image corresponding to the first image and the second image.
  • the image stitching method described above acquires the first image and the second image to be stitched, performs light compensation on the second image, determines the initial stitched image based on the first image and the light-compensated second image, and uses the first neural network model to
  • the target overlapping area in the initial stitched image is fused to obtain a fused overlapping area corresponding to the target overlapping area, and based on the fused overlapping area and the initial stitched image, the target stitched image corresponding to the first image and the second image is determined.
  • the first neural network model is used to fuse the target overlapping area in the initial spliced image to obtain the corresponding fused overlapping area. Fusion correlation calculation is performed for each pixel in the initial stitching image, which saves the time for fusion calculation of all pixels in the initial stitching image, improves the fusion processing efficiency, and further improves the processing efficiency of image stitching.
  • the embodiment of the present disclosure also provides another image mosaic method, which is implemented on the basis of the methods in the above embodiments; the method focuses on the use of the first neural network model to perform fusion processing on the target overlapping areas in the initial stitching image to obtain the target
  • the specific process of fused overlapping areas corresponding to the overlapping areas, in this method, the target overlapping areas include: the third sub-overlapping area corresponding to the first image and the fourth sub-overlapping area corresponding to the second image;
  • the first neural network model includes Splicing model and fusion model;
  • the splicing model can be realized by a variety of convolutional neural networks, such as residual network, VGG network, etc.;
  • the fusion model can also be realized by various convolutional neural networks, such as residual network, VGG network, etc. ;
  • the splicing model and the fusion model can be submodules or submodels in the first neural network model, or two separate neural network models; as shown in Figure 4, the method includes the following steps:
  • Step S402 acquiring the first image and the second image to be spliced.
  • Step S404 determining an initial spliced image of the first image and the second image.
  • Step S406 input the third sub-overlapping area and the fourth sub-overlapping area into the mosaic model, and search the seam between the third sub-overlapping area and the fourth sub-overlapping area through the mosaic model to obtain the third sub-overlapping area The stitching area corresponding to the fourth sub-overlapping area.
  • the third sub-overlapping area and the fourth sub-overlapping area included in the target overlapping area can be determined according to the target overlapping area of the initial stitching image, and the third sub-overlapping area
  • the overlapping area and the fourth sub-overlapping area are input into the pre-trained splicing model, and the splicing seam between the third sub-overlapping area and the fourth sub-overlapping area is searched by the splicing model, and the searched splicing area is output
  • the The stitching area is usually a point set or a stitching mask of the stitching, and the stitching area can also be a stitching line.
  • graphcut is a An energy optimization algorithm, which can be used for foreground and background segmentation, stereo vision or matting, etc. in the field of image processing; in this embodiment, the above-mentioned traditional calculation method can be distilled based on the pre-trained splicing model, without the need for CPU calculation, so that Realize hardware acceleration, speed up the processing speed and improve the processing efficiency.
  • Step S408 based on the patchwork area, fuse the third sub-overlapping area and the fourth sub-overlapping area to obtain an initial fused overlapping area.
  • the patchwork area is the point set of the non-feathering patchwork, based on the point set of the non-feathering patchwork, the third sub-overlapping area and the fourth sub- The overlapping area is directly fused to obtain the initial fusion overlapping area.
  • Step S410 input the initial fused overlapping area, the third sub-overlapping area, and the fourth sub-overlapping area into the fusion model, and perform fusion processing on the initial fused overlapping area, the third sub-overlapping area, and the fourth sub-overlapping area through the fusion model , to get the fusion overlapping area.
  • performing fusion processing on overlapping regions usually requires calculation and fusion of item-by-item data by CPU, and the calculation time is relatively long.
  • image fusion processing can be performed based on NN-blending, that is, a neural network fusion method, which can improve processing efficiency.
  • the initial fusion overlapping area obtained above can be combined with the two unfused original
  • the graph, that is, the third sub-overlapping area and the fourth sub-overlapping area, are sent into the fusion model together, and the initial fusion overlapping area, the third sub-overlapping area, and the fourth sub-overlapping area are fused through the fusion model, for example, after Image-to-image processing, etc., to obtain the result of fusion optimization.
  • the above-mentioned fusion model can be trained through the following steps 5 to 8:
  • Step five get the first picture.
  • Step 6 performing translation and/or rotation processing on the first picture to obtain the second picture.
  • the above-mentioned first picture can be any picture; that is, the above-mentioned second picture can be obtained after processing the first picture through small-scale translation and/or rotation; in actual implementation, the training process of the fusion model
  • the training sample can be constructed from a single picture, which is the first picture above, and a changed picture is obtained after the single picture is shifted and rotated in a small range, which is the second picture above.
  • step seven fusion processing is performed on the first picture and the second picture to obtain an initial fusion picture.
  • the two pictures may be fused through a randomly generated seam, for example, the fusion may be performed in an Alpha-Blending manner to obtain the above initial fused picture.
  • Step eight train the fusion model based on the first picture, the second picture and the initial fusion picture.
  • the first picture, the second picture and the initial fusion picture are used together as training samples to train the fusion model.
  • the above-mentioned training samples composed of the first picture, the second picture and the initial fusion picture can be input into the initial fusion model to output the fusion picture optimized for the initial fusion picture, based on the fusion picture and the first picture to determine Loss value, update the weight parameters of the initial fusion model based on the loss value; continue to perform the step of obtaining the first picture until the initial fusion model converges, and obtain the fusion model.
  • the loss value can be understood as the gap between the fused image optimized for the initial fused image output above and the first image;
  • the above weight parameters can include all parameters in the initial fused model, such as convolution kernel parameters, etc., in
  • the output of the initial fusion model can be supervised with the first picture, and the supervised LOSS is L1 distance, where the L1-loss of the patchwork area can be weighted by 2x.
  • Step S412 based on the fused overlapping area and the initial stitched image, determine the target stitched image corresponding to the first image and the second image.
  • the first image and the second image to be spliced are acquired.
  • An initial stitched image of the first image and the second image is determined.
  • the third sub-overlapping area and the fourth sub-overlapping area are input into the stitching model to obtain the stitching area corresponding to the third sub-overlapping area and the fourth sub-overlapping area.
  • the third sub-overlapping area and the fourth sub-overlapping area are fused to obtain an initial fused overlapping area.
  • the initial fused overlapping area, the third sub-overlapping area, and the fourth sub-overlapping area are input into the fusion model to obtain the fused overlapping area.
  • the first neural network model includes a stitching model and a fusion model, and the stitching region corresponding to the third sub-overlapping region corresponding to the first image and the fourth sub-overlapping region corresponding to the second image are searched out through the stitching model, and then Obtain the initial fusion overlapping area, and then obtain the fusion overlapping area through the fusion model.
  • This method only needs to pass the first neural network model to complete the seam search and fusion processing process, which improves the fusion processing efficiency and improves the efficiency of image stitching. Processing efficiency.
  • the embodiment of the present disclosure also provides another image mosaic method, which is implemented on the basis of the methods in the above embodiments; the method focuses on the use of the first neural network model to perform fusion processing on the target overlapping areas in the initial stitching image to obtain the target
  • the specific process of fusing the overlapping area corresponding to the overlapping area, and the specific process of determining the target stitched image corresponding to the first image and the second image based on the fusion overlapping area and the initial stitched image, in this method, the target overlapping area includes: The corresponding 3rd sub-overlapping area of an image and the 4th sub-overlapping area corresponding to the second image;
  • the first neural neural network model comprises splicing model; As shown in Figure 5, this method comprises the following steps:
  • Step S502 acquiring the first image and the second image to be spliced.
  • the first image and the second image are usually RAW images, and the format conversion of the RAW image is usually performed first, that is, the color channels of the first image and the second image are rearranged to facilitate subsequent lighting Compensation, seam search and other processing generally process the bayer-pattern of the original RAW domain into RGGB (R: Red, red; G: Green, green; G: Green, green; B: Blue, blue) arrangement, That is, the color channels of the first image and the second image are arranged in an RGGB manner.
  • both the first image and the second image can be RAW images, and stitching can be performed directly based on the RAW images. It is not necessary to convert the first image and the second image into RGB images before stitching. In contrast, there will be no lack of image details, so the RAW image stitching results are more conducive to later image detection, segmentation and other processing.
  • Step S504 determining an initial spliced image of the first image and the second image.
  • RAW image is usually CMOS (Complementary Metal-Oxide-Semiconductor, Complementary Metal Oxide Semiconductor) or CCD (Charge Coupled Device, Charge Coupled Device) image sensor converts the captured light source signal into digital signal raw data, which is not yet After processing, compared with RGB images, RAW images contain complete image details and have greater advantages in post-processing, such as adding and subtracting exposure, adjusting highlights/shadows, increasing and decreasing contrast, and adjusting color levels and curves. In actual implementation, when images need to be spliced, it is generally required that both the first image and the second image acquired first are RAW images.
  • Step S506 inputting the third sub-overlapping area and the fourth sub-overlapping area into the mosaic model to obtain the stitching area corresponding to the third sub-overlapping area and the fourth sub-overlapping area.
  • Step S508 perform feathering processing on the patchwork area to obtain a feathered overlapping area, and determine the feathered overlapping area as a fused overlapping area.
  • the above-mentioned feathering process usually blurs the edge of the pixel selection area, blending the selected area with the surrounding pixels, that is, blurring the connection between the inside and outside of the pixel selection area, and plays the role of gradient to achieve a natural connection effect.
  • the larger the feathering value the The wider the blur range, that is to say, the softer the color gradient; the smaller the feather value, the narrower the blur range, which can be adjusted according to the actual situation.
  • the feather value should be set smaller.
  • Repeated feathering is a trick of feathering.
  • the fusion overlapping area can be obtained based on the blending operation, that is, the fusion operation.
  • the feathering effect can be constructed for the patchwork area obtained by the patchwork search through the Alpha-Blending fusion method.
  • the feathered pixel value can be selected from 16-22pixel, and finally the fusion overlapping area is obtained.
  • Step S510 using the fused overlapping area to replace the target overlapping area in the initial stitched image to obtain the target stitched image corresponding to the first image and the second image.
  • the above-mentioned target stitched image is usually also a RAW image, that is, when the fusion overlapping area is used to replace After initially stitching target overlapping regions in the image, a stitched RAW image, that is, the above-mentioned target stitched image, can be obtained.
  • the target mosaic image can be detected, segmented or identified, and the corresponding analysis results can be obtained. Insight.
  • the core algorithm of ISP image Signal Processing, image signal processing
  • the ISP is mainly used to process the signal output by the front-end image sensor Post-processing, the main functions include linear correction, noise removal, dead point removal, interpolation, white balance, automatic exposure control, etc.
  • ISP includes Demasaic and other core algorithms. After processing by related algorithms, ISP can output images in RGB domain to Get stitching results that are easy to visualize.
  • the first image and the second image to be spliced are acquired.
  • the third sub-overlapping area and the fourth sub-overlapping area are input into the stitching model to obtain the stitching area corresponding to the third sub-overlapping area and the fourth sub-overlapping area.
  • Feathering is performed on the patchwork area to obtain a feathered overlapping area, and the feathered overlapping area is determined as a fusion overlapping area.
  • the target overlapping area in the initial stitched image is replaced by the fused overlapping area to obtain the target stitched image corresponding to the first image and the second image.
  • the first neural network model includes a splicing model, and the stitching area corresponding to the third sub-overlapping area corresponding to the first image and the fourth sub-overlapping area corresponding to the second image are searched out through the splicing model, and then processed through feathering In this way, the fusion overlapping area is obtained, which simplifies the fusion processing process, improves the fusion processing efficiency, and further improves the processing efficiency of image mosaic.
  • An embodiment of the present disclosure also provides a lighting compensation method, which includes the following steps:
  • Step 602 acquiring the first image and the second image to be spliced.
  • Step 604 perform illumination compensation on the second image based on the second neural network model and the first image.
  • This step 604 can be specifically implemented through the following steps 11 to 15:
  • Step 11 Determine a projection transformation matrix based on the first image and the second image.
  • the eleventh step can be specifically realized through the following steps M to O:
  • Step M extracting at least one first feature point in the first image and at least one second feature point in the second image.
  • Step N Determine at least one matching feature point pair based on at least one first feature point and at least one second feature point.
  • Step O determine a projection transformation matrix based on at least one matching feature point pair.
  • Step 12 based on the projection transformation matrix, determine the initial overlapping area of the first image and the second image; wherein, the initial overlapping area includes: the first sub-overlapping area corresponding to the first image and the second sub-overlapping area corresponding to the second image .
  • This step 12 can specifically be realized through the following steps P to S:
  • Step P acquiring boundary coordinates of the second image; wherein, the boundary coordinates are used to indicate an image area of the second image.
  • Step Q based on the projection transformation matrix and the boundary coordinates of the second image, determine boundary coordinates after projection transformation.
  • Step R based on the boundary coordinates after projection transformation, determine the second image after projection transformation.
  • Step S determining the overlapping image area of the projectively transformed second image and the first image as an initial overlapping area.
  • Step 13 input the first sub-overlapping area and the second sub-overlapping area into the second neural network model, and determine the first pixel value of each pixel in the first sub-overlapping area and the second sub-overlapping area through the second neural network model The mapping relationship of the second pixel value of the pixel at the same position in the area.
  • Step fourteen obtaining the mapping relationship output by the second neural network model.
  • Step fifteen for each color channel, based on the mapping relationship, match the pixel value of each pixel point in the color channel in the second image with the pixel value of each pixel point in the color channel in the first image, so as to match the pixel value in the color channel of each pixel point in the first image.
  • the second image is illuminated with compensation.
  • This step 604 can also specifically be realized through the following step 20:
  • Step 20 Input the first image and the second image into the second neural network model, perform illumination compensation on the second image based on the first image through the second neural network, and obtain an illumination-compensated second image.
  • Step 20 Input the first image and the second image into the second neural network model, perform illumination compensation on the second image based on the first image through the second neural network, and obtain an illumination-compensated second image.
  • illumination compensation is performed on the second image based on the second neural network model and the first image.
  • the pre-trained neural network model is used to perform light compensation on the second image, and there is no need to calculate each pixel based on the CPU, which saves the time for compensating all pixels in the image and improves the processing efficiency of light compensation.
  • the following provides a flow chart of a RAW domain image stitching method as shown in FIG. One image) and src image (corresponding to the second image above), and multiple images can also be spliced in this way.
  • the bayer-pattern (Bayer array) image in the original RAW domain is processed into an rggb arrangement, that is, for the bayer -tar performs format conversion, converts to rggb-tar image, performs format conversion on bayer-src, converts to rggb-src image.
  • the rggb-tar image and the rggb-src image are spatially registered as follows.
  • the specific spatial registration process can be as follows: first, feature points are extracted from the two images, which can be extracted using traditional SIFT or neural network-based SuperPoint. feature points, and then match the feature points to obtain matching feature point pairs. Based on the feature point matching information of the matching feature point pairs, calculate the projection transformation matrix. Based on the projection transformation matrix and the boundary coordinates of the rggb-src image, calculate rggb- Intersect the projected area information of src with the rggb-tar image to obtain the overlapping area (corresponding to the above initial overlapping area).
  • light compensation can be performed on the rggb-src image, and the color consistency of different camera images can be optimized through light compensation.
  • the following methods can be used, one of which is to combine the above rggb-tar image with The two small images of the overlapping area of the rggb-src image are input to the neural network model, and the LUT table with a GT of 0-65535 is used for learning.
  • LUT tables of 0-65535 can be calculated through the neural network model, based on The LUT table performs light compensation on the rggb-src image; another way is to input the rggb-tar image and the rggb-src image into the neural network model, use GT as the original rggb-tar image, and perform light compensation based on histogram matching
  • the rggb-src image is supervised, and finally the rggb-src image after illumination compensation can be calculated through the neural network model.
  • the rggb-src image and the rggb-src image after illumination compensation can be projected and changed. Specifically, based on the projection transformation matrix, the rggb-src image after illumination compensation can be projected On the rggb-tar image, the initial stitching image is obtained; based on the rggb-src image after illumination compensation, the new overlapping area is re-extracted, and the two small images based on the new overlapping area of the rggb-tar image and the rggb-src image are carried out Patchwork search and blending (fusion), specifically, you can input two small images of the new overlapping area into the pre-trained stitching model, and output the point set of the stitching mask or stitching through the stitching model (corresponding to the above stitching Seam area), and then through Alpha-blending fusion, or fusion based on a pre-trained fusion model, to obtain a fused overlapping area (corresponding to the
  • the processes of illumination compensation, seam search and fusion in the above-mentioned RAW domain image stitching method can be realized based on the neural network, so that hardware acceleration can be realized, the processing speed can be accelerated, and the processing efficiency can be improved.
  • This method provides an intelligent analysis of RAW domain images. A solution with a large field of view.
  • RAW domain images it can also handle steps such as splicing and light alignment, so as to fuse the images to be stitched with high quality as much as possible.
  • stitching RAW domain images you can obtain a large field of view.
  • the original RAW domain image is convenient for low-light RAW domain detection and recognition tasks under a large field of view.
  • an embodiment of the present disclosure also provides a schematic structural diagram of an image stitching device.
  • the device includes: an acquisition module 70 configured to acquire a first image and a second image to be stitched;
  • the determination module 71 may be configured to determine the initial stitched image of the first image and the second image;
  • the fusion module 72 may be configured to fuse the target overlapping area in the initial stitched image by using the first neural network model processing to obtain the fused overlapping area corresponding to the target overlapping area;
  • the second determination module 73 may be configured to determine the target fused image corresponding to the first image and the second image based on the fused overlapping area and the initial stitched image.
  • the above-mentioned image stitching device first acquires the first image and the second image to be stitched; determines the initial stitched image of the first image and the second image; then uses the first neural network model to perform fusion processing on the target overlapping area in the initial stitched image , to obtain the fused overlapping area corresponding to the target overlapping area; finally, based on the fused overlapping area and the initial stitched image, determine the target stitched image corresponding to the first image and the second image.
  • the first neural network model is used to perform fusion processing on the target overlapping area in the initial spliced image to obtain the corresponding fused overlapping area, which does not need to be based on CPU Fusion correlation calculation is performed for each pixel in the initial stitching image, which saves the time for fusion calculation of all pixels in the initial stitching image, improves the fusion processing efficiency, and further improves the processing efficiency of image stitching.
  • the device is further configured to: perform illumination compensation on the second image; correspondingly, the first determination module may also be configured to: determine an initial spliced image based on the first image and the second image after illumination compensation .
  • the first determination module may also be configured to: perform illumination compensation on the second image based on the second neural network model and the first image.
  • the first determination module may also be configured to: determine a projection transformation matrix based on the first image and the second image; determine an initial overlapping area of the first image and the second image based on the projection transformation matrix; wherein, The initial overlapping area includes: the first sub-overlapping area corresponding to the first image and the second sub-overlapping area corresponding to the second image; the first sub-overlapping area and the second sub-overlapping area are input to the second neural network model, and the Two neural network models determine the mapping relationship between the first pixel value of each pixel in the first sub-overlapping area and the second pixel value of the same position pixel in the second sub-overlapping area; obtain the mapping relationship output by the second neural network model; for each The color channel, based on the mapping relationship, matches the pixel value of each pixel point in the color channel in the second image with the pixel value of each pixel point in the color channel in the first image, so as to perform light compensation on the second image.
  • the first determination module may also be configured to: acquire boundary coordinates of the second image; wherein, the boundary coordinates are used to indicate an image area of the second image; based on the projection transformation matrix and the boundary coordinates of the second image, Determine the boundary coordinates after projective transformation; determine the second image after projective transformation based on the boundary coordinates after projective transformation; determine the overlapping image area of the second image after projective transformation and the first image as the initial overlapping region.
  • the first determination module may also be configured to: extract at least one first feature point in the first image, and at least one second feature point in the second image; based on at least one first feature point and Determining at least one pair of matching feature points for at least one second feature point; determining a projection transformation matrix based on the at least one pair of matching feature points.
  • the first determination module may also be configured to: input the first image and the second image into the second neural network model, and perform illumination compensation on the second image based on the first image through the second neural network , to obtain the second image after illumination compensation.
  • the target overlapping area includes: the third sub-overlapping area corresponding to the first image and the fourth sub-overlapping area corresponding to the second image;
  • the first neural network model includes a stitching model and a fusion model;
  • the fusion module can also be configured It is used to: input the third sub-overlapping area and the fourth sub-overlapping area into the splicing model, and search the seam between the third sub-overlapping area and the fourth sub-overlapping area through the splicing model to obtain the third sub-overlapping area region and the patchwork area corresponding to the fourth sub-overlapping area; based on the patchwork area, the third sub-overlapping area and the fourth sub-overlapping area are fused to obtain the initial fusion overlapping area; the initial fusion overlapping area, the third sub-overlapping area , and the fourth sub-overlapping area are input to the fusion model, and the initial fusion overlapping area, the third sub-overlapping area, and the fourth sub-overlapping area are fused through the fusion model to obtain the fusion overlapping area.
  • the target overlapping area includes: a third sub-overlapping area corresponding to the first image and a fourth sub-overlapping area corresponding to the second image;
  • the first neural network model includes a stitching model;
  • the fusion module can also be configured to : Input the third sub-overlapping area and the fourth sub-overlapping area into the stitching model to obtain the stitching area corresponding to the third sub-overlapping area and the fourth sub-overlapping area; perform feathering on the stitching area to obtain the feathered overlap area, the overlapping area after feathering is determined as the fusion overlapping area.
  • the fusion module can also be configured to: acquire the first picture; perform translation and/or rotation processing on the first picture to obtain the second picture; perform fusion processing on the first picture and the second picture to obtain the initial Fusion pictures; based on the first picture, the second picture and the initial fusion picture, train the fusion model.
  • the second determining module may also be configured to: replace the target overlapping area in the initial stitched image with the fused overlapping area to obtain the target stitched image corresponding to the first image and the second image.
  • color channels of the first image and the second image are arranged in an RGGB manner.
  • the implementation principle and technical effect of the image stitching device provided by the embodiment of the present disclosure are the same as those of the aforementioned image stitching method embodiment.
  • the parts not mentioned in the image stitching device embodiment can refer to the aforementioned image stitching method Corresponding content in the embodiment.
  • An embodiment of the present disclosure also provides an electronic device, including a processing device and a storage device, the storage device stores a computer program, and the computer program executes the image stitching method according to any one of the above when the processed device is run.
  • An embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processing device, the steps of the above-mentioned image stitching method are executed.
  • An embodiment of the present disclosure also provides a computer program product, including a computer-readable storage medium storing program codes, the program codes including instructions capable of executing the steps of the above image stitching method.
  • the computer program product of the image mosaic method, device, and electronic equipment provided by the embodiments of the present disclosure includes a computer-readable storage medium storing program codes, and the instructions included in the program codes can be used to execute the methods described in the preceding method embodiments. For the specific implementation of the method, reference may be made to the method embodiments, which will not be repeated here.
  • connection should be interpreted in a broad sense, for example, it can be a fixed connection or a detachable connection , or integrally connected; it may be mechanically connected or electrically connected; it may be directly connected or indirectly connected through an intermediary, and it may be the internal communication of two components.
  • installation e.g., it can be a fixed connection or a detachable connection , or integrally connected; it may be mechanically connected or electrically connected; it may be directly connected or indirectly connected through an intermediary, and it may be the internal communication of two components.
  • the computer software product is stored in a storage medium, including several
  • the instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present disclosure.
  • the aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc, etc., which can store program codes. .
  • the present application provides an image mosaic method and electronic equipment, acquiring the first image and the second image to be mosaic; determining the initial mosaic image of the first image and the second image; utilizing the first neural network model to The target overlapping area is fused to obtain a fused overlapping area corresponding to the target overlapping area; based on the fused overlapping area and the initial stitched image, the target stitched image corresponding to the first image and the second image is determined.
  • the first neural network model is used to fuse the target overlapping area in the initial spliced image to obtain the corresponding fused overlapping area. Fusion correlation calculation is performed for each pixel in the initial stitching image, which saves the time for fusion calculation of all pixels in the initial stitching image, improves the fusion processing efficiency, and further improves the processing efficiency of image stitching.
  • the image stitching method and electronic device of the present application are reproducible and can be used in various industrial applications.
  • the image stitching method and electronic device of the present application can be used in the technical field of image processing.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Processing (AREA)

Abstract

本公开实施例提供了一种图像拼接方法和电子设备，获取待拼接的第一图像和第二图像；确定第一图像和第二图像的初始拼接图像；利用第一神经网络模型对初始拼接图像中的目标重叠区域进行融合处理，得到目标重叠区域所对应的融合重叠区域；基于融合重叠区域和初始拼接图像，确定第一图像和第二图像所对应的目标拼接图像。该方式中，在确定第一图像和第二图像的初始拼接图像后，采用第一神经网络模型对初始拼接图像中的目标重叠区域进行融合处理，得到对应的融合重叠区域，不需要基于CPU对初始拼接图像中的每个像素进行融合相关计算，节省了对初始拼接图像中所有像素进行融合计算的时间，提高了融合处理效率，进而提高了图像拼接的处理效率。

Description

图像拼接方法和电子设备
相关申请的交叉引用
本申请要求于2021年08月26日提交中国国家知识产权局的申请号为202110990427.0、名称为“图像拼接方法、装置和电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本公开涉及图像处理技术领域，尤其是涉及一种图像拼接方法和电子设备。
背景技术
图像拼接是将具有重叠视场的多幅图像组合在一起，以产生具有更大视场角及更高分辨率的图像的过程。在图像拼接过程中通常需要对待拼接图片进行融合处理，相关技术中，通常是基于CPU(central processing unit，中央处理器)对初始拼接后的拼接图片中的每个像素进行融合相关的计算处理，由于CPU对每个像素计算时需要一定的处理时间，导致对初始拼接后的拼接图片中所有像素进行融合计算的时间较长，降低了融合处理效率，进而降低了图像拼接的处理效率。
发明内容
本公开提供了一种图像拼接方法和电子设备，以提高图像拼接的处理效率。
本公开提供的一种图像拼接方法，方法包括：获取待拼接的第一图像和第二图像；确定第一图像和第二图像的初始拼接图像；利用第一神经网络模型对初始拼接图像中的目标重叠区域进行融合处理，得到目标重叠区域所对应的融合重叠区域；基于融合重叠区域和初始拼接图像，确定第一图像和第二图像所对应的目标拼接图像。
可选地，确定第一图像和第二图像的初始拼接图像之前，方法还包括：对第二图像进行光照补偿；相应的，确定第一图像和第二图像的初始拼接图像，包括：基于第一图像和光照补偿后的第二图像，确定初始拼接图像。
可选地，对第二图像进行光照补偿的步骤包括：基于第二神经网络模型和第一图像，对第二图像进行光照补偿。
可选地，基于第二神经网络模型和第一图像，对第二图像进行光照补偿，包括：基于第一图像和第二图像，确定投影变换矩阵；基于投影变换矩阵，确定第一图像和第二图像的初始重叠区域；其中，初始重叠区域包括：第一图像对应的第一子重叠区域以及第二图像对应的第二子重叠区域；将第一子重叠区域和第二子重叠区域，输入至第二神经网络模型，通过第二神经网络模型确定第一子重叠区域中各个像素的第一像素值与第二子重叠区域中相同位置像素的第二像素值的映射关系；获取第二神经网络模型输出的映射关系；针对各个颜色通道，基于映射关系，将第二图像中的各个像素点在颜色通道的像素值，与第一图像中的各个像素点在颜色通道的像素值进行匹配,以对第二图像进行光照补偿。
可选地，基于投影变换矩阵，确定第一图像和第二图像的初始重叠区域的步骤包括：获取第二图像的边界坐标；其中，边界坐标用于指示第二图像的图像区域；基于投影变换矩阵和第二图像的边界坐标，确定投影变换后的边界坐标；基于投影变换后的边界坐标，确定投影变换后的第二图像；将投影变换后的第二图像与第一图像的重合图像区域，确定为初始重叠区域。
可选地，基于第一图像和第二图像，确定投影变换矩阵的步骤包括：提取第一图像中的至少一个第 一特征点，以及第二图像中的至少一个第二特征点；基于至少一个第一特征点和至少一个第二特征点，确定至少一个匹配特征点对；基于至少一个匹配特征点对，确定投影变换矩阵。
可选地，基于第二神经网络模型和第一图像，对第二图像进行光照补偿的步骤包括：将第一图像和第二图像，输入至第二神经网络模型中，通过第二神经网络基于第一图像对第二图像进行光照补偿，得到光照补偿后的第二图像。
可选地，目标重叠区域包括：第一图像对应的第三子重叠区域和第二图像对应的第四子重叠区域；第一神经神经网络模型包括拼接模型和融合模型；利用第一神经网络模型对初始拼接图像中的目标重叠区域进行融合处理，得到目标重叠区域所对应的融合重叠区域的步骤包括：将第三子重叠区域和第四子重叠区域，输入至拼接模型，通过拼接模型对第三子重叠区域和第四子重叠区域之间的拼缝进行搜索，得到第三子重叠区域和第四子重叠区域对应的拼缝区域；基于拼缝区域，对第三子重叠区域和第四子重叠区域进行融合，得到初始融合重叠区域；将初始融合重叠区域，第三子重叠区域，以及第四子重叠区域输入至融合模型，通过融合模型对初始融合重叠区域，第三子重叠区域，以及第四子重叠区域进行融合处理，得到融合重叠区域。
可选地，目标重叠区域包括：第一图像对应的第三子重叠区域和第二图像对应的第四子重叠区域；第一神经神经网络模型包括拼接模型；利用第一神经网络模型对初始拼接图像中的目标重叠区域进行融合处理，得到目标重叠区域所对应的融合重叠区域的步骤包括：将第三子重叠区域和第四子重叠区域，输入至拼接模型，得到第三子重叠区域和第四子重叠区域对应的拼缝区域；对拼缝区域进行羽化处理，得到羽化后的重叠区域，将羽化后的重叠区域确定为融合重叠区域。
可选地，融合模型通过以下方式训练得到：获取第一图片；对第一图片进行平移和/或旋转处理，得到第二图片；对第一图片和第二图片进行融合处理，得到初始融合图片；基于第一图片、第二图片和初始融合图片，训练融合模型。
可选地，基于融合重叠区域和初始拼接图像，确定第一图像和第二图像所对应的目标拼接图像的步骤包括：使用融合重叠区域替换初始拼接图像中的目标重叠区域，得到第一图像和第二图像所对应的目标拼接图像。
可选地，第一图像和第二图像的颜色通道为RGGB排列方式。
可选地，第一图像和所述第二图像为RAW域图像，所述确定第一图像和第二图像的初始拼接图像，包括：
对所述第一图像进行格式转换，得到RGGB排列方式的第一图像，以及对所述第二图像进行格式转换，得到RGGB排列方式的第二图像；
基于RGGB排列方式的第一图像和RGGB排列方式的第二图像确定所述初始拼接图像。
本公开提供的一种电子设备，包括处理设备和存储装置，存储装置存储有计算机程序，计算机程序在被处理设备运行时执行如上所述的图像拼接方法。
本公开提供的一种机器可读存储介质，机器可读存储介质存储有计算机程序，计算机程序被处理设备运行时执行如上所述的图像拼接方法。
本公开提供的一种计算机程序产品，包括存储有程序代码的计算机可读存储介质，所述程序代码包 括的指令能够执行如上所述的图像拼接方法。
本公开提供的图像拼接方法和电子设备，首先获取待拼接的第一图像和第二图像；确定第一图像和第二图像的初始拼接图像；然后利用第一神经网络模型对初始拼接图像中的目标重叠区域进行融合处理，得到目标重叠区域所对应的融合重叠区域；最后基于融合重叠区域和初始拼接图像，确定第一图像和第二图像所对应的目标拼接图像。该方式中，在确定第一图像和第二图像的初始拼接图像后，采用第一神经网络模型对初始拼接图像中的目标重叠区域进行融合处理，得到对应的融合重叠区域，不需要基于CPU对初始拼接图像中的每个像素进行融合相关计算，节省了对初始拼接图像中所有像素进行融合计算的时间，提高了融合处理效率，进而提高了图像拼接的处理效率。
附图说明
为了更清楚地说明本公开具体实施方式或相关技术中的技术方案，下面将对具体实施方式或相关技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本公开的一些实施方式，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本公开实施例提供的一种电子系统的结构示意图；
图2为本公开实施例提供的一种图像拼接方法的流程图；
图3为本公开实施例提供的另一种图像拼接方法的流程图；
图4为本公开实施例提供的另一种图像拼接方法的流程图；
图5为本公开实施例提供的另一种图像拼接方法的流程图；
图6为本公开实施例提供的一种RAW域图像拼接方法的流程图；
图7为本公开实施例提供的一种图像拼接装置的结构示意图。
具体实施方式
下面将结合实施例对本公开的技术方案进行清楚、完整地描述，显然，所描述的实施例是本公开一部分实施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。
近年来，基于人工智能的计算机视觉、深度学习、机器学习、图像处理、图像识别等技术研究取得了重要进展。人工智能(Artificial Intelligence，AI)是研究、开发用于模拟、延伸人的智能的理论、方法、技术及应用系统的新兴科学技术。人工智能学科是一门综合性学科，涉及芯片、大数据、云计算、物联网、分布式存储、深度学习、机器学习、神经网络等诸多技术种类。计算机视觉作为人工智能的一个重要分支，具体是让机器识别世界，计算机视觉技术通常包括人脸识别、活体检测、指纹识别与防伪验证、生物特征识别、人脸检测、行人检测、目标检测、行人识别、图像处理、图像识别、图像语义理解、图像检索、文字识别、视频处理、视频内容识别、三维重建、虚拟现实、增强现实、同步定位与地图构建(SLAM)、计算摄影、机器人导航与定位等技术。随着人工智能技术的研究和进步，该项技术在众多领域展开了应用，例如安全防范、城市管理、交通管理、楼宇管理、园区管理、人脸通行、人脸考勤、物流管理、仓储管理、机器人、智能营销、计算摄影、手机影像、云服务、智能家居、穿戴设备、无人驾驶、自动驾驶、智能医疗、人脸支付、人脸解锁、指纹解锁、人证核验、智慧屏、智能电视、摄像机、移动互联网、网络直播、美颜、美妆、医疗美容、智能测温等领域。
图像拼接是将具有重叠视场的多幅图像组合在一起，以产生更大的视场角及高分辨率图像的过程。 普通人的单只眼睛的视场角大概120°左右，两只眼睛的视场角一般能160°至220°左右，而普通相机的FOV(Field Of Vision，视场角)一般只有40°至60°左右，难以在对细节物体进行高清成像的同时保证大的成像视野，通过图像拼接技术能够将多个小视场角的摄像机组合为大视场角的多摄相机，在安全防范、远程会议、运动赛事导播等领域有着重要的应用价值。相关技术中，通常是基于RGB(R：Red，红色；G：Green，绿色；B：Blue，蓝色)域图像进行图像拼接处理，而当前较为新颖的图像处理及分析过程通常是基于RAW域图像实现的，比如，通过RAW域图像的检测、分割或识别等来解决RGB域图像无法解决的暗光、逆光等场景问题，在对图像处理及分析主要集中在RAW域图像的情况下，当对相应的RGB域图像进行图像拼接时，由于RGB域图像与RAW域图像相比，缺少了一部分图像细节，导致在RGB域的拼接图像缺少了相应的图像分析结果，从而不利于后期的图像检测、分割等处理；相关技术中，通常是基于CPU(central processing unit，中央处理器)对初始拼接后的拼接图片中的每个像素进行融合相关的计算处理，由于CPU对每个像素计算时需要一定的处理时间，导致对初始拼接后的拼接图片中所有像素进行融合计算的时间较长，降低了融合处理效率，进而降低了图像拼接的处理效率。基于此，本公开实施例提供了一种图像拼接方法、装置和电子设备，该技术可以应用于需要对图像进行拼接处理的应用中，以下对本公开实施例进行详细介绍。
首先，参照图1来描述用于实现本公开实施例的图像拼接方法、装置和电子设备的示例电子系统100。
如图1所示的一种电子系统的结构示意图，电子系统100可以包括一个或多个处理设备102、一个或多个存储装置104、输入装置106、输出装置108以及一个或多个图像采集设备110，这些组件通过总线系统112和/或其它形式的连接机构(未示出)互连。应当注意，图1所示的电子系统100的组件和结构只是示例性的，而非限制性的，根据需要，所述电子系统也可以具有其他组件和结构。
所述处理设备102可以是网关，也可以为智能终端，或者是包含中央处理单元(CPU)或者具有数据处理能力和/或指令执行能力的其它形式的处理单元的设备，可以对所述电子系统100中的其它组件的数据进行处理，还可以控制所述电子系统100中的其它组件以执行期望的功能。
所述存储装置104可以包括一个或多个计算机程序产品，所述计算机程序产品可以包括各种形式的计算机可读存储介质，例如易失性存储器和/或非易失性存储器。所述易失性存储器例如可以包括随机存取存储器(RAM)和/或高速缓冲存储器(cache)等。所述非易失性存储器例如可以包括只读存储器(ROM)、硬盘、闪存等。在所述计算机可读存储介质上可以存储一个或多个计算机程序指令，处理设备102可以运行所述程序指令，以实现下文所述的本公开实施例中(由处理设备实现)的客户端功能以及/或者其它期望的功能。在所述计算机可读存储介质中还可以存储各种应用程序和各种数据，例如所述应用程序使用和/或产生的各种数据等。
所述输入装置106可以是用户用来输入指令的装置，并且可以包括键盘、鼠标、麦克风和触摸屏等中的一个或多个。
所述输出装置108可以向外部(例如，用户)输出各种信息(例如，图像或声音)，并且可以包括显示器、扬声器等中的一个或多个。
所述图像采集设备110可以采集预览视频帧或图像数据，并且将采集到的预览视频帧或图像数据存储在所述存储装置104中以供其它组件使用。
示例性地，用于实现根据本公开实施例的图像拼接方法、装置和电子设备的示例电子系统中的各器 件可以集成设置，也可以分散设置，诸如将处理设备102、存储装置104、输入装置106和输出装置108集成设置于一体，而将图像采集设备110设置于可以采集到目标图像的指定位置。当上述电子系统中的各器件集成设置时，该电子系统可以被实现为诸如相机、智能手机、平板电脑、计算机、车载终端等智能终端，也可以是服务器。
本实施例提供了一种图像拼接方法，该方法由上述电子系统中的处理设备执行；该处理设备可以是具有数据处理能力的任何设备或芯片。如图2所示，该方法包括如下步骤：
步骤S202，获取待拼接的第一图像和第二图像。
上述第一图像和第二图像中，可以以第一图像作为基准图像，以第二图像作为待配准图像，也可以以第二图像作为基准图像，以第一图像作为待配准图像，具体可以根据实际需求进行选择；为方便说明，以第一图像为基准图像，第二图像为待配准图像为例，该第一图像通常位于基准图像坐标系中，该第一图像可以以tar图像表示等；该第二图像通常位于待配准图像坐标系中，该第二图像可以以src图像表示等；该第一图像和第二图像可以是RAW域图像，也可以是RGB域图像等，在实际实现时，当需要对图像进行拼接处理时，通常需要先获取到待拼接的第一图像和第二图像，且该第一图像和第二图像通常是具有一部分重叠视场的图像。
步骤S204，确定第一图像和第二图像的初始拼接图像。
将获取到的第一图像和第二图像进行初始拼接，比如，如果第一图像作为基准图像，第二图像作为待配准图像，可以将第二图像投影至第一图像进行初始拼接，得到第一图像和第二图像拼接的初始拼接图像。
步骤S206，利用第一神经网络模型对初始拼接图像中的目标重叠区域进行融合处理，得到目标重叠区域所对应的融合重叠区域。
上述第一神经网络模型可以通过多种卷积神经网络实现，如残差网络、VGG网络等；由于第一图像和第二图像是具有一部分重叠视场的图像，该目标重叠区域可以理解为，对第一图像和第二图像进行初始拼接处理后的重合区域；在实际实现时，在确定上述初始拼接图像后，可以从该初始拼接图像中确定第一图像和第二图像相重合的目标重叠区域，利用第一神经网络对该目标重叠区域进行融合处理，得到对应的融合重叠区域。
步骤S208，基于融合重叠区域和初始拼接图像，确定第一图像和第二图像所对应的目标拼接图像。
在实际实现时，在确定上述融合重叠区域后，可以基于该融合重叠区域和初始拼接图像，得到第一图像和第二图像的目标拼接图像；该目标拼接图像可以是将两张或数张有重叠部分的图像拼成一幅无缝的全景图或高分辨率图像；其中，两张或数张有重叠部分的图像可能是不同时间、不同视角或者不同传感器获得的图像。
上述图像拼接方法，首先获取待拼接的第一图像和第二图像；确定第一图像和第二图像的初始拼接图像；然后利用第一神经网络模型对初始拼接图像中的目标重叠区域进行融合处理，得到目标重叠区域所对应的融合重叠区域；最后基于融合重叠区域和初始拼接图像，确定第一图像和第二图像所对应的目标拼接图像。该方式中，在确定第一图像和第二图像的初始拼接图像后，采用第一神经网络模型对初始拼接图像中的目标重叠区域进行融合处理，得到对应的融合重叠区域，不需要基于CPU对初始拼接图像中的每个像素进行融合相关计算，节省了对初始拼接图像中所有像素进行融合计算的时间，提高了融 合处理效率，进而提高了图像拼接的处理效率。
本公开实施例还提供另一种图像拼接方法，该方法在上述实施例方法的基础上实现；如图3所示，该方法可以包括如下步骤：
步骤S302，获取待拼接的第一图像和第二图像。
步骤S304，对第二图像进行光照补偿。
该步骤S304具体可以通过下述步骤一实现：
步骤一，基于第二神经网络模型和第一图像，对第二图像进行光照补偿。
该第二神经网络模型可以通过多种卷积神经网络实现，如残差网络、VGG网络等；该第二神经网络模型可以为不同于第一神经网络模型的网络模型，也可以是第一神经网络模型，只不过进行光照补偿的过程由第一神经网络模型中的某个子模型或子模块执行即可，在实际实现时，考虑到由于光线原因，所拍摄的第二图像可能会存在因光线不平衡而造成的色彩偏差，为了抵消第二图像中存在的色彩偏差，可以基于预先训练好的第二神经网络模型和第一图像，对该第二图像进行光照补偿。
该步骤一具体可以通过下述步骤A至步骤D实现：
步骤A，基于第一图像和第二图像，确定投影变换矩阵。
在实际实现时，可以根据获取到的第一图像和第二图像，确定投影变换矩阵，其中，第一图像和第二图像通常具有相同数量的颜色通道，比如，第一图像有R、G、G和B四个颜色通道，第二图像同样也有R、G、G和B四个颜色通道；具体的，该步骤A具体可以通过步骤a至步骤c实现：
步骤a，提取第一图像中的至少一个第一特征点，以及第二图像中的至少一个第二特征点。
为了能够更好的进行图像匹配，通常需要在图像中选择具有代表性的区域，如图像中的角点、边缘点、暗区的亮点或亮区的暗点等，上述第一特征点可以是从第一图像中提取出的角点、边缘点、暗区的亮点或亮区的暗点等，该第一特征点的数量可以是一个或多个；上述第二特征点可以是从第二图像中提取的角点、边缘点、暗区的亮点或亮区的暗点等，该第二特征点的数量可以是一个或多个。
本实施例中，第一图像的每个像素的颜色通道中，每种颜色通道预设有对应的第一权重值；第二图像的颜色通道中，每种颜色通道预设有对应的第二权重值；在实际实现时，上述第一权重值和第二权重值通常是预先设定好的固定值，具体可以根据实际需求进行设置，在此不作限定，比如，以第一图像和第二图像都具有R、G、G和B这四个颜色通道为例，第一图像中，R通道对应的第一权重值为0.3，两个G通道对应的第一权重值均为0.2，B通道对应的第一权重值为0.3；第二图像中，R通道对应的第二权重值为0.3，两个G通道对应的第二权重值均为0.2，B通道对应的第二权重值为0.3。
上述步骤a中，提取第一图像中的至少一个第一特征点的步骤可以包括步骤a0至步骤a3：
步骤a0，针对第一图像中的每个像素，将该像素中每种颜色通道的分量乘以该颜色通道对应的第一权重值，得到每种颜色通道的第一计算结果。
上述颜色通道可以理解为保存图像颜色信息的通道，比如RGB图像有三个颜色通道，分别为R通道、G通道和B通道；上述每种颜色通道的分量可以理解为每种颜色通道的亮度值；在实际实现时，第一图像中通常包括多个像素，每个像素通常具有多个颜色通道，对于每个像素来说，可以将该像素中每种颜色通道的分量乘以所对应的第一权重值，得到该颜色通道的第一计算结果，比如，以第一图像具有R、G、G和B这四个颜色通道，其中，R通道对应的第一权重值为0.3，两个G通道对应的第一权重值 均为0.2，B通道对应的第一权重值为0.3，其中一个指定像素的R通道对应的分量为150、两个G通道对应的分量均为100，B通道对应的分量为80为例，则该指定像素中，R通道的第一计算结果为150*0.3＝45，两个G通道的第一计算结果为100*0.2＝20，B通道的第一计算结果为80*0.3＝24。
步骤a1，将该像素中每种颜色通道的第一计算结果进行加和，得到该像素对应的第一灰度值。
当通过上述步骤a0得到第一图像中，每个像素中的每种颜色通道的第一计算结果后，可以将该像素中每种颜色通道的第一计算结果相加，得到该像素的第一灰度值，比如，仍以上述步骤a0中的指定像素为例，可以将该指定像素中的每种颜色通道的第一计算结果相加，得到该指定像素对应的第一灰度值，即45+20+20+24＝109。
步骤a2，基于第一图像中每个像素对应的第一灰度值，确定第一图像的灰度图。
当通过上述步骤a0和步骤a1得到第一图像中每个像素对应的第一灰度值后，可以根据所得到的多个第一灰度值，得到该第一图像的灰度图。
步骤a3，从第一图像的灰度图中，提取至少一个第一特征点。
在实际实现时，可以基于SIFT(Scale-Invariant Feature Transform，尺度不变特征变换)或SuperPoint等算法，从所得到的第一图像的灰度图中，提取出一个或多个第一特征点，具体可参考相关技术中，采用SIFT或SuperPoint等方式提取特征点的过程，在此不再赘述；其中，SIFT具有尺度不变性，可在图像中检测出关键点，是一种局部特征描述子；SuperPoint是一种基于自监督训练的特征点检测和描述符提取方法。
上述步骤a中，提取第二图像中的至少一个第二特征点的步骤可以包括步骤a4至步骤a7：
步骤a4，针对第二图像中的每个像素，将该像素中每种颜色通道的分量乘以该颜色通道对应的第二权重值，得到每种颜色通道的第二计算结果。
在实际实现时，第二图像中通常也包括多个像素，每个像素通常具有多个颜色通道，对于每个像素来说，可以将该像素中每种颜色通道的分量乘以所对应的第二权重值，得到该颜色通道的第二计算结果，比如，仍以第二图像具有R、G、G和B这四个颜色通道，其中，R通道对应的第一权重值为0.3，两个G通道对应的第一权重值均为0.2，B通道对应的第一权重值为0.3，其中一个目标像素的R通道对应的分量为100、两个G通道对应的分量均为120，B通道对应的分量为150为例，则该目标像素中，R通道的第二计算结果为100*0.3＝30，两个G通道的第二计算结果为120*0.2＝24，B通道的第二计算结果为150*0.3＝45。
步骤a5，将该像素中每种颜色通道的第二计算结果进行加和，得到该像素对应的第二灰度值。
当通过上述步骤a4得到第二图像中，每个像素中的每种颜色通道的第二计算结果后，可以将该像素中每种颜色通道的第二计算结果相加，得到该像素的第二灰度值，比如，仍以上述步骤a4中的目标像素为例，可以将该目标像素中的每种颜色通道的第二计算结果相加，得到该目标像素对应的第二灰度值，即30+24+24+45＝123。
步骤a6，基于第二图像中每个像素对应的第二灰度值，确定第二图像的灰度图。
当通过上述步骤a4和步骤a5得到第二图像中每个像素对应的第二灰度值后，可以根据所得到的多个第二灰度值，得到该第二图像的灰度图。
步骤a7，从第二图像的灰度图中，提取至少一个第二特征点。
在实际实现时，可以基于SIFT或SuperPoint等算法，从所得到的第二图像的灰度图中，提取出一个或多个第二特征点。
步骤b，基于至少一个第一特征点和至少一个第二特征点，确定至少一个匹配特征点对。
当通过上述步骤a提取到第一图像中的至少一个第一特征点和第二图像中的至少一个第二特征点后，可以进行特征点匹配，具体的，可以基于至少一个第一特征点和至少一个第二特征点，采用KNN(K-NearestNeighbor，K最邻近分类算法)或RANSAC(RANdomSAmple Consensus，随机抽样一致性算法)等算法，得到第一图像和第二图像中的一个或多个匹配特征点对，该匹配特征点对的数量一般不少于四对；具体可参考相关技术中，采用KNN或RANSAC等方式进行特征点匹配的过程，在此不再赘述；其中，KNN是每个样本都可以用它最接近的K个邻近值来代表，其核心思想是，如果一个样本在特征空间中的K个最相邻的样本中的大多数属于某一个类别，则该样本也属于这个类别，并具有这个类别上样本的特性；RANSAC为随机采样一致性匹配，该方式利用匹配点计算两个图像之间单应矩阵，然后利用重投影误差来判定某一个匹配是不是正确的匹配。
步骤c，基于至少一个匹配特征点对，确定投影变换矩阵。
在确定至少一个匹配特征点对后，可以基于匹配特征点对的匹配信息，如匹配特征点对的坐标等，计算投影变换矩阵，具体可参考相关技术中，根据匹配特征点对，计算投影变换矩阵的过程，在此不再赘述。基于该投影变换矩阵，可以将第二图像投影至第一图像，该投影变换矩阵的作用方式是，对于第二图像中，每个像素包括的每个颜色通道，都按照所计算的投影变换矩阵进行投影变换，另外，在第二图像投影至第一图像后，变换出来的对应的坐标可能是小数，这时通常需要依据临近像素的值来对该坐标进行插值处理，具体可参考相关技术中的插值处理方式，比如，可以采用Cubic双立方插值方式等，以降低解析力影响，其中，该解析力可以理解为投影变换后的第二图像的清晰度。
步骤B，基于投影变换矩阵，确定第一图像和第二图像的初始重叠区域；其中，初始重叠区域包括：第一图像对应的第一子重叠区域以及第二图像对应的第二子重叠区域。
基于上述步骤所确定的投影变换矩阵，将第二图像投影至第一图像，由于第一图像和第二图像是具有一部分重叠视场的图像，因此，在完成投影变换后，可以得到第一图像与投影变换后的第二图像的初始重叠区域，该初始重叠区域包括第一图像的第一子重叠区域，以及，投影变换后的第二图像的第二子重叠区域。
该步骤B具体可以通过步骤h至步骤k实现：
步骤h，获取第二图像的边界坐标；其中，该边界坐标用于指示第二图像的图像区域。
上述边界坐标可以理解为用于指示第二图像整体形状的边缘坐标，通过该边缘坐标可以得到第二图像所对应的图像区域。在实际实现时，当需要确定初始重叠区域时，通常需要先获取到该第二图像的边界坐标，该边界坐标的数量通常为多个。
步骤i，基于投影变换矩阵和第二图像的边界坐标，确定投影变换后的边界坐标。
当获取到第二图像的边界坐标，并基于投影变换矩阵，将第二图像投影至第一图像后，就可以得到投影变换后的第二图像的边界坐标；比如，如果第二图像的边界坐标为四个角点坐标，在基于投影变换矩阵完成投影变换后，就可以得到投影变换后的第二图像的四个角点坐标。
步骤j，基于投影变换后的边界坐标，确定投影变换后的第二图像。
在确定投影变换后的第二图像的边界坐标后，根据投影变换后的边界坐标所围成的图像区域确定投影变换后的第二图像。比如，仍以第二图像的边界坐标为四个角点坐标为例，则投影变换后的第二图像的四个角点坐标所围成的图像区域，即为投影变换后的第二图像。
步骤k，将投影变换后的第二图像与第一图像的重合图像区域，确定为初始重叠区域。
当确定上述投影变换后的第二图像后，将该投影变换后的第二图像与第一图像取交集，即可得到上述初始重叠区域。
步骤C，将第一子重叠区域和第二子重叠区域，输入至第二神经网络模型，通过第二神经网络模型确定第一子重叠区域中各个像素的第一像素值与第二子重叠区域中相同位置像素的第二像素值的映射关系。
在实际实现时，在确定初始重叠区域后，可以将第一图像对应的第一子重叠区域，以及第二图像对应的第二子重叠区域的像素值全部提取出来，分别计算出第一图像中，第一子重叠区域对应图像的像素值直方图分布，以及第二图像中，第二子重叠区域对应图像的像素值直方图分布，基于直方图匹配的方式将第二子重叠区域对应图像的像素值直方图分布变换为与第一子重叠区域对应图像的像素值直方图分布相匹配，具体可以通过构造像素值映射表的方式实现，该像素值映射表可以是LUT(Look-Up-Table，显示查找表)表等，并且，第一图像和第二图像所包括的颜色通道的数量相同，每个颜色通道通常都有其对应的像素值映射表；比如，第一图像和第二图像均包括R、G、G和B这四个颜色通道，将第一子重叠区域和第二子重叠区域，输入至预先训练好的第二神经网络模型，确定第一子重叠区域中各个像素的第一像素值与第二子重叠区域中相同位置像素的第二像素值的映射关系，可以以GT(Ground Truth，表示有监督学习的训练集的分类准确性，用于证明或者推翻某个假设)为0-65535的四个LUT表进行学习，最后通过该第二神经网络模型可以计算得到四个0-65535的LUT表，其中，四个0-65535的LUT表通常是不同的，LUT表的数量与颜色通道的数量相关联。而相关技术中，通常是基于CPU传统方式计算得到上述像素值映射表，由于通常需要逐像素进行计算，导致计算过程时间较长，效率较低，本实施例可以通过NN(Neural Network，神经网络)实现对相关技术中计算方式的加速。
上述像素值映射表可以理解为是一种像素灰度值的映射表，该映射表将实际采样到的像素灰度值经过一定的变换，如阈值、反转、二值化、对比度调整、线性变换等，变成了另外一个与之对应的灰度值，这样可以起到突出图像的有用信息，增强图像的光对比度的作用。
步骤D，获取第二神经网络模型输出的映射关系；
步骤E，针对各个颜色通道，基于映射关系，将第二图像中的各个像素点在颜色通道的像素值，与第一图像中的各个像素点在颜色通道的像素值进行匹配,以对第二图像进行光照补偿。
当获取到第二神经网络输出的上述映射关系后，比如，在得到每个颜色通道对应的像素值映射表后，可以将该像素值映射表作用于整个第二图像，即通过该像素值映射表可以将第二图像的每个颜色通道的值域映射到与第一图像类似的分布上，即，可以将第二图像中的各个像素点在该颜色通道的像素值与第一图像中的各个像素点在该颜色通道的像素值进行匹配。比如，仍以第一图像和第二图像均包括R、G、G和B这四个颜色通道，通过该第二神经网络模型计算得到四个颜色通道分别对应的四个0-65535的LUT表为例，则对于每个颜色通道来说，通过该颜色通道对应的LUT表，可以将第二图像的各个像素点在该颜色通道的像素值映射到与第一图像类似的分布上，从而实现对第二图像的光照补偿。
该步骤一具体还可以通过下述步骤H实现：
步骤H，将第一图像和第二图像，输入至第二神经网络模型中，通过第二神经网络基于第一图像对第二图像进行光照补偿，得到光照补偿后的第二图像。
在实际实现时，可以将第一图像和第二图像输入至预先训练好的第二神经网络模型中，通过该第二神经网络基于该第一图像对该第二图像进行光照补偿，可以以第一图像、基于直方图匹配进行光照补偿后的第二图像为GT来对第二神经网络模型训练进行监督，最后通过该第二神经网络模型计算得到光照补偿后的第二图像。
相关技术中，通常是基于CPU对待拼接图片中的每个像素计算相应的补偿系数，再对每个像素进行补偿，由于CPU对每个像素计算时需要一定的处理时间，导致对图片中所有像素进行补偿的时间较长，降低了光照补偿的处理效率。本实施例中的光照补偿方式，采用预先训练好的神经网络模型对第二图像进行光照补偿，不需要基于CPU对每个像素进行计算，节省了对图像中所有像素进行补偿的时间，提高了光照补偿的处理效率。
步骤S306，基于第一图像和光照补偿后的第二图像，确定初始拼接图像。
根据上述获取到的投影变换矩阵，将光照补偿后的第二图像投影至第一图像并进行初始拼接，得到第一图像与光照补偿后的第二图像所拼接的初始拼接图像。
步骤S308，利用第一神经网络模型对初始拼接图像中的目标重叠区域进行融合处理，得到目标重叠区域所对应的融合重叠区域。
步骤S310，基于融合重叠区域和初始拼接图像，确定第一图像和第二图像所对应的目标拼接图像。
上述图像拼接方法，获取待拼接的第一图像和第二图像，对第二图像进行光照补偿，基于第一图像和光照补偿后的第二图像，确定初始拼接图像，利用第一神经网络模型对初始拼接图像中的目标重叠区域进行融合处理，得到目标重叠区域所对应的融合重叠区域，基于融合重叠区域和初始拼接图像，确定第一图像和第二图像所对应的目标拼接图像。该方式中，在确定第一图像和第二图像的初始拼接图像后，采用第一神经网络模型对初始拼接图像中的目标重叠区域进行融合处理，得到对应的融合重叠区域，不需要基于CPU对初始拼接图像中的每个像素进行融合相关计算，节省了对初始拼接图像中所有像素进行融合计算的时间，提高了融合处理效率，进而提高了图像拼接的处理效率。
本公开实施例还提供另一种图像拼接方法，该方法在上述实施例方法的基础上实现；该方法重点描述利用第一神经网络模型对初始拼接图像中的目标重叠区域进行融合处理，得到目标重叠区域所对应的融合重叠区域的具体过程，该方法中，目标重叠区域包括：第一图像对应的第三子重叠区域和第二图像对应的第四子重叠区域；第一神经神经网络模型包括拼接模型和融合模型；该拼接模型可以通过多种卷积神经网络实现，如残差网络、VGG网络等；该融合模型也可以通过多种卷积神经网络实现，如残差网络、VGG网络等；该拼接模型和融合模型可以是第一神经网络模型中的子模块或子模型，也可以是两个单独的神经网络模型；如图4所示，该方法包括如下步骤：
步骤S402，获取待拼接的第一图像和第二图像。
步骤S404，确定第一图像和第二图像的初始拼接图像。
步骤S406，将第三子重叠区域和第四子重叠区域，输入至拼接模型，通过拼接模型对第三子重叠区域和第四子重叠区域之间的拼缝进行搜索，得到第三子重叠区域和第四子重叠区域对应的拼缝区域。
在实际实现时，当确定初始拼接图像后，根据该初始拼接图像的目标重叠区域，即可确定出该目标重叠区域所包含的第三子重叠区域和第四子重叠区域，将该第三子重叠区域和第四子重叠区域输入至预先训练好的拼接模型中，通过该拼接模型搜索第三子重叠区域和第四子重叠区域之间的拼缝，并输出搜索到的拼缝区域，该拼缝区域通常为拼缝的点集或拼接mask等，该拼缝区域也可以是缝合线，相关技术中，通常采用graphcut、vonorio等传统方式进行拼缝搜索，以计算第三子重叠区域和第四子重叠区域的最佳缝合线，通过搜索最佳缝合线，可以降低拼缝连接处的空间连续性，一般需要基于CPU对逐项数据进行计算，计算时间较长，其中，graphcut是一种能量优化算法，在图像处理领域可以用于前背景分割、立体视觉或抠图等；本实施例中，可以基于预先训练好的拼接模型蒸馏上述传统计算方式，不需要通过CPU计算，从而可以实现硬件加速，加快了处理速度，提高了处理效率。
步骤S408，基于拼缝区域，对第三子重叠区域和第四子重叠区域进行融合，得到初始融合重叠区域。
在实际实现时，在得到拼缝区域后，比如，该拼缝区域为未羽化的拼缝的点集，可以基于该未羽化的拼缝的点集，对第三子重叠区域和第四子重叠区域直接进行融合，得到初始融合重叠区域。
步骤S410，将初始融合重叠区域，第三子重叠区域，以及第四子重叠区域输入至融合模型，通过融合模型对初始融合重叠区域，第三子重叠区域，以及第四子重叠区域进行融合处理，得到融合重叠区域。
相关技术中，对重叠区域进行融合处理通常需要通过CPU对逐项数据进行计算融合，计算时间较长。而本方式在实际实现时，可以基于NN-blending，即神经网络的融合方式进行图像融合处理，可以提高处理效率，具体的，可以将上述得到的初始融合重叠区域，与两张未融合的原图，即第三子重叠区域和第四子重叠区域，一起送入融合模型中，通过融合模型对初始融合重叠区域，第三子重叠区域，以及第四子重叠区域进行融合处理，比如，经过image-to-image处理等，得到融合优化后的结果。
上述融合模型可以通过以下步骤五至步骤八训练得到：
步骤五，获取第一图片。
步骤六，对第一图片进行平移和/或旋转处理，得到第二图片。
在实际实现时，上述第一图片可以是任意一张图片；即上述第二图片可以是对第一图片经过小范围平移和/或旋转等处理后得到；在实际实现时，融合模型的训练过程可以以单张图片构造训练样本，该单张图片即为上述第一图片，将该单张图片经过小范围平移、旋转后得到一张变化后的图片，即为上述第二图片。
步骤七，对第一图片和第二图片进行融合处理，得到初始融合图片。
在得到上述第一图片和第二图片后，可以将这两个图片经过随机生成的拼缝进行融合，比如，可以通过Alpha-Blending方式进行融合，得到上述初始融合图片。
步骤八，基于第一图片、第二图片和初始融合图片，训练融合模型。
将上述第一图片、第二图片和初始融合图片共同作为训练样本，以对融合模型进行训练。具体的，可以将上述由第一图片、第二图片和初始融合图片组成的训练样本输入至初始融合模型中，以输出对初始融合图片优化后的融合图片，基于该融合图片和第一图片确定损失值，基于损失值更新初始融合模型的权重参数；继续执行获取第一图片的步骤，直至初始融合模型收敛，得到融合模型。其中，该损失值可以理解为上述输出的对初始融合图片优化后的融合图片与第一图片之间的差距；上述权重参数可以包括初始融合模型中的所有参数，如卷积核参数等，在对初始融合模型进行训练时，通常需要基于上述融 合图片和第一图片，更新初始融合模型中的所有参数，以对该初始融合模型进行训练。然后继续执行获取训练样本的步骤，直至初始融合模型收敛，或损失值收敛，最终得到训练完成的融合模型，在实际实现时，初始融合模型的输出可以以第一图片进行监督，监督的LOSS为L1距离，其中，拼缝区域的L1-loss可以被施加2x的权重。
步骤S412，基于融合重叠区域和初始拼接图像，确定第一图像和第二图像所对应的目标拼接图像。
上述图像拼接方法，获取待拼接的第一图像和第二图像。确定第一图像和第二图像的初始拼接图像。将第三子重叠区域和第四子重叠区域，输入至拼接模型，得到第三子重叠区域和第四子重叠区域对应的拼缝区域。基于拼缝区域，对第三子重叠区域和第四子重叠区域进行融合，得到初始融合重叠区域。将初始融合重叠区域，第三子重叠区域，以及第四子重叠区域输入至融合模型，得到融合重叠区域。基于融合重叠区域和初始拼接图像，确定第一图像和第二图像所对应的目标拼接图像。该方式中，第一神经网络模型包含了拼接模型和融合模型，通过拼接模型搜索出第一图像对应的第三子重叠区域和第二图像对应的第四子重叠区域对应的拼缝区域，进而得到初始融合重叠区域，再通过融合模型，得到融合重叠区域，该方式只需要通过第一神经网络模型就可以完成拼缝搜索和融合的处理过程，提高了融合处理效率，进而提高了图像拼接的处理效率。
本公开实施例还提供另一种图像拼接方法，该方法在上述实施例方法的基础上实现；该方法重点描述利用第一神经网络模型对初始拼接图像中的目标重叠区域进行融合处理，得到目标重叠区域所对应的融合重叠区域的具体过程，以及基于融合重叠区域和初始拼接图像，确定第一图像和第二图像所对应的目标拼接图像的具体过程，该方法中，目标重叠区域包括：第一图像对应的第三子重叠区域和第二图像对应的第四子重叠区域；第一神经神经网络模型包括拼接模型；如图5所示，该方法包括如下步骤：
步骤S502，获取待拼接的第一图像和第二图像。
在实际实现时，第一图像和第二图像通常为RAW图像，通常会先对该RAW图像进行格式转换，即会对第一图像和第二图像的颜色通道进行重排布，以便于后续光照补偿、拼缝搜索等处理，一般会将原始RAW域的bayer-pattern处理为RGGB(R：Red，红色；G：Green，绿色；G：Green，绿色；B：Blue，蓝色)排列方式，即第一图像和第二图像的颜色通道为RGGB排列方式。该方式中，第一图像和第二图像可以均为RAW图像，可以直接基于RAW图像进行拼接，不需要将第一图像和第二图像转化为RGB图像后再进行拼接，由于RAW图像与RGB图像相比，不会缺少图像细节，因此，RAW的图像拼接结果更有利于后期的图像检测、分割等处理。
步骤S504，确定第一图像和第二图像的初始拼接图像。
上述RAW图像通常是CMOS(Complementary Metal-Oxide-Semiconductor，互补金属氧化物半导体)或者CCD(Charge Coupled Device，电荷耦合器件)图像感应器将捕捉到的光源信号转化为数字信号的原始数据，是未经处理的，与RGB图像相比，RAW图像包含完整的图像细节，在后期处理，如加减曝光、调整高光/阴影、增减对比度、调整色阶和曲线等方面有更大的优势。在实际实现时，当需要对图像进行拼接处理时，通常需要先获取到的第一图像和第二图像均为RAW图像。
步骤S506，将第三子重叠区域和第四子重叠区域，输入至拼接模型，得到第三子重叠区域和第四子重叠区域对应的拼缝区域。
步骤S508，对拼缝区域进行羽化处理，得到羽化后的重叠区域，将羽化后的重叠区域确定为融合重 叠区域。
上述羽化处理通常是将像素选区的边缘变得模糊，使所选区域与周围的像素混合，即将像素选区内外衔接部分虚化，起到渐变的作用从而达到自然衔接的效果，羽化值越大，虚化范围越宽，也就是说颜色递变的柔和；羽化值越小，虚化范围越窄，可根据实际情况进行调，通常把羽化值设置小一点，反复羽化是羽化处理的一个技巧。在实际实现时，在得到拼缝区域后，可以基于blending操作，即融合操作得到融合重叠区域，具体的，可以通过Alpha-Blending融合的方式，对拼缝搜索得到的拼缝区域构造羽化效果，羽化的像素值可选16-22pixel，最终得到融合重叠区域。具体可参考相关技术中，采用Alpha-Blending方式融合的过程，在此不再赘述。
步骤S510，使用融合重叠区域替换初始拼接图像中的目标重叠区域，得到第一图像和第二图像所对应的目标拼接图像。
在实际实现时，由于第一图像和第二图像通常均为RAW图像，即都是未经过有损压缩处理的图像，因此，上述目标拼接图像通常也是RAW图像，即，在使用融合重叠区域替换初始拼接图像中的目标重叠区域后，就可以得到拼接后的RAW图，即上述目标拼接图像。
通常在得到上述目标拼接图像后，可以对目标拼接图像进行检测、分割或识别等分析处理，得到相应的分析结果；由于该图像拼接结果为RAW图像，因此，该方式可以实现对RAW图像的图像智能分析。还可以采用Demasaic等ISP(Image Signal Processing，图像信号处理)的核心算法等方式对目标拼接图像进行处理，得到对应的RGB图像拼接结果，其中，ISP主要用于对前端图像像传感器输出的信号做后期处理，主要功能有线性纠正、噪声去除、坏点去除、内插、白平衡、自动曝光控制等，ISP包括Demasaic等多种核心算法，ISP经过相关算法处理，可以输出RGB域的图像，以得到便于可视化的拼接结果。
上述图像拼接方法，获取待拼接的第一图像和第二图像。将第三子重叠区域和第四子重叠区域，输入至拼接模型，得到第三子重叠区域和第四子重叠区域对应的拼缝区域。对拼缝区域进行羽化处理，得到羽化后的重叠区域，将羽化后的重叠区域确定为融合重叠区域。使用融合重叠区域替换初始拼接图像中的目标重叠区域，得到第一图像和第二图像所对应的目标拼接图像。该方式中，第一神经网络模型包含了拼接模型，通过拼接模型搜索出第一图像对应的第三子重叠区域和第二图像对应的第四子重叠区域对应的拼缝区域，再通过羽化处理方式，得到融合重叠区域，该方式简化了融合处理过程，提高了融合处理效率，进而提高了图像拼接的处理效率。
本公开实施例还提供了一种光照补偿方法，该方法包括如下步骤：
步骤602，获取待拼接的第一图像和第二图像。
步骤604，基于第二神经网络模型和第一图像，对第二图像进行光照补偿。
该步骤604具体可以通过下述步骤十一至步骤十五来实现：
步骤十一，基于第一图像和第二图像，确定投影变换矩阵。
该步骤十一具体可以通过下述步骤M至步骤O实现：
步骤M，提取第一图像中的至少一个第一特征点，以及第二图像中的至少一个第二特征点。
步骤N，基于至少一个第一特征点和至少一个第二特征点，确定至少一个匹配特征点对。
步骤O，基于至少一个匹配特征点对，确定投影变换矩阵。
步骤十二，基于投影变换矩阵，确定第一图像和第二图像的初始重叠区域；其中，初始重叠区域包括：第一图像对应的第一子重叠区域以及第二图像对应的第二子重叠区域。
该步骤十二具体可以通过下述步骤P至步骤S实现：
步骤P，获取第二图像的边界坐标；其中，边界坐标用于指示第二图像的图像区域。
步骤Q，基于投影变换矩阵和第二图像的边界坐标，确定投影变换后的边界坐标。
步骤R，基于投影变换后的边界坐标，确定投影变换后的第二图像。
步骤S，将投影变换后的第二图像与第一图像的重合图像区域，确定为初始重叠区域。
步骤十三，将第一子重叠区域和第二子重叠区域，输入至第二神经网络模型，通过第二神经网络模型确定第一子重叠区域中各个像素的第一像素值与第二子重叠区域中相同位置像素的第二像素值的映射关系。
步骤十四，获取第二神经网络模型输出的映射关系。
步骤十五，针对各个颜色通道，基于映射关系，将第二图像中的各个像素点在颜色通道的像素值，与第一图像中的各个像素点在颜色通道的像素值进行匹配,以对第二图像进行光照补偿。
该步骤604具体还可以通过下述步骤二十来实现：
步骤二十，将第一图像和第二图像，输入至第二神经网络模型中，通过第二神经网络基于第一图像对第二图像进行光照补偿，得到光照补偿后的第二图像。本实施例中，各个步骤的具体实施方式可参考前述实施例中的相关描述，在此不再赘述。
上述光照补偿方法，在获取到待拼接的第一图像和第二图像后，基于第二神经网络模型和第一图像，对第二图像进行光照补偿。该方式中，采用预先训练好的神经网络模型对第二图像进行光照补偿，不需要基于CPU对每个像素进行计算，节省了对图像中所有像素进行补偿的时间，提高了光照补偿的处理效率。
为进一步理解上述实施例，下面提供如图6所示的一种RAW域图像拼接方法的流程图，为方便说明，以两幅图像为例进行介绍，两幅图像分别以tar图像(对应上述第一图像)和src图像(对应上述第二图像)来表示，多幅图像也可以采用该方式进行拼接，首先对原始RAW域的bayer-pattern(拜耳阵列)图像处理为rggb排列方式，即对bayer-tar进行格式转换，转换为rggb-tar图像，对bayer-src进行格式转换，转换为rggb-src图像。
下面对rggb-tar图像和rggb-src图像进行空间配准，具体空间配准的流程可以是，先对两幅图像进行特征点提取，可以采用传统的SIFT或基于神经网络的SuperPoint等方式提取特征点，再对特征点进行匹配，得到匹配特征点对，基于匹配特征点对的特征点匹配信息，计算投影变换矩阵，基于该投影变换矩阵以及rggb-src图像的边界坐标，计算得到rggb-src的投影后的区域信息，与rggb-tar图像取交即可得到重叠区域(对应上述初始重叠区域)。
接着可以基于预先训练好的神经网络模型，对rggb-src图像进行光照补偿，通过光照补偿可以优化不同相机图像的色彩一致性，可以采用以下方式，其中一种方式是将上述rggb-tar图像和rggb-src图像的重叠区域的两个小图输入至神经网络模型，以GT为0-65535的LUT表来进行学习，最后通过该神经网络模型可以计算得到四个0-65535的LUT表，基于该LUT表对rggb-src图像进行光照补偿；另一种方式是将rggb-tar图像和rggb-src图像输入至神经网络模型，以GT为原始rggb-tar图像、基于直方图匹 配进行光照补偿后的rggb-src图像进行监督，最后通过该神经网络模型可以计算得到光照补偿后的rggb-src图像。
在得到光照补偿后的rggb-src图像后，可以对rggb-tar图像和光照补偿后的rggb-src图像进行投影变化处理，具体的，基于投影变换矩阵，将光照补偿后的rggb-src图像投影至rggb-tar图像上，得到初始拼接图像；基于光照补偿后的rggb-src图像，重新提取新的重叠区域，基于rggb-tar图像和rggb-src图像的新的重叠区域的两个小图进行拼缝搜索和blending(融合)，具体的，可以将新的重叠区域的两个小图输入至预先训练好的拼接模型，通过该拼接模型，输出拼接mask或拼缝的点集(对应上述拼缝区域)，然后通过Alpha-blending融合的方式，或者基于预先训练好的融合模型进行融合的方式，得到融合的重叠区域(对应上述融合重叠区域)。最后进行替换重叠区处理，将初始拼接图像中的重叠区域替换为该融合的重叠区域，得到RAW域拼接结果；可选的，在得到该RAW域拼接结果后，可以对该RAW域拼接结果进行RAW域图像智能分析，还可以通过Demasaic等ISP步骤处理得到RGB可视化结果。
上述RAW域图像拼接方法中的光照补偿、拼缝搜索和融合等过程，可以基于神经网络实现，从而可以实现硬件加速，加快处理速度，提高处理效率，该方式为RAW域图像智能分析提供了一种大视场角的解决方案，对RAW域图像也可以处理好拼缝衔接、光照对齐等步骤，以尽量高质量融合待拼接的图像，通过对RAW域图像进行拼接可以获得大视场角的原始RAW域图片，便于大视场角下的暗光RAW域检测、识别等任务。
本公开实施例还提供了一种图像拼接装置的结构示意图，如图7所示，该装置包括：获取模块70，可以被配置成用于获取待拼接的第一图像和第二图像；第一确定模块71，可以被配置成用于确定第一图像和第二图像的初始拼接图像；融合模块72，可以被配置成用于利用第一神经网络模型对初始拼接图像中的目标重叠区域进行融合处理，得到目标重叠区域所对应的融合重叠区域；第二确定模块73，可以被配置成用于基于融合重叠区域和初始拼接图像，确定第一图像和第二图像所对应的目标拼接图像。
上述图像拼接装置，首先获取待拼接的第一图像和第二图像；确定第一图像和第二图像的初始拼接图像；然后利用第一神经网络模型对初始拼接图像中的目标重叠区域进行融合处理，得到目标重叠区域所对应的融合重叠区域；最后基于融合重叠区域和初始拼接图像，确定第一图像和第二图像所对应的目标拼接图像。该装置中，在确定第一图像和第二图像的初始拼接图像后，采用第一神经网络模型对初始拼接图像中的目标重叠区域进行融合处理，得到对应的融合重叠区域，不需要基于CPU对初始拼接图像中的每个像素进行融合相关计算，节省了对初始拼接图像中所有像素进行融合计算的时间，提高了融合处理效率，进而提高了图像拼接的处理效率。
可选地，该装置还用于：对第二图像进行光照补偿；相应的，第一确定模块还可以被配置成用于：基于第一图像和光照补偿后的第二图像，确定初始拼接图像。
可选地，第一确定模块还可以被配置成用于：基于第二神经网络模型和第一图像，对第二图像进行光照补偿。
可选地，第一确定模块还可以被配置成用于：基于第一图像和第二图像，确定投影变换矩阵；基于投影变换矩阵，确定第一图像和第二图像的初始重叠区域；其中，初始重叠区域包括：第一图像对应的第一子重叠区域以及第二图像对应的第二子重叠区域；将第一子重叠区域和第二子重叠区域，输入至第二神经网络模型，通过第二神经网络模型确定第一子重叠区域中各个像素的第一像素值与第二子重叠区 域中相同位置像素的第二像素值的映射关系；获取第二神经网络模型输出的映射关系；针对各个颜色通道，基于映射关系，将第二图像中的各个像素点在颜色通道的像素值，与第一图像中的各个像素点在颜色通道的像素值进行匹配,以对第二图像进行光照补偿。
可选地，第一确定模块还可以被配置成用于：获取第二图像的边界坐标；其中，边界坐标用于指示第二图像的图像区域；基于投影变换矩阵和第二图像的边界坐标，确定投影变换后的边界坐标；基于投影变换后的边界坐标，确定投影变换后的第二图像；将投影变换后的第二图像与第一图像的重合图像区域，确定为初始重叠区域。
可选地，第一确定模块还可以被配置成用于：提取第一图像中的至少一个第一特征点，以及第二图像中的至少一个第二特征点；基于至少一个第一特征点和至少一个第二特征点，确定至少一个匹配特征点对；基于至少一个匹配特征点对，确定投影变换矩阵。
可选地，第一确定模块还可以被配置成用于：将第一图像和第二图像，输入至第二神经网络模型中，通过第二神经网络基于第一图像对第二图像进行光照补偿，得到光照补偿后的第二图像。
可选地，目标重叠区域包括：第一图像对应的第三子重叠区域和第二图像对应的第四子重叠区域；第一神经神经网络模型包括拼接模型和融合模型；融合模块还可以被配置成用于：将第三子重叠区域和第四子重叠区域，输入至拼接模型，通过拼接模型对第三子重叠区域和第四子重叠区域之间的拼缝进行搜索，得到第三子重叠区域和第四子重叠区域对应的拼缝区域；基于拼缝区域，对第三子重叠区域和第四子重叠区域进行融合，得到初始融合重叠区域；将初始融合重叠区域，第三子重叠区域，以及第四子重叠区域输入至融合模型，通过融合模型对初始融合重叠区域，第三子重叠区域，以及第四子重叠区域进行融合处理，得到融合重叠区域。
可选地，目标重叠区域包括：第一图像对应的第三子重叠区域和第二图像对应的第四子重叠区域；第一神经神经网络模型包括拼接模型；融合模块还可以被配置成用于：将第三子重叠区域和第四子重叠区域，输入至拼接模型，得到第三子重叠区域和第四子重叠区域对应的拼缝区域；对拼缝区域进行羽化处理，得到羽化后的重叠区域，将羽化后的重叠区域确定为融合重叠区域。
可选地，融合模块还可以被配置成用于：获取第一图片；对第一图片进行平移和/或旋转处理，得到第二图片；对第一图片和第二图片进行融合处理，得到初始融合图片；基于第一图片、第二图片和初始融合图片，训练融合模型。
可选地，第二确定模块还可以被配置成用于：使用融合重叠区域替换初始拼接图像中的目标重叠区域，得到第一图像和第二图像所对应的目标拼接图像。
可选地，第一图像和第二图像的颜色通道为RGGB排列方式。
本公开实施例所提供的图像拼接装置，其实现原理及产生的技术效果和前述图像拼接方法实施例相同，为简要描述，图像拼接装置实施例部分未提及之处，可参考前述图像拼接方法实施例中相应内容。
本公开实施例还提供了一种电子设备，包括处理设备和存储装置，存储装置存储有计算机程序，计算机程序在被处理设备运行时执行如上述任一项的图像拼接方法。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的电子设备的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
本公开实施例还提供了一种计算机可读存储介质，计算机可读存储介质上存储有计算机程序，计算 机程序被处理设备运行时执行如上述图像拼接方法的步骤。
本公开实施例还提供了一种计算机程序产品，包括存储有程序代码的计算机可读存储介质，所述程序代码包括的指令能够执行如上所述的图像拼接方法的步骤。
本公开实施例所提供的图像拼接方法、装置和电子设备的计算机程序产品，包括存储了程序代码的计算机可读存储介质，所述程序代码包括的指令可用于执行前面方法实施例中所述的方法，具体实现可参见方法实施例，在此不再赘述。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的设备和/或装置的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
另外，在本公开实施例的描述中，除非另有明确的规定和限定，术语“安装”、“相连”、“连接”应做广义理解，例如，可以是固定连接，也可以是可拆卸连接，或一体地连接；可以是机械连接，也可以是电连接；可以是直接相连，也可以通过中间媒介间接相连，可以是两个元件内部的连通。对于本领域的普通技术人员而言，可以具体情况理解上述术语在本公开中的具体含义。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本公开的技术方案本质上或者说对相关技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本公开各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
在本公开的描述中，需要说明的是，术语“中心”、“上”、“下”、“左”、“右”、“竖直”、“水平”、“内”、“外”等指示的方位或位置关系为基于附图所示的方位或位置关系，仅是为了便于描述本公开和简化描述，而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作，因此不能理解为对本公开的限制。此外，术语“第一”、“第二”、“第三”仅用于描述目的，而不能理解为指示或暗示相对重要性。
最后应说明的是：以上各实施例仅用以说明本公开的技术方案，而非对其限制；尽管参照前述各实施例对本公开进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本公开各实施例技术方案的范围。
工业实用性
本申请提供了一种图像拼接方法和电子设备，获取待拼接的第一图像和第二图像；确定第一图像和第二图像的初始拼接图像；利用第一神经网络模型对初始拼接图像中的目标重叠区域进行融合处理，得到目标重叠区域所对应的融合重叠区域；基于融合重叠区域和初始拼接图像，确定第一图像和第二图像所对应的目标拼接图像。该方式中，在确定第一图像和第二图像的初始拼接图像后，采用第一神经网络模型对初始拼接图像中的目标重叠区域进行融合处理，得到对应的融合重叠区域，不需要基于CPU对初始拼接图像中的每个像素进行融合相关计算，节省了对初始拼接图像中所有像素进行融合计算的时间，提高了融合处理效率，进而提高了图像拼接的处理效率。
此外，可以理解的是，本申请的图像拼接方法和电子设备是可以重现的，并且可以用在多种工业应用中。例如，本申请的图像拼接方法和电子设备可以用于图像处理技术领域。

Claims (16)

  1. 一种图像拼接方法，其特征在于，所述方法包括：
    获取待拼接的第一图像和第二图像；
    确定所述第一图像和所述第二图像的初始拼接图像；
    利用第一神经网络模型对所述初始拼接图像中的目标重叠区域进行融合处理，得到所述目标重叠区域所对应的融合重叠区域；
    基于所述融合重叠区域和所述初始拼接图像，确定所述第一图像和所述第二图像所对应的目标拼接图像。
  2. 根据权利要求1所述的方法，其特征在于，所述确定所述第一图像和所述第二图像的初始拼接图像之前，所述方法还包括：
    对所述第二图像进行光照补偿；
    相应的，所述确定所述第一图像和所述第二图像的初始拼接图像，包括：
    基于所述第一图像和光照补偿后的第二图像，确定所述初始拼接图像。
  3. 根据权利要求2所述的方法，其特征在于，所述对所述第二图像进行光照补偿的步骤包括：
    基于第二神经网络模型和所述第一图像，对所述第二图像进行光照补偿。
  4. 根据权利要求3所述的方法，其特征在于，所述基于第二神经网络模型和所述第一图像，对所述第二图像进行光照补偿，包括：
    基于所述第一图像和所述第二图像，确定投影变换矩阵；
    基于所述投影变换矩阵，确定所述第一图像和所述第二图像的初始重叠区域；其中，所述初始重叠区域包括：所述第一图像对应的第一子重叠区域以及所述第二图像对应的第二子重叠区域；
    将所述第一子重叠区域和所述第二子重叠区域，输入至所述第二神经网络模型，通过所述第二神经网络模型确定所述第一子重叠区域中各个像素的第一像素值与所述第二子重叠区域中相同位置像素的第二像素值的映射关系；
    获取所述第二神经网络模型输出的所述映射关系；
    针对各个颜色通道，基于所述映射关系，将所述第二图像中的各个像素点在所述颜色通道的像素值，与所述第一图像中的各个像素点在所述颜色通道的像素值进行匹配,以对所述第二图像进行光照补偿。
  5. 根据权利要求4所述的方法，其特征在于，所述基于所述投影变换矩阵，确定所述第一图像和所述第二图像的初始重叠区域的步骤包括：
    获取所述第二图像的边界坐标；其中，所述边界坐标用于指示所述第二图像的图像区域；
    基于所述投影变换矩阵和所述第二图像的边界坐标，确定投影变换后的边界坐标；
    基于所述投影变换后的边界坐标，确定投影变换后的第二图像；
    将所述投影变换后的第二图像与所述第一图像的重合图像区域，确定为所述初始重叠区域。
  6. 根据权利要求4或5所述的方法，其特征在于，所述基于所述第一图像和所述第二图像，确定投影变换矩阵的步骤包括：
    提取所述第一图像中的至少一个第一特征点，以及所述第二图像中的至少一个第二特征点；
    基于所述至少一个第一特征点和所述至少一个第二特征点，确定至少一个匹配特征点对；
    基于所述至少一个匹配特征点对，确定投影变换矩阵。
  7. 根据权利要求3至6中任一项所述的方法，其特征在于，所述基于第二神经网络模型和所述第一图像，对所述第二图像进行光照补偿的步骤包括：
    将所述第一图像和所述第二图像，输入至所述第二神经网络模型中，通过所述第二神经网络基于所述第一图像对所述第二图像进行光照补偿，得到光照补偿后的所述第二图像。
  8. 根据权利要求1至7中任一项所述的方法，其特征在于，所述目标重叠区域包括：所述第一图像对应的第三子重叠区域和所述第二图像对应的第四子重叠区域；所述第一神经神经网络模型包括拼接模型和融合模型；所述利用第一神经网络模型对所述初始拼接图像中的目标重叠区域进行融合处理，得到所述目标重叠区域所对应的融合重叠区域的步骤包括：
    将所述第三子重叠区域和所述第四子重叠区域，输入至所述拼接模型，通过所述拼接模型对所述第三子重叠区域和所述第四子重叠区域之间的拼缝进行搜索，得到所述第三子重叠区域和所述第四子重叠区域对应的拼缝区域；
    基于所述拼缝区域，对所述第三子重叠区域和所述第四子重叠区域进行融合，得到初始融合重叠区域；
    将所述初始融合重叠区域，所述第三子重叠区域，以及所述第四子重叠区域输入至所述融合模型，通过所述融合模型对初始融合重叠区域，所述第三子重叠区域，以及所述第四子重叠区域进行融合处理，得到所述融合重叠区域。
  9. 根据权利要求1至7中任一项所述的方法，其特征在于，所述目标重叠区域包括：所述第一图像对应的第三子重叠区域和所述第二图像对应的第四子重叠区域；所述第一神经神经网络模型包括拼接模型；所述利用第一神经网络模型对所述初始拼接图像中的目标重叠区域进行融合处理，得到所述目标重叠区域所对应的融合重叠区域的步骤包括：
    将所述第三子重叠区域和所述第四子重叠区域，输入至所述拼接模型，得到所述第三子重叠区域和所述第四子重叠区域对应的拼缝区域；
    对所述拼缝区域进行羽化处理，得到羽化后的重叠区域，将所述羽化后的重叠区域确定为所述融合重叠区域。
  10. 根据权利要求8所述的方法，其特征在于，所述融合模型通过以下方式训练得到：
    获取第一图片；
    对所述第一图片进行平移和/或旋转处理，得到第二图片；
    对所述第一图片和所述第二图片进行融合处理，得到初始融合图片；
    基于所述第一图片、所述第二图片和所述初始融合图片，训练所述融合模型。
  11. 根据权利要求1至10中任一项所述的方法，其特征在于，所述基于所述融合重叠区域和所述初始拼接图像，确定所述第一图像和所述第二图像所对应的目标拼接图像的步骤包括：
    使用所述融合重叠区域替换所述初始拼接图像中的目标重叠区域，得到所述第一图像和所述第二图像所对应的所述目标拼接图像。
  12. 根据权利要求1至11中任一项所述的方法，其特征在于，所述第一图像和所述第二图像的颜色通道的排列方式为RGGB排列方式。
  13. 根据权利要求1至11中任一项所述的方法，其特征在于，所述第一图像和所述第二图像为RAW域图像，所述确定第一图像和第二图像的初始拼接图像，包括：
    对所述第一图像进行格式转换，得到RGGB排列方式的第一图像，以及对所述第二图像进行格式转换，得到RGGB排列方式的第二图像；
    基于RGGB排列方式的第一图像和RGGB排列方式的第二图像确定所述初始拼接图像。
  14. 一种电子设备，其特征在于，包括处理设备和存储装置，所述存储装置存储有计算机程序，所述计算机程序在被所述处理设备运行时执行如权利要求1至13中任一项所述的图像拼接方法。
  15. 一种机器可读存储介质，其特征在于，所述机器可读存储介质存储有计算机程序，所述计算机程序被处理设备运行时执行如权利要求1至13中任一项所述的图像拼接方法。
  16. 一种计算机程序产品，包括存储有程序代码的计算机可读存储介质，所述程序代码包括的指令能够执行如权利要求1至13中任一项所述的图像拼接方法。
PCT/CN2022/102342 2021-08-26 2022-06-29 图像拼接方法和电子设备 WO2023024697A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110990427.0A CN113902657A (zh) 2021-08-26 2021-08-26 图像拼接方法、装置和电子设备
CN202110990427.0 2021-08-26

Publications (1)

Publication Number Publication Date
WO2023024697A1 true WO2023024697A1 (zh) 2023-03-02

Family

ID=79188035

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/102342 WO2023024697A1 (zh) 2021-08-26 2022-06-29 图像拼接方法和电子设备

Country Status (2)

Country Link
CN (1) CN113902657A (zh)
WO (1) WO2023024697A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116167921A (zh) * 2023-04-21 2023-05-26 深圳市南天门网络信息有限公司 一种飞行太空舱全景图像拼接方法及系统
CN117372252A (zh) * 2023-12-06 2024-01-09 国仪量子技术(合肥)股份有限公司 图像拼接方法、装置及存储介质、电子设备
CN117575976A (zh) * 2024-01-12 2024-02-20 腾讯科技（深圳）有限公司 图像阴影的处理方法、装置、设备及存储介质
CN117680977A (zh) * 2024-02-04 2024-03-12 季华实验室 机器人上料拼接对位方法、装置、设备及介质

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113902657A (zh) * 2021-08-26 2022-01-07 北京旷视科技有限公司 图像拼接方法、装置和电子设备
CN116055659B (zh) * 2023-01-10 2024-02-20 如你所视(北京)科技有限公司 原始图像的处理方法、装置、电子设备和存储介质
CN117544862B (zh) * 2024-01-09 2024-03-29 北京大学 一种基于图像矩并行处理的图像拼接方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101882308A (zh) * 2010-07-02 2010-11-10 上海交通大学 提高图像拼接精度和稳定性的方法
US20180061006A1 (en) * 2016-08-26 2018-03-01 Multimedia Image Solution Limited Method for ensuring perfect stitching of a subject's images in a real-site image stitching operation
CN109523491A (zh) * 2018-12-13 2019-03-26 深圳市路畅智能科技有限公司 用于环视辅助泊车的环视光照均一化方法和装置
CN110473143A (zh) * 2019-07-23 2019-11-19 平安科技（深圳）有限公司 一种三维mra医学图像拼接方法及装置、电子设备
CN112862685A (zh) * 2021-02-09 2021-05-28 北京迈格威科技有限公司 图像拼接的处理方法、装置和电子系统
CN113902657A (zh) * 2021-08-26 2022-01-07 北京旷视科技有限公司 图像拼接方法、装置和电子设备

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101882308A (zh) * 2010-07-02 2010-11-10 上海交通大学 提高图像拼接精度和稳定性的方法
US20180061006A1 (en) * 2016-08-26 2018-03-01 Multimedia Image Solution Limited Method for ensuring perfect stitching of a subject's images in a real-site image stitching operation
CN109523491A (zh) * 2018-12-13 2019-03-26 深圳市路畅智能科技有限公司 用于环视辅助泊车的环视光照均一化方法和装置
CN110473143A (zh) * 2019-07-23 2019-11-19 平安科技（深圳）有限公司 一种三维mra医学图像拼接方法及装置、电子设备
CN112862685A (zh) * 2021-02-09 2021-05-28 北京迈格威科技有限公司 图像拼接的处理方法、装置和电子系统
CN113902657A (zh) * 2021-08-26 2022-01-07 北京旷视科技有限公司 图像拼接方法、装置和电子设备

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116167921A (zh) * 2023-04-21 2023-05-26 深圳市南天门网络信息有限公司 一种飞行太空舱全景图像拼接方法及系统
CN117372252A (zh) * 2023-12-06 2024-01-09 国仪量子技术(合肥)股份有限公司 图像拼接方法、装置及存储介质、电子设备
CN117372252B (zh) * 2023-12-06 2024-02-23 国仪量子技术(合肥)股份有限公司 图像拼接方法、装置及存储介质、电子设备
CN117575976A (zh) * 2024-01-12 2024-02-20 腾讯科技（深圳）有限公司 图像阴影的处理方法、装置、设备及存储介质
CN117575976B (zh) * 2024-01-12 2024-04-19 腾讯科技（深圳）有限公司 图像阴影的处理方法、装置、设备及存储介质
CN117680977A (zh) * 2024-02-04 2024-03-12 季华实验室 机器人上料拼接对位方法、装置、设备及介质
CN117680977B (zh) * 2024-02-04 2024-04-16 季华实验室 机器人上料拼接对位方法、装置、设备及介质

Also Published As

Publication number Publication date
CN113902657A (zh) 2022-01-07

Similar Documents

Publication Publication Date Title
WO2023024697A1 (zh) 图像拼接方法和电子设备
WO2020192483A1 (zh) 图像显示方法和设备
CN107301620B (zh) 基于相机阵列的全景成像方法
Liao et al. DR-GAN: Automatic radial distortion rectification using conditional GAN in real-time
KR101679290B1 (ko) 영상 처리 방법 및 장치
WO2023011013A1 (zh) 视频图像的拼缝搜索方法、视频图像的拼接方法和装置
CN108875900B (zh) 视频图像处理方法和装置、神经网络训练方法、存储介质
CN108171735B (zh) 基于深度学习的十亿像素视频对齐方法及系统
CN108111768A (zh) 控制对焦的方法、装置、电子设备及计算机可读存储介质
CN113630549A (zh) 变焦控制方法、装置、电子设备和计算机可读存储介质
CN111866523A (zh) 全景视频合成方法、装置、电子设备和计算机存储介质
Shen et al. Distortion-tolerant monocular depth estimation on omnidirectional images using dual-cubemap
Kim et al. Unpaired cross-spectral pedestrian detection via adversarial feature learning
CN111836058B (zh) 用于实时视频播放方法、装置、设备以及存储介质
Song et al. Real-scene reflection removal with raw-rgb image pairs
CN113298177B (zh) 夜间图像着色方法、装置、介质和设备
WO2022083118A1 (zh) 一种数据处理方法及相关设备
Fu et al. Image stitching techniques applied to plane or 3-D models: a review
KR102261544B1 (ko) 스트리밍 서버 및 이를 이용한 다시점 동영상에서의 객체 처리 방법
CN116579923A (zh) 图像拼接方法、设备及存储介质
CN116261043A (zh) 对焦距离确定方法、装置、电子设备和可读存储介质
CN113592777B (zh) 双摄拍照的图像融合方法、装置和电子系统
Huang et al. Learning image-adaptive lookup tables with spatial awareness for image harmonization
Guo et al. Low-light color imaging via cross-camera synthesis
WO2022193132A1 (zh) 图像检测方法、装置和电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22860044

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE