CN117437122A

CN117437122A - Method and system for splicing panoramic images of container

Info

Publication number: CN117437122A
Application number: CN202311767000.XA
Authority: CN
Inventors: 黄深广; 黄昂涛; 夏侃; 乔耿嘉; 梅浪奇; 吴高德; 王君宇; 鲍朝前; 张金硕; 贺冰之
Original assignee: NINGBO PORT INFORMATION COMMUNICATION CO Ltd
Current assignee: NINGBO PORT INFORMATION COMMUNICATION CO Ltd
Priority date: 2023-12-21
Filing date: 2023-12-21
Publication date: 2024-01-23
Anticipated expiration: 2043-12-21
Also published as: CN117437122B

Abstract

The invention discloses a container panoramic image stitching method and system, which relate to the technical field of image processing and comprise the following steps: acquiring a box surface correction image sequence; splicing the plurality of corrected images based on pixel displacement values when adjacent corrected images are spliced to obtain a one-time spliced panoramic image; judging whether the aspect ratio of the once spliced panoramic image is in the aspect ratio interval of the container panoramic image or not; if yes, taking the once spliced panoramic image as a finally spliced container panoramic image; if not, splicing the plurality of corrected images based on the corrected pixel displacement values when adjacent corrected images in the box surface corrected image sequence are spliced to obtain a secondary spliced panoramic image, and taking the secondary spliced panoramic image as a final spliced container panoramic image. The method solves the technical problem that when the gate environment has light reflection and shadow, a better container panoramic image splicing result is difficult to obtain, and achieves the technical effect of improving the container panoramic image splicing result under various adverse conditions such as light reflection and shadow.

Description

Method and system for splicing panoramic images of container

Technical Field

The invention relates to the technical field of image processing, in particular to a container panoramic image stitching method and system.

Background

For the safety of goods and the safety consideration of container trucks in the driving process, the container needs to be damaged and detected, and because the container is generally longer, and the camera shooting acquisition equipment of the gate is too close, the whole of the container body is difficult to acquire at one time, so when the container body passes through the gate by using the collection card, the complete panoramic image of the container surface is spliced by the partial images of the container surface acquired by the camera, and the damage and detection is carried out, so that the accurate splicing of each panoramic image of the container is one of the foundations of the established mature automatic wharf.

The container is the main body of each header and has three sides, left, right and top, generally exposed, and is typically a sheet of embossed, metallic luster. The common container face splicing algorithm generally calculates similar block matching results of spliced images through a gray level image registration method to obtain pixel displacement values during splicing, has a good effect on splicing container faces with large image changes, but is difficult to calculate correct similar block matching results for repeated concave-convex metal plates of a container body, particularly when light reflection and shadow exist in a gate environment, is difficult to obtain good splicing results, cannot meet requirements on instantaneity and stability, and is difficult to popularize and apply to a task of container splicing.

In view of the above problems, no effective solution has been proposed at present.

Disclosure of Invention

The invention provides a container panoramic image splicing method and system, which at least solve the technical problem that in the related art, when a gate environment has light reflection and shadow, a better container panoramic image splicing result is difficult to obtain.

According to one aspect of the present invention, there is provided a stitching method of container panoramic images, comprising: acquiring a box surface image sequence of a container, wherein the box surface image sequence comprises a plurality of box surface images, and the box surface images are sequenced in an increasing way according to time; carrying out distortion correction on the box face images in the box face image sequence to obtain a box face correction image sequence of the container, wherein the box face correction image sequence comprises a plurality of correction images with the same size, and each correction image corresponds to one box face image; acquiring pixel displacement values when adjacent corrected images in the box surface corrected image sequence are spliced, and splicing a plurality of corrected images into a container panoramic image based on the pixel displacement values to obtain a one-time spliced panoramic image; acquiring an aspect ratio section of the primary spliced panoramic image and an aspect ratio section of the container panoramic image, and judging whether the aspect ratio section of the primary spliced panoramic image is in the aspect ratio section of the container panoramic image; if the aspect ratio of the primary spliced panoramic image is in the aspect ratio interval of the container panoramic image, determining the primary spliced panoramic image as a final spliced container panoramic image; and if the aspect ratio of the primary stitching panoramic image is not in the aspect ratio interval of the container panoramic image, acquiring a correction pixel displacement value when adjacent correction images in the box surface correction image sequence are stitched, stitching a plurality of correction images into the container panoramic image based on the correction pixel displacement value, obtaining a secondary stitching panoramic image, and determining the secondary stitching panoramic image as a final stitched container panoramic image.

Optionally, distortion correction is performed on a box surface image in the box surface image sequence to obtain a box surface correction image sequence of the container, including: obtaining distortion correction parameters of fixed camera points; and correcting radial distortion and tangential distortion of the box surface image in the box surface image sequence based on the distortion correction parameters to obtain the box surface correction image sequence.

Optionally, acquiring a pixel displacement value when adjacent corrected images in the box surface corrected image sequence are spliced includes: extracting characteristic points and matching the characteristic points based on two adjacent correction images in the box surface correction image sequence to obtain matching characteristic point pairs of the adjacent correction images; and calculating to obtain pixel displacement values when the adjacent corrected images are spliced based on the matched characteristic point pairs of the adjacent corrected images.

Optionally, extracting feature points and matching feature points based on two adjacent correction images in the box surface correction image sequence to obtain a matching feature point pair of the adjacent correction images, including: extracting characteristic points of two adjacent correction images in the box surface correction image sequence to obtain characteristic points of the adjacent correction images; roughly matching the characteristic points of the adjacent correction images to obtain matching characteristic point pairs of the initial adjacent correction images; and removing the false matching characteristic point pairs existing in the matching characteristic point pairs of the initial adjacent corrected images to obtain the matching characteristic point pairs of the adjacent corrected images.

Optionally, calculating, based on the matching feature point pairs of the adjacent corrected images, a pixel displacement value when the adjacent corrected images are spliced, including: matching feature point pairs based on the adjacent rectified imagesThe pixel error threshold value in the direction is removed, the abnormal matching characteristic point pair is removed, and the connecting line of the matching characteristic point pair and +.>Matching feature point pairs with directions close to parallel; if the matching characteristic point pair is connected with + ->The directions are close to parallel, and the rear graph of the adjacent correction image is calculated to be in the range of +.>The pixel displacement required to be moved in the direction is subjected to incremental sequencing, so that a pixel displacement set of the adjacent corrected images is obtained; when the adjacent correction images are spliced based on the pixel displacement set, the rear image of the adjacent correction image is in the +.>And processing the pixel displacement values in the direction to obtain the pixel displacement values when the adjacent corrected images are spliced.

Optionally, the back image of the adjacent corrected image is at the same time relative to the front image of the adjacent corrected image when the adjacent corrected image is stitched based on the pixel displacement setProcessing the pixel displacement value in the direction to obtain the pixel displacement value when the adjacent corrected images are spliced, wherein the pixel displacement value comprises: judging whether the adjacent corrected images have splicing feasibility or not based on the pixel displacement set; if the stitching feasibility is not provided, setting 0 as the sum of the rear image of the adjacent correction image and the front image of the adjacent correction image when the adjacent correction image is stitched >The pixel displacement value in the direction is 0 when the adjacent correction images are spliced; if the stitching feasibility is provided, dividing the pixel displacement of the pixel displacement set into a plurality of pixel displacement sections, determining the pixel displacement section with the largest pixel displacement, extracting the intermediate value of the pixel displacement section with the largest pixel displacement, and setting the intermediate value as the adjacent correctionThe back image of the adjacent corrected image is +.>And the pixel displacement value in the direction is the intermediate value when the adjacent corrected images are spliced.

Optionally, stitching the plurality of corrected images into a container panoramic image based on the pixel displacement value to obtain a once stitched panoramic image, including: calculating a first fusion radius of the adjacent corrected images during image fusion based on the pixel displacement values, wherein the first fusion radius represents that a transition region of the boundary of the corrected images before and after image fusion is positioned inA first width in the direction; re-selecting the length of the rear image of the adjacent correction image based on the pixel displacement value and the first fusion radius, wherein the front image of the adjacent correction image is used as a reference image and still keeps the size of the original image, the rear image of the adjacent correction image is used as an image matched with the front image of the adjacent correction image and spliced, the front image of the adjacent correction image and the rear image of the intercepted adjacent correction image are subjected to seamless splicing according to the pixel displacement value and the first fusion radius, and a spliced first result image is obtained; and taking the first result image as a reference image for the next loop stitching until a plurality of corrected images are stitched into a complete container panoramic image, so as to obtain the one-time stitching panoramic image.

Optionally, obtaining the corrected pixel displacement value when the adjacent corrected images in the box surface corrected image sequence are spliced includes: obtaining the aspect ratio of the panoramic image of the secondary spliced container and the size of the corrected image; calculating the length of the panoramic image of the secondary spliced container according to the aspect ratio of the panoramic image of the secondary spliced container and the size of the corrected image; calculating to obtain adjacent correction images according to the length of the panoramic image of the secondary spliced container and the number of correction images participating in splicingAdjacent two corrected images in positive image stitchingAverage pixel displacement value in the direction; and calculating the corrected pixel displacement value according to the pixel displacement value and the average pixel displacement value.

Optionally, stitching the plurality of corrected images into a container panoramic image based on the corrected pixel displacement value to obtain a secondary stitched panoramic image, including: calculating a second fusion radius of the adjacent corrected images during image fusion based on the corrected pixel displacement values, wherein the second fusion radius represents that a transition region of the boundary of the corrected images before and after image fusion is inA second width in the direction; re-selecting the length of the rear image of the adjacent correction image based on the corrected pixel displacement value and the second fusion radius, wherein the front image of the adjacent correction image is used as a reference image and still keeps the size of the original image, the rear image of the adjacent correction image is used as an image matched with the front image of the adjacent correction image and then spliced, the corrected pixel displacement value and the second fusion radius are required to be intercepted, and the front image of the adjacent correction image and the intercepted rear image of the adjacent correction image are subjected to seamless splicing to obtain a spliced second result image; and taking the second result image as a reference image for the next loop stitching until a plurality of corrected images are stitched into a complete container panoramic image, so as to obtain the secondary stitching panoramic image.

According to another aspect of the present invention, there is provided a stitching system of container panoramic images, comprising: the first processing module is used for acquiring a box surface image sequence of the container, wherein the box surface image sequence comprises a plurality of box surface images, and the box surface images are sorted according to time increment; the second processing module is used for carrying out distortion correction on the box surface images in the box surface image sequence to obtain a box surface correction image sequence of the container, wherein the box surface correction image sequence comprises a plurality of correction images with the same size, and each correction image corresponds to one box surface image; the third processing module is used for acquiring pixel displacement values when adjacent correction images in the box surface correction image sequence are spliced, and splicing a plurality of correction images into a container panoramic image based on the pixel displacement values to obtain a one-time spliced panoramic image; the fourth processing module is used for acquiring the length-width ratio section of the primary spliced panoramic image and the length-width ratio section of the container panoramic image and judging whether the length-width ratio section of the primary spliced panoramic image is in the length-width ratio section of the container panoramic image or not; a fifth processing module, configured to determine the primary stitched panoramic image as a final stitched container panoramic image if the aspect ratio of the primary stitched panoramic image is within the aspect ratio interval of the container panoramic image; and a sixth processing module, configured to obtain a corrected pixel displacement value when adjacent corrected images in the sequence of case surface corrected images are stitched if the aspect ratio of the primary stitched panoramic image is not within the aspect ratio interval of the container panoramic image, stitch a plurality of corrected images into the container panoramic image based on the corrected pixel displacement value, obtain a secondary stitched panoramic image, and determine the secondary stitched panoramic image as a final stitched container panoramic image.

In the invention, a container face image sequence of a container is acquired; carrying out distortion correction on the box surface images in the box surface image sequence to obtain a box surface correction image sequence of the container; acquiring pixel displacement values of adjacent correction images in a box surface correction image sequence when splicing, and splicing a plurality of correction images into a container panoramic image based on the pixel displacement values to obtain a once spliced panoramic image; judging whether the aspect ratio of the once spliced panoramic image is in the aspect ratio interval of the container panoramic image or not; if the aspect ratio of the primary spliced panoramic image is within the aspect ratio interval of the container panoramic image, determining the primary spliced panoramic image as the final spliced container panoramic image; if the aspect ratio of the primary spliced panoramic image is not in the aspect ratio section of the container panoramic image, the corrected pixel displacement value of the adjacent corrected images in the box surface corrected image sequence is obtained, a plurality of corrected images are spliced into the container panoramic image based on the corrected pixel displacement value, a secondary spliced panoramic image is obtained, and the secondary spliced panoramic image is determined to be the final spliced container panoramic image, so that the technical problem that in the related art, when the gate environment has light reflection and shadow, a good container panoramic image splicing result is difficult to obtain is solved, and the technical effect of improving the container panoramic image splicing result under various adverse conditions such as the light reflection and the shadow is achieved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below. It is evident that the drawings in the following description are only some embodiments of the invention, from which other embodiments can be obtained for a person skilled in the art without inventive effort.

Fig. 1 is a flowchart of a method for stitching panoramic images of a container according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of container panoramic image stitching provided by an alternative embodiment of the present invention;

FIG. 3 is a flow chart of a method for rectified image feature extraction and matching according to an alternative embodiment of the present invention;

FIG. 4 is a flowchart of calculating a stitched pixel displacement value according to an alternative embodiment of the present invention;

FIG. 5 is a flow chart of one-time stitching of panoramic images of a container provided by an alternative embodiment of the present invention;

FIG. 6 is a flow chart of a container panoramic image secondary stitching provided by an alternative embodiment of the present invention;

FIG. 7 is a graph showing a comparison of images before and after stitching according to an alternative embodiment of the present invention;

Fig. 8 is a schematic diagram of a stitching system for panoramic images of a container according to an embodiment of the present invention;

fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the invention have been shown in the accompanying drawings, it is to be understood that embodiments of the invention may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but are provided to provide a more thorough and complete understanding of the embodiments of the invention. It should be understood that the drawings and embodiments of the present invention are designed solely for the purposes of illustration and not as a definition of the limits of the invention.

According to one aspect of the embodiment of the invention, a method for stitching panoramic images of a container is provided. Fig. 1 is a flowchart of a method for stitching panoramic images of a container according to an embodiment of the present invention, as shown in fig. 1, the method includes the following steps:

step S102, a box surface image sequence of a container is obtained, wherein the box surface image sequence comprises a plurality of box surface images, and the box surface images are sorted in an increasing way according to time;

Step S104, carrying out distortion correction on the box surface images in the box surface image sequence to obtain a box surface correction image sequence of the container, wherein the box surface correction image sequence comprises a plurality of correction images with the same size, and each correction image corresponds to one box surface image;

step S106, obtaining pixel displacement values when adjacent correction images in the box surface correction image sequence are spliced, and splicing a plurality of correction images into a container panoramic image based on the pixel displacement values to obtain a primary spliced panoramic image; the pixel displacement value is calculated based on the matched characteristic point pairs of the adjacent corrected images;

step S108, obtaining the length-width ratio section of the primary spliced panoramic image and the length-width ratio section of the container panoramic image, and judging whether the length-width ratio section of the primary spliced panoramic image is within the length-width ratio section of the container panoramic image;

step S110, if the aspect ratio of the primary spliced panoramic image is within the aspect ratio interval of the container panoramic image, determining the primary spliced panoramic image as the final spliced container panoramic image;

step S112, if the aspect ratio of the primary spliced panoramic image is not in the aspect ratio section of the container panoramic image, acquiring a corrected pixel displacement value when adjacent corrected images in the sequence of the container face corrected images are spliced, splicing a plurality of corrected images into the container panoramic image based on the corrected pixel displacement value, obtaining a secondary spliced panoramic image, and determining the secondary spliced panoramic image as a final spliced container panoramic image; the corrected pixel displacement value is calculated based on the pixel displacement value and the pixel displacement value during secondary stitching.

In the embodiment of the invention, a container face image sequence of a container is acquired; carrying out distortion correction on the box surface images in the box surface image sequence to obtain a box surface correction image sequence of the container; acquiring pixel displacement values of adjacent correction images in a box surface correction image sequence when splicing, and splicing a plurality of correction images into a container panoramic image based on the pixel displacement values to obtain a once spliced panoramic image; judging whether the aspect ratio of the once spliced panoramic image is in the aspect ratio interval of the container panoramic image or not; if the aspect ratio of the primary spliced panoramic image is within the aspect ratio interval of the container panoramic image, determining the primary spliced panoramic image as the final spliced container panoramic image; if the aspect ratio of the primary spliced panoramic image is not in the aspect ratio section of the container panoramic image, the corrected pixel displacement value of the adjacent corrected images in the box surface corrected image sequence is obtained, a plurality of corrected images are spliced into the container panoramic image based on the corrected pixel displacement value, a secondary spliced panoramic image is obtained, and the secondary spliced panoramic image is determined to be the final spliced container panoramic image, so that the technical problem that in the related art, when the gate environment has light reflection and shadow, a good container panoramic image splicing result is difficult to obtain is solved, and the technical effect of improving the container panoramic image splicing result under various adverse conditions such as the light reflection and the shadow is achieved.

In addition, the container panoramic image splicing method provided by the embodiment of the invention is more suitable for container splicing tasks, and can effectively improve the splicing efficiency and stability.

Optionally, calculating an aspect ratio section of the container panoramic image based on the box type recognition result; if the aspect ratio of the panoramic image is spliced once, the panoramic image can be regarded as a correct panoramic image of the container is obtained by splicing once in the aspect ratio interval of the panoramic image of the container specified by the box type; and otherwise, taking the container panoramic image obtained by the primary stitching as a stitching abnormal image, and correcting the container panoramic image through the secondary stitching.

FIG. 2 is a schematic diagram of container panoramic image stitching according to an alternative embodiment of the present invention, as shown in FIG. 2, inputZhang Zhakou continuously capturing box surface images by a camera; carrying out distortion correction on the box surface image to obtain a corrected image; extracting characteristic points and matching the characteristic points of the corrected image; calculating a pixel displacement value when two images are spliced; performing seamless stitching by adopting image weighted average fusion to obtain a primary stitching panoramic image; judging whether to perform secondary splicing; if not, taking the once spliced panoramic image as a finally spliced container panoramic image, and ending the container panoramic image splicing process; if so, calculating the length of the panoramic image of the container during secondary stitching according to the box type identification result, and calculating a corrected pixel displacement value when two images are stitched by combining a pixel displacement value set of the panoramic image which is stitched once; and performing seamless stitching by adopting image weighted average fusion to obtain a secondary stitching panoramic image, taking the secondary stitching panoramic image as a final stitched container panoramic image, and ending the container panoramic image stitching flow.

Optionally, the method for stitching the container panoramic image based on image feature point matching provided by the optional embodiment of the invention includes the following steps:

step 1: acquiring an image set of a certain container surface acquired by a camera when the container passes through a gate, and sequencing the container surface images according to a time increasing sequence;

step 2: carrying out distortion correction on the box surface image by combining the distortion correction parameters to obtain corresponding corrected images, wherein the corrected images have the same size;

step 3: according to the sorting order of the corrected images, extracting characteristic points and matching the characteristic points based on two adjacent corrected images;

step 4: according to the matched characteristic point pairs of two adjacent corrected images, calculating a pixel displacement value d of the rear image relative to the front image in the x direction during splicing;

step 5: calculating a first fusion radius of the two images during stitching according to the pixel displacement value, and performing seamless stitching on the reference image and the correction image by adopting an image weighted average fusion method, wherein the stitched result image is used as a reference image for stitching in the next cycle;

step 6: repeating the steps 2 to 5 until all the images in the box surface image set are spliced into a complete container panoramic image, namely splicing the panoramic image once;

Step 7: judging whether secondary stitching is needed or not by combining the box type identification result and the aspect ratio of the primary stitching panoramic image, if the secondary stitching is not needed, the primary stitching panoramic image is regarded as a stitched container panoramic image, and if the secondary stitching is needed, the following steps are carried out;

step 8: when container panoramic image secondary splicing is carried out, the collecting card passing gate is regarded as a uniform speed passing gate, when the collecting card uniform speed passing gate is calculated according to the box type identification result and the length and width of the correction image, the length of the container panoramic image during secondary splicing is calculated, and when two adjacent images are splicedAverage pixel displacement value in the direction;

step 9: acquiring a corrected pixel displacement value according to the pixel displacement value and the average pixel displacement value of two adjacent corrected images when the two corrected images are spliced at one time;

step 10: calculating a second fusion radius of the two images during splicing according to the corrected pixel displacement value, and adopting an image weighted average fusion method to seamlessly splice the reference image and the corrected image, wherein the spliced result image is used as the reference image for the next circular splicing;

step 11: repeating the steps 9 to 10 until all the images in the box surface image set are spliced into a complete container panoramic image, namely, a secondary spliced panoramic image;

Step 12: and outputting the container panoramic image obtained by splicing the box surface image sets.

As an alternative embodiment, acquiring a sequence of facial images of a container includes: continuously acquiring a box surface image of the container based on the fixed camera point positions, and naming the box surface image in a box surface and timestamp mode; wherein the fixed camera point is configured to cover one face of the container; and sequencing the plurality of box surface images according to the time increment sequence to obtain a box surface image sequence.

Optionally, the images used for container panoramic image stitching are box surface images continuously captured by a gate camera when the truck is automatically passed through a gate, and the camera can cover three angles of a left box surface, a right box surface and a top box surface of the container to respectively stitch the container panoramic images of the left box surface, the right box surface and the top box surface.

In an alternative embodiment of the invention, the sequence of the container face images is obtained by acquiring an image set of a certain container face acquired by a camera when the container passes through a gate and sequencing the container face images according to the time increasing sequence.

As an alternative embodiment, the distortion correction is performed on a case surface image in a case surface image sequence, to obtain a case surface corrected image sequence of a container, including: obtaining distortion correction parameters of fixed camera points; and correcting radial distortion and tangential distortion of the box surface image in the box surface image sequence based on the distortion correction parameters to obtain the box surface correction image sequence.

Optionally, the box surface image acquired by the gate camera has deformation distortion of radial distortion and tangential distortion, and the radial distortion and the tangential distortion of the box surface image are corrected in sequence through the distortion correction parameters, so that corrected images are acquired, and the corrected images have the same size. The distortion correction parameters include, but are not limited to, focal length, pixel size, principal point shift, CCD correction, radial distortion, decentration distortion, and the like. The tangential distortion described above is also referred to as decentering distortion.

In an alternative embodiment of the invention, distortion correction is performed on each container face image in combination with the distortion correction parameters, so as to obtain corresponding corrected images with the same size, and further obtain a container face corrected image sequence of the container.

As an alternative embodiment, acquiring pixel displacement values when adjacent rectified images in the bin-surface rectified image sequence are spliced includes: extracting feature points and matching the feature points based on two adjacent correction images in the box surface correction image sequence to obtain matching feature point pairs of the adjacent correction images; and calculating to obtain pixel displacement values when the adjacent correction images are spliced based on the matched characteristic point pairs of the adjacent correction images.

As an optional embodiment, performing feature point extraction and feature point matching based on two adjacent corrected images in the sequence of box surface corrected images to obtain a matched feature point pair of the adjacent corrected images, including: extracting characteristic points of two adjacent correction images in the box surface correction image sequence to obtain characteristic points of the adjacent correction images; roughly matching the characteristic points of the adjacent correction images to obtain matching characteristic point pairs of the initial adjacent correction images; and removing the false matching characteristic point pairs existing in the matching characteristic point pairs of the initial adjacent corrected images to obtain the matching characteristic point pairs of the adjacent corrected images.

Optionally, extracting feature points of two adjacent corrected images by using a Scale-Invariant Feature Transform algorithm to obtain feature points of the adjacent corrected images; rough matching is carried out on the characteristic points of the adjacent correction images by using a knnMatch (K-NearestNeighborMatch) algorithm, and matching characteristic point pairs of initial adjacent correction images are obtained; and removing the wrong matching characteristic point pairs existing in the matching result by using a GMS (GMS-Grid-based Motion Statistics) algorithm to obtain final matching characteristic point pairs, namely, the matching characteristic point pairs of the adjacent corrected images.

FIG. 3 is a flowchart of feature extraction and matching of corrected images according to an alternative embodiment of the present invention, as shown in FIG. 3, feature detection is performed on a front image and a rear image of an adjacent corrected image by using a SIFT algorithm; then, carrying out rough matching on characteristic points by using a knnMatch algorithm; and finally, carrying out accurate matching of the characteristic points by using a GMS algorithm, judging the matching logarithm, and obtaining the characteristic point pairs.

Optionally, according to the sorting order of the corrected images, based on two adjacent corrected images, feature point extraction and feature point matching are performed, and the specific steps are as follows:

step 1: extracting feature points of a corrected image based on a SIFT algorithm, wherein the SIFT algorithm is feature detection irrelevant to scaling, has scale invariance, can detect stable feature points and establish a corresponding relation even if scaling, deformation, blurring, brightness change, illumination change and noise addition are carried out on the image or even if photos with different angles are shot by using different cameras, the SIFT image features are expressed as key point descriptors, and when the images are checked to be matched, two groups of key point descriptors are provided as input of Nearest Neighbor Search (NNS) and matched key point descriptors are generated;

Step 2: rough matching is carried out on the feature points based on a knnMatch algorithm, the knnMatch is an image matching algorithm based on feature descriptors, in practical application, the feature points in an image are usually used by combining feature extraction algorithms such as SIFT, SURF and the like, the knnMatch algorithm can find the best matching in the matching points for each feature point in the image, whether the matching is reliable or not is judged, if so, the matching is reserved, otherwise, the matching is abandoned;

step 3: the GMS algorithm is a rapid robust feature matching filtering algorithm based on motion statistics, can obviously improve matching results, and can rapidly remove incorrect matching by a grid division and motion statistics characteristic method so as to improve matching stability and obtain matched feature point pairs in two corrected images.

The key points with scale and rotation invariance are searched in the spliced images through a SIFT algorithm and matched, and the overlapped part of the two images during splicing is calculated according to the pixel distance between the matched characteristic points. The characteristic points established by the algorithm have good stability, even if the image is zoomed, deformed, blurred, changed in brightness, changed in illumination and added with noise, even if different cameras are used for shooting pictures with different angles, the algorithm can detect the stable characteristic points and establish a corresponding relation, so that the difficulty in calculating the moving distance of the pixels of the characteristic map and selecting the splicing position in the process of splicing the container surfaces in the gate environment is greatly reduced, and the method has certain feasibility.

As an optional embodiment, calculating, based on the matched feature point pairs of the adjacent corrected images, a pixel displacement value when the adjacent corrected images are spliced includes: matching feature point pairs based on adjacent rectified imagesThe pixel error threshold value in the direction is removed, the abnormal matching characteristic point pair is removed, and the connecting line of the matching characteristic point pair and +.>Matching feature point pairs with directions close to parallel; if the matching characteristic point pair is connected with + ->The directions are close to parallel, and the back image of the adjacent correction image is calculated to be +.>The pixel displacement required to be moved in the direction is subjected to incremental sequencing, so that a pixel displacement set of the adjacent corrected images is obtained; stitching the back image of the adjacent corrected image relative to the front image of the adjacent corrected image based on the set of pixel displacements>And processing the pixel displacement values in the direction to obtain the pixel displacement values when the adjacent corrected images are spliced.

FIG. 4 is a flowchart of calculating a pixel shift value for stitching, according to an alternative embodiment of the present invention, as shown in FIG. 4, by inputting a matched pair of features of two corrected imagesRemoving abnormal matching characteristic point pairs by using a pixel error threshold value in the direction; extracting characteristic point pairs >A set of directional pixel displacements; judging whether the corrected image has splicing feasibility or not according to the pixel displacement set; if the splicing feasibility is not provided, the pixel displacement value is 0; if the segment is provided with splicing feasibility, segment statistics with the interval of 5 is carried out on the pixel distances in the set; and extracting a pixel distance interval with the largest elements in the interval, wherein the pixel displacement value is the interval median value.

Optionally, calculating the position of the post graph relative to the pre graph during stitching according to the matched characteristic point pairs of the two adjacent corrected imagesPixel shift value in direction +.>The method comprises the following specific steps:

step 1: because the case face images in the same case face image set are all obtained by shooting with cameras with the same fixed angle, the corrected images have the same size, and the original characteristics and shapes of the restored container case face are the same, the connection line of the correct characteristic point pairs in the two corrected images which are spliced should be the same asThe directions are nearly parallel, so that the matching characteristic point pairs can be arranged at +.>Directional pixel error threshold +.>Removing abnormal matching characteristic point pairs, and setting a coordinate of a certain pair of matching characteristic point pairs of a front image and a rear image in two corrected images>And->If->The matching feature point pair is connected with the lineThe directions are nearly parallel;

Step 2: connecting line based on matching characteristic point pairsWhen the directions are nearly parallel, the calculated post-graph is +.>Pixel Displacement of the required movement in the direction +.>，/>Sequentially calculating pixel displacement of all matched characteristic point pairs in two corrected images>And performing incremental sequencing to obtain a pixel displacement setThe aggregate length is +.>；

Step 3: based on pixel displacement setsCalculating the position of the post graph relative to the pre graph at the time of splicing>Pixel shift value in direction +.>The specific method is as follows:

1) For pixel displacement setJudging whether all elements are larger than 3, storing elements not larger than 3 into a collection +.>In (3) the aggregate length is +.>Elements larger than 3 are stored in the collection +.>In (3) the aggregate length is +.>；

2) If you doAnd->When the matched corrected images have no coordinate change with more than 90% of the characteristic point pairs having pixel displacement exceeding 3, the characteristic point pairs having pixel displacement in the two spliced corrected images are too few and do not have splicing feasibility, so that the rear image is opposite to the front image during splicing>Pixel shift value in direction +.>The back graph is represented without being spliced with the front graph;

3) If two adjacent corrected images have splicing feasibility, combining The segment statistics with interval 5 is carried out on all elements in the table, namely the statistical interval is +.>Extracting the interval with the largest number of elements in the intervalI.e. after splicing in relation to before>Pixel shift value in direction +.>The probability of being located in the interval is the largest, and the intermediate value of the interval is taken as the pixel displacement value, namely +.>The back graph is shown to be +.>The pixel distance shifted in the direction is +.>。

By setting the matching characteristic point pairs inDirectional pixel error threshold +.>Removing abnormal matching characteristic point pairs, and reserving connecting lines of the matching characteristic point pairs and +.>Matching feature point pairs with directions close to parallel; based on matching characteristic point pair connection line and +.>When the directions are nearly parallel, the calculated post-graph is +.>Pixel Displacement of the required movement in the direction +.>And performing incremental sequencing to obtain pixel displacement sets of adjacent corrected imagesThe method comprises the steps of carrying out a first treatment on the surface of the Pixel displacement set based on adjacent rectified imagesCalculating the position of the post graph relative to the pre graph at the time of splicing>Pixel shift value in direction +.>。

As an alternative embodiment, stitching the back image of the adjacent corrected image relative to the front image of the adjacent corrected image based on the set of pixel displacements is performed Processing the pixel displacement value in the direction to obtain the pixel displacement value when the adjacent corrected images are spliced, including: judging whether the adjacent corrected images have splicing feasibility or not based on the pixel displacement set; if the stitching feasibility is not available, setting 0 as the stitching feasibility of the adjacent corrected images, wherein the rear image of the adjacent corrected images is in the range of +.>The pixel displacement value in the direction is 0 when the adjacent correction images are spliced; if the stitching feasibility is provided, dividing the pixel displacement of the pixel displacement set into a plurality of pixel displacement sections, determining the pixel displacement section with the largest pixel displacement, extracting the intermediate value of the pixel displacement section with the largest pixel displacement, setting the intermediate value as the condition that the rear image of the adjacent corrected image is in the position of the front image of the adjacent corrected image when the adjacent corrected image is stitched>Pixel displacement in the directionAnd if so, the pixel displacement value of the adjacent correction images during stitching is an intermediate value.

As an alternative embodiment, stitching the plurality of corrected images into the container panoramic image based on the pixel displacement values, to obtain a stitched panoramic image, includes: calculating a first fusion radius of adjacent corrected images during image fusion based on pixel displacement values, wherein the first fusion radius represents that a transition region of a boundary of two corrected images before and after image fusion is positioned A first width in the direction; re-selecting the length of the rear image of the adjacent correction image based on the pixel displacement value and the first fusion radius, wherein the front image of the adjacent correction image is used as a reference image, the size of the original image is still kept, the rear image of the adjacent correction image is used as an image which is matched with the front image of the adjacent correction image and then spliced, the rear image of the adjacent correction image is required to be intercepted according to the pixel displacement value and the first fusion radius, and the front image of the adjacent correction image and the rear image of the intercepted adjacent correction image are subjected to seamless splicing, so that a spliced first result image is obtained; and taking the first result image as a reference image for the next loop stitching until a plurality of corrected images are stitched into a complete container panoramic image, so as to obtain a primary stitched panoramic image.

FIG. 5 is a flow chart of one-time stitching of container panoramic images according to an alternative embodiment of the present invention, as shown in FIG. 5, inputZhang Zhakou continuously-captured case face images, [ the way ]) by using camera>The method comprises the steps of carrying out a first treatment on the surface of the Judging whether or not->The method comprises the steps of carrying out a first treatment on the surface of the If->Distortion correction is performed on the box-face image, respectively for the +.>Zhang and->Performing SIFT feature point detection on the images, then performing feature point matching, calculating pixel displacement values when the two images are spliced, participating in the length of the spliced images, performing seamless splicing by adopting image weighted average fusion, and adding the pixel displacement values to a pixel displacement value set; judging whether the length of the pixel displacement value set is greater than 150, if so, splicing to obtain a primary spliced panoramic image; if no, then- >And continue to judge whether or not +.>The method comprises the steps of carrying out a first treatment on the surface of the If it isAnd ending the one-time splicing process.

Optionally, a gate camera is used for collecting an image set of a certain box surface when the container passes through a gate, the gate camera covers three angles of a left box surface (left), a right box surface (right) and a top box surface (top) of the container, a strict time stamp is given to the shooting time of each image, the box surface images are named in the form of 'box surface plus time stamp', and the box surface images are ordered according to the time increment sequence.

Further, in the case face image obtained by the camera, distortion phenomena such as distortion, stretching or compression occur in the image due to physical characteristics of light passing through the lens, so that distortion correction parameters are combined to correct distortion of the case face original image one by one, corresponding corrected images are obtained, the corrected images have the same size, and the length of the corrected images isWidth is->。

Further, pixel bits relative to the front image according to the rear imageShift valueThe images are registered and aligned in pairs and then spliced, and when the images are directly spliced, the two images are spliced at the juncture of the spliced images due to the obvious splice at the juncture of the two images caused by illumination color and luster and the like, so that an image weighted average fusion method is adopted according to the pixel displacement value +. >Calculating the fusion radius of two images during splicing>Seamless splicing is carried out on the reference image and the correction image, and the specific steps are as follows:

step 1: when the image weighted average fusion algorithm is used for splicing, the overlapped part is gradually transited from the previous image to the second image, namely, pixel values of the overlapped area of the images are added according to a certain weight to form a new image, and the image weighted average fusion algorithm can be expressed as follows:

wherein,representing the previous reference image in the fusion, < +.>Representing the next corrected image in fusion,/->Representing the fused image and using the fused image as a reference image for image fusion in the next loop stitching>Representing the row coordinates of the image, ">Column coordinates representing an image, +.>For the weighting factor>；

Step 2: based on pixel displacement valuesCalculating a first blend radius +.>First blend radius->The transition area representing the boundary of the two images before and after image fusion is +.>A first width in the direction, if the pixel displacement valueThen->On the contrary->；

Step 3: based on pixel displacement valuesAnd a first blend radius->Re-selecting the length of the corrected image of the rear image, wherein the front image is used as a reference image and still keeps the size of the original image, the rear image is used as an image which is matched with the front image and then spliced, and the image is selected from + >The range of the interval in the direction is +.>Is taken part in the stitching as a new post-graph, wherein +.>Length pixel values for the rectified image;

step 4: calculating a weighting coefficient based on the above information，/>Weighting coefficients representing the front graph, < >>，/>Weighting coefficients representing the postmap, < >>Wherein->A length pixel value representing a reference image;

step 5: and performing seamless splicing on the reference image and the corrected image through the formulas and parameters in the image weighted average fusion algorithm to obtain a seamless splicing result, wherein the seamless splicing result is used as a new reference image to participate in image fusion of next loop splicing.

Further, the steps 1 to 5 are circularly executed until all the images in the box surface image set are spliced into a complete container panoramic image, namely, the panoramic image is spliced once, and pixel displacement values when two adjacent correction images in the image set are spliced are reservedTo the pixel shift value set +.>Length of pixel displacement value setWherein->For the length of the image set, if the pixel displacement value set length +.>And finishing the stitching cycle in advance without waiting for all images in the image set to participate in stitching, and taking a stitching result diagram at the end of the stitching cycle as a one-time stitching panoramic image.

When the image weighted average fusion algorithm is adopted for splicing, pixel values of the image overlapping areas are added according to a certain weight to form a new image; based on pixel displacement valuesCalculating a first blend radius +.>First blend radius->The transition area representing the boundary of the two images before and after image fusion is +.>A first width in the direction; based on the pixel displacement value +.>And a first blend radius->The length of the corrected image of the rear image is selected again, the front image is used as a reference image, the size of the original image is still kept, the rear image is used as an image which is matched with the front image and then spliced, and the image is required to be spliced according to the pixel displacement value +.>And a first blend radius->Intercepting, and performing seamless splicing on the front graph and the intercepted rear graph; and (3) taking the spliced result image as a reference image of the next loop splice, and repeating the operation until all the images in the box surface image set are spliced into a complete container box surface image.

In an alternative embodiment of the invention, based on two adjacent corrected images, feature point extraction and feature point matching are performed to obtain matched feature point pairs, pixel displacement values during splicing of the adjacent corrected images are calculated, seamless splicing is performed on the reference images and the corrected images by adopting an image weighted average fusion method, the spliced result images are used as reference images for next circular splicing, and the operations are repeated until all images in a box surface image set are spliced into a complete container box surface image, namely, a container panoramic image is spliced once.

As an alternative embodiment, obtaining corrected pixel displacement values when adjacent corrected images in the box correction image sequence are spliced includes: obtaining the aspect ratio of the panoramic image of the secondary spliced container and the size of the corrected image; calculating the length of the panoramic image of the secondary spliced container according to the aspect ratio of the panoramic image of the secondary spliced container and the size of the corrected image; according to the length of the panoramic image of the secondary spliced container and the number of the corrected images participating in splicing, calculating to obtain two adjacent corrected images when the adjacent corrected images are splicedAverage pixel displacement value in the direction; and calculating a corrected pixel displacement value according to the pixel displacement value and the average pixel displacement value.

Further, stitching the plurality of corrected images into the container panoramic image based on the corrected pixel displacement value to obtain a secondary stitched panoramic image, comprising: calculating a second fusion radius when adjacent correction images are fused based on the correction pixel displacement value, wherein the second fusion radius represents that a transition region of a boundary of two correction images before and after image fusion is positionedA second width in the direction; re-selecting the length of the rear image of the adjacent correction image based on the corrected pixel displacement value and the second fusion radius, wherein the front image of the adjacent correction image is used as a reference image, the size of the original image is still kept, the rear image of the adjacent correction image is used as an image which is matched with the front image of the adjacent correction image and then spliced, the front image of the adjacent correction image and the rear image of the spliced adjacent correction image are subjected to seamless splicing according to the corrected pixel displacement value and the second fusion radius, and a spliced second result image is obtained; and taking the second result image as a reference image of the next loop stitching until a plurality of corrected images are stitched into a complete container panoramic image, so as to obtain a secondary stitching panoramic image.

FIG. 6 is a flow chart of secondary stitching of panoramic images of a container according to an alternative embodiment of the present invention, as shown in FIG. 6, the result of box recognition is read, and whether secondary stitching is needed is determined by combining the primary stitched panoramic images; if not, taking the once spliced panoramic image as a final spliced container panoramic image; if so, inputZhang Jiaozheng image and set of pixel displacement values, +.>The method comprises the steps of carrying out a first treatment on the surface of the Judging whether or not->The method comprises the steps of carrying out a first treatment on the surface of the If yes, the first +.>The individual elements are->The method comprises the steps of carrying out a first treatment on the surface of the Judging whether or not->The method comprises the steps of carrying out a first treatment on the surface of the If->Then the corrected pixel displacement value is +.>The method comprises the steps of carrying out a first treatment on the surface of the If->Then the corrected pixel displacement value is +.>The method comprises the steps of carrying out a first treatment on the surface of the Further calculating the length of the rear graph participating in the splicing; seamless stitching by adopting image weighted average fusion, < > and the like>The method comprises the steps of carrying out a first treatment on the surface of the And if not, acquiring and splicing to obtain a secondary spliced panoramic image. According to the length-width ratio of the box type calculation box surface, the length of the container panoramic image during secondary splicing is calculated by combining the length-width pixel value of the obtained distortion corrected image, and then the average pixel displacement value +_in the secondary splicing is calculated>。

Optionally, when some correction images participating in stitching have fewer matching feature points and are insufficient to obtain reasonable pixel displacement values, the length of the container panoramic image stitched once is obviously smaller than the actual length of the container in the image, and defects of picture loss, picture compression or unreasonable stitching are present in the stitched image, so that based on a box type recognition result and the aspect ratio of the panoramic image stitched once, whether secondary stitching is needed or not is judged, and the judgment steps are as follows:

Step 1: the container type codes of the containers determine the length, width and height dimensions of the containers, the common 20-ruler container type codes are 22G1, the container type dimensions are 6.058M x 2.438M x 2.591M, the 40-ruler container type codes are 42G1 and 45G1, the 42G1 type dimensions are 12.190M x 2.435M x 2.588M, the 45G1 type dimensions are 12.192M x 2.438M x 2.896M, a headstock or tailstock blank area possibly exists in a container splicing panoramic image, parts which do not belong to container faces also exist above and below the container, therefore, the length of the spliced panoramic image is slightly longer than the length of the container in the image, the width of the container is slightly longer than the width or the height of the container in the image, the aspect ratio (aspect ratio example = image length: image width) of the container panoramic image under different container type codes can be set as a threshold value based on the above information, according to a large number of container panoramic image splicing result setting rules as follows:

1) In case of the box code bit 45G1, the aspect ratio section of the panoramic image of the left box surface and the right box surface isThe aspect ratio of the top box panoramic image is +.>；

2) In the case of the box code bit 42G1, the aspect ratio section of the panoramic image of the left box surface and the right box surface isThe aspect ratio section of the top box panoramic image is +. >；

3) When the container is a single small container, the aspect ratio section of the panoramic images of the left container surface and the right container surface isThe aspect ratio section of the top box panoramic image is +.>If the container is a double-small box, the aspect ratio section of the panoramic images of the left box surface and the right box surface is +.>The aspect ratio section of the top box panoramic image is；

Step 2: according to the box type recognition result obtained by carrying out box number recognition on the box surface image through the YOLOv4 and the PaddleOCR when the collection card passes the gate, taking the 3 types of boxes as examples (22G1,42G1,45G1), the aspect ratio (aspect ratio=image length: image width) of the panoramic image obtained after the primary stitching is in the range of the rule set in the step 1, and can be regarded as a primary stitching to obtain a correct container panoramic image, if the aspect ratio of the primary stitching panoramic image is smaller than the lower limit of the rule range or larger than the upper limit of the range, the container panoramic image obtained by the primary stitching is regarded as a stitching abnormal image, the container panoramic image is corrected through the secondary stitching, and if the secondary stitching is not needed, the primary stitching image is regarded as a stitched container panoramic image.

Further, when the container panoramic image is spliced secondarily, the collecting card passing gate is regarded as a uniform speed passing gate, and when the collecting card uniform speed passing gate is calculated according to the box recognition result and the width pixel value of the corrected image, two adjacent images are in the process of The average pixel displacement value in the direction is as follows:

step 1: the container type code of the container determines the length, width and height of the container, and the aspect ratio of the panoramic image of the container in secondary splicing is determined based on the container type identification result of the container in the process of passing the gate of the containerThe aspect ratio is obtained by analyzing the splicing results of a large number of container panoramic images, and the aspect ratio of the secondary spliced container panoramic images when different container types are set is as follows:

1) In case of the box code bit 45G1, aspect ratios of the left and right box panoramic images are as followsThe aspect ratio of the top box panoramic image is +.>；

2) When the box code bit 42G1 is used, the left box surfaceAspect ratio of the panoramic image with right box surface is as followsThe aspect ratio of the top box panoramic image is +.>；

3) When the container is a single small container, the aspect ratio of the panoramic images of the left container surface and the right container surface is as followsThe aspect ratio of the top box panoramic image is +.>If the container is a double small box, the aspect ratio of the panoramic images of the left box surface and the right box surface is +.>The aspect ratio of the top box panoramic image is +.>；

Step 2: the rectified images have the same size and are therefore based on the width pixel values of the rectified images Determining the length of the panoramic image of the container during the secondary stitching>；

Step 3: acquired pixel displacement value set during one-time stitchingHaving a length ofThe number of corrected images participating in one stitching is described as +.>There is->Zhang Jiaozheng images are spliced according to pixel displacement values on the basis of the previous image, and two adjacent correction images are in +.>Average pixel displacement value in the direction +.>Wherein->To correct the length pixel value of the image.

Further, based on pixel displacement values of two adjacent corrected images in one stitchingAnd average pixel displacement value +.>Calculate the corrected pixel displacement value +.>If->Then->On the contrary->。

Further, based on the corrected pixel displacement valueCalculating a second blend radius of the two images during stitching>The correction chart is subjected to weighted average fusion by adopting an image weighted average fusion methodAnd (3) performing seamless stitching on the images until all the images in the box surface image set are stitched into a complete container panoramic image, namely, stitching the panoramic image secondarily.

It should be noted that, based on two adjacent corrected images, the corrected pixel displacement value between the adjacent images is calculated by combining the box type recognition result and the pixel displacement value of one-time stitching, and the method includes the following steps: based on the result of identifying the container type during the passing of the gate of the collecting card, the aspect ratio of the panoramic image of the container during the secondary splicing is determined The aspect ratio is obtained by analyzing the splicing result of a large number of container panoramic images; determining the length of the container panoramic image at the time of secondary stitching based on the size of the corrected image and the aspect ratio of the container panoramic image>The method comprises the steps of carrying out a first treatment on the surface of the Length of panoramic image based on container>And the number of corrected images participating in stitching, wherein when two-time stitching is calculated, two adjacent corrected images are in +.>Average pixel displacement value in the direction +.>The method comprises the steps of carrying out a first treatment on the surface of the Pixel displacement value based on adjacent two corrected images in one stitching>And average pixel displacement value +.>Calculate the corrected pixel displacement value +.>。

In an alternative embodiment of the invention, based on two adjacent corrected images, a box type recognition result and a pixel displacement value spliced at one time are combined, corrected pixel displacement values between the adjacent images are calculated, an image weighted average fusion method is adopted to seamlessly splice the reference image and the corrected images, the spliced result image is used as a reference image spliced in the next cycle, and the operation is repeated until all images in a box surface image set are spliced into a complete container box surface image, namely a secondary spliced container panoramic image.

Fig. 7 is a comparison chart of images before and after stitching, which is provided by an alternative embodiment of the present invention, as shown in fig. 7, a complete panoramic image of a container is finally obtained through distortion correction, image feature point extraction and matching, stitching pixel displacement value calculation, image weighted average fusion and stitching elimination, image frame-by-frame stitching, secondary stitching judgment, secondary stitching and other processes of a sequence of images of a container formed by a plurality of images of the container.

According to another aspect of an embodiment of the present invention, there is provided a stitching system for panoramic images of a container. Fig. 8 is a schematic diagram of a stitching system of panoramic images of a container according to an embodiment of the present invention, as shown in fig. 8, where the stitching system of panoramic images of a container includes: a first processing module 802, a second processing module 804, a third processing module 806, a fourth processing module 808, a fifth processing module 810, and a sixth processing module 812. The system for stitching the panoramic image of the container will be described in detail.

A first processing module 802, configured to obtain a sequence of case top images of a container, where the sequence of case top images includes a plurality of case top images, and the plurality of case top images are sorted in an incremental manner according to time;

the second processing module 804 is configured to perform distortion correction on a box surface image in the box surface image sequence to obtain a box surface correction image sequence of the container, where the box surface correction image sequence includes a plurality of correction images with the same size, and each correction image corresponds to one box surface image;

a third processing module 806, configured to obtain pixel displacement values when adjacent corrected images in the sequence of corrected images of the box surface are spliced, and splice a plurality of corrected images into a container panoramic image based on the pixel displacement values, so as to obtain a once spliced panoramic image;

A fourth processing module 808, configured to obtain an aspect ratio section of the first stitched panoramic image and an aspect ratio section of the container panoramic image, and determine whether the aspect ratio section of the first stitched panoramic image is within the aspect ratio section of the container panoramic image;

a fifth processing module 810, configured to determine the once stitched panoramic image as a final stitched panoramic image of the container if the aspect ratio of the once stitched panoramic image is within the aspect ratio interval of the panoramic image of the container;

and a sixth processing module 812, configured to obtain a corrected pixel displacement value when the adjacent corrected images in the sequence of the case surface corrected images are stitched if the aspect ratio of the first stitched panoramic image is not within the aspect ratio interval of the container panoramic image, stitch the plurality of corrected images into the container panoramic image based on the corrected pixel displacement value, obtain a second stitched panoramic image, and determine the second stitched panoramic image as the final stitched container panoramic image.

It should be noted that, the first processing module 802, the second processing module 804, the third processing module 806, the fourth processing module 808, the fifth processing module 810, and the sixth processing module 812 correspond to steps S102 to S112 in the method embodiment, and the foregoing modules are the same as examples and application scenarios implemented by the corresponding steps, but are not limited to the disclosure of the foregoing method embodiment.

In the embodiment of the invention, the system is used for acquiring a container face image sequence of the container; carrying out distortion correction on the box surface images in the box surface image sequence to obtain a box surface correction image sequence of the container; acquiring pixel displacement values of adjacent correction images in a box surface correction image sequence when splicing, and splicing a plurality of correction images into a container panoramic image based on the pixel displacement values to obtain a once spliced panoramic image; judging whether the aspect ratio of the once spliced panoramic image is in the aspect ratio interval of the container panoramic image or not; if the aspect ratio of the primary spliced panoramic image is within the aspect ratio interval of the container panoramic image, determining the primary spliced panoramic image as the final spliced container panoramic image; if the aspect ratio of the primary spliced panoramic image is not in the aspect ratio section of the container panoramic image, the corrected pixel displacement value of the adjacent corrected images in the box surface corrected image sequence is obtained, a plurality of corrected images are spliced into the container panoramic image based on the corrected pixel displacement value, a secondary spliced panoramic image is obtained, and the secondary spliced panoramic image is determined to be the final spliced container panoramic image, so that the technical problem that in the related art, when the gate environment has light reflection and shadow, a good container panoramic image splicing result is difficult to obtain is solved, and the technical effect of improving the container panoramic image splicing result under various adverse conditions such as the light reflection and the shadow is achieved.

According to another aspect of an embodiment of the present invention, there is provided an electronic apparatus including: at least one processor; and a memory communicatively coupled to the at least one processor. The memory stores a computer program executable by the at least one processor, which when executed by the at least one processor is configured to cause an electronic device to perform the method for stitching panoramic images of containers according to embodiments of the present invention.

According to another aspect of an embodiment of the present invention, there is provided a non-transitory machine-readable medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is configured to cause the computer to perform the method for stitching panoramic images of a container according to the embodiment of the present invention.

According to another aspect of an embodiment of the present invention, there is provided a computer program product comprising a computer program, wherein the computer program, when executed by a processor of a computer, is for causing the computer to perform the method of stitching panoramic images of a container according to an embodiment of the present invention.

With reference to fig. 9, a block diagram of an electronic device that may be a server or a client of an embodiment of the present invention will now be described, which is an example of a hardware device that may be applied to aspects of the present invention. Electronic devices are intended to represent various forms of digital electronic computer devices, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.

As shown in fig. 9, the electronic device includes a computing unit 901 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 902 or a computer program loaded from a storage unit 908 into a Random Access Memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other by a bus 904. An input/output (I/O) interface 905 is also connected to the bus 904.

A number of components in the electronic device are connected to the I/O interface 905, including: an input unit 906, an output unit 907, a storage unit 908, and a communication unit 909. The input unit 906 may be any type of device capable of inputting information to an electronic device, and the input unit 906 may receive input numeric or character information and generate key signal inputs related to user settings and/or function controls of the electronic device. The output unit 907 may be any type of device capable of presenting information and may include, but is not limited to, a display, speakers, video/audio output terminals, vibrators, and/or printers. Storage unit 908 may include, but is not limited to, magnetic disks, optical disks. The communication unit 909 allows the electronic device to exchange information/data with other devices through a computer network such as the internet and/or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers and/or chipsets, such as bluetooth devices, wiFi devices, wiMax devices, cellular communication devices, and/or the like.

The computing unit 901 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 901 include, but are not limited to, a CPU, a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the respective methods and processes described above. For example, in some embodiments, method embodiments of the present invention may be implemented as a computer program tangibly embodied on a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device via the ROM 902 and/or the communication unit 909. In some embodiments, the computing unit 901 may be configured to perform the methods described above by any other suitable means (e.g., by means of firmware).

A computer program for implementing the methods of embodiments of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of embodiments of the present invention, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable signal medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

It should be noted that the term "comprising" and its variants as used in the embodiments of the present invention are open-ended, i.e. "including but not limited to". The term "based on" is based at least in part on. The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments. References to "one or more" modifications in the examples of the invention are intended to be illustrative rather than limiting, and it will be understood by those skilled in the art that "one or more" is intended to be interpreted as "one or more" unless the context clearly indicates otherwise.

The steps described in the method embodiments provided in the embodiments of the present invention may be performed in a different order and/or performed in parallel. Furthermore, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the invention is not limited in this respect.

The term "embodiment" in this specification means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the invention. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive. The various embodiments in this specification are described in a related manner, with identical and similar parts being referred to each other. In particular, for apparatus, devices, system embodiments, the description is relatively simple as it is substantially similar to method embodiments, see for relevant part of the description of method embodiments.

The foregoing examples illustrate only a few embodiments of the invention, which are described in detail and are not to be construed as limiting the scope of the claims. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of the invention should be assessed as that of the appended claims.

Claims

1. A method for stitching panoramic images of a container, comprising:

acquiring a box surface image sequence of a container, wherein the box surface image sequence comprises a plurality of box surface images, and the box surface images are sequenced in an increasing way according to time;

carrying out distortion correction on the box face images in the box face image sequence to obtain a box face correction image sequence of the container, wherein the box face correction image sequence comprises a plurality of correction images with the same size, and each correction image corresponds to one box face image;

acquiring pixel displacement values when adjacent corrected images in the box surface corrected image sequence are spliced, and splicing a plurality of corrected images into a container panoramic image based on the pixel displacement values to obtain a one-time spliced panoramic image;

acquiring an aspect ratio section of the primary spliced panoramic image and an aspect ratio section of the container panoramic image, and judging whether the aspect ratio section of the primary spliced panoramic image is in the aspect ratio section of the container panoramic image;

if the aspect ratio of the primary spliced panoramic image is in the aspect ratio interval of the container panoramic image, determining the primary spliced panoramic image as a final spliced container panoramic image;

And if the aspect ratio of the primary stitching panoramic image is not in the aspect ratio interval of the container panoramic image, acquiring a correction pixel displacement value when adjacent correction images in the box surface correction image sequence are stitched, stitching a plurality of correction images into the container panoramic image based on the correction pixel displacement value, obtaining a secondary stitching panoramic image, and determining the secondary stitching panoramic image as a final stitched container panoramic image.

2. The method for stitching panoramic images of a container according to claim 1, wherein the performing distortion correction on the face images in the face image sequence to obtain the face corrected image sequence of the container comprises:

obtaining distortion correction parameters of fixed camera points;

and correcting radial distortion and tangential distortion of the box surface image in the box surface image sequence based on the distortion correction parameters to obtain the box surface correction image sequence.

3. The method for stitching panoramic images of a container according to claim 1, wherein obtaining a pixel displacement value when stitching adjacent corrected images in the sequence of corrected images of the container surface comprises:

Extracting characteristic points and matching the characteristic points based on two adjacent correction images in the box surface correction image sequence to obtain matching characteristic point pairs of the adjacent correction images;

and calculating to obtain pixel displacement values when the adjacent corrected images are spliced based on the matched characteristic point pairs of the adjacent corrected images.

4. The method for stitching panoramic images of a container according to claim 3, wherein feature point extraction and feature point matching are performed based on two adjacent corrected images in the sequence of corrected images of the container surface, so as to obtain a matched feature point pair of the adjacent corrected images, comprising:

extracting characteristic points of two adjacent correction images in the box surface correction image sequence to obtain characteristic points of the adjacent correction images;

roughly matching the characteristic points of the adjacent correction images to obtain matching characteristic point pairs of the initial adjacent correction images;

and removing the false matching characteristic point pairs existing in the matching characteristic point pairs of the initial adjacent corrected images to obtain the matching characteristic point pairs of the adjacent corrected images.

5. A method of stitching container panoramic images according to claim 3, wherein calculating a pixel displacement value when stitching adjacent corrected images based on the matched pair of feature points of the adjacent corrected images comprises:

Removing abnormal matching characteristic point pairs based on pixel error thresholds of the matching characteristic point pairs of the adjacent corrected images in the y direction, and reserving the matching characteristic point pairs with connecting lines of the matching characteristic point pairs being close to parallel to the x direction;

if the connecting line of the matched characteristic point pairs is nearly parallel to the x direction, calculating the pixel displacement of the back image of the adjacent correction image, which is required to move in the x direction, relative to the front image of the adjacent correction image, and carrying out incremental sequencing to obtain a pixel displacement set of the adjacent correction image;

and processing pixel displacement values of a rear image of the adjacent correction image relative to a front image of the adjacent correction image in the x direction when the adjacent correction images are spliced based on the pixel displacement set to obtain the pixel displacement values when the adjacent correction images are spliced.

6. The method according to claim 5, wherein processing, based on the pixel displacement set, pixel displacement values in an x direction of a rear image of an adjacent corrected image relative to a front image of the adjacent corrected image when the adjacent corrected image is stitched, to obtain pixel displacement values when the adjacent corrected image is stitched, includes:

Judging whether the adjacent corrected images have splicing feasibility or not based on the pixel displacement set;

if the stitching feasibility is not provided, setting 0 as a pixel displacement value of a rear image of the adjacent corrected image relative to a front image of the adjacent corrected image in the x direction when the adjacent corrected image is stitched, and setting the pixel displacement value of the adjacent corrected image to be 0;

if the stitching feasibility is provided, dividing the pixel displacement of the pixel displacement set into a plurality of pixel displacement sections, determining the pixel displacement section with the most pixel displacement, extracting the intermediate value of the pixel displacement section with the most pixel displacement, setting the intermediate value as the pixel displacement value of the rear image of the adjacent corrected image relative to the front image of the adjacent corrected image in the x direction when the adjacent corrected image is stitched, and setting the pixel displacement value when the adjacent corrected image is stitched as the intermediate value.

7. The stitching method of container panoramic images according to claim 1, wherein stitching a plurality of corrected images into a container panoramic image based on the pixel displacement values, to obtain a stitched panoramic image at a time, comprises:

Calculating a first fusion radius of the adjacent corrected images during image fusion based on the pixel displacement values, wherein the first fusion radius represents a first width of a transition region of a boundary of the two corrected images before and after image fusion in an x direction;

re-selecting the length of the rear image of the adjacent correction image based on the pixel displacement value and the first fusion radius, wherein the front image of the adjacent correction image is used as a reference image and still keeps the size of the original image, the rear image of the adjacent correction image is used as an image matched with the front image of the adjacent correction image and spliced, the front image of the adjacent correction image and the rear image of the intercepted adjacent correction image are subjected to seamless splicing according to the pixel displacement value and the first fusion radius, and a spliced first result image is obtained;

and taking the first result image as a reference image for the next loop stitching until a plurality of corrected images are stitched into a complete container panoramic image, so as to obtain the one-time stitching panoramic image.

8. The method for stitching panoramic images of a container according to claim 1, wherein obtaining a corrected pixel displacement value for stitching adjacent corrected images in the sequence of corrected images of the container surface comprises:

Obtaining the aspect ratio of the panoramic image of the secondary spliced container and the size of the corrected image;

calculating the length of the panoramic image of the secondary spliced container according to the aspect ratio of the panoramic image of the secondary spliced container and the size of the corrected image;

according to the length of the panoramic image of the secondary spliced container and the number of the corrected images participating in splicing, calculating to obtain two adjacent corrected images when the adjacent corrected images are splicedAverage pixel displacement value in the direction;

and calculating the corrected pixel displacement value according to the pixel displacement value and the average pixel displacement value.

9. The stitching method of container panoramic images according to any one of claims 1 to 8, wherein stitching a plurality of corrected images into a container panoramic image based on the corrected pixel displacement values, to obtain a secondary stitched panoramic image, comprises:

calculating a second fusion radius of the adjacent corrected images when the images are fused based on the corrected pixel displacement values, wherein the second fusion radius represents a second width of a transition region of the boundary of the two corrected images before and after the image fusion in the x direction;

Re-selecting the length of the rear image of the adjacent correction image based on the corrected pixel displacement value and the second fusion radius, wherein the front image of the adjacent correction image is used as a reference image and still keeps the size of the original image, the rear image of the adjacent correction image is used as an image matched with the front image of the adjacent correction image and then spliced, the corrected pixel displacement value and the second fusion radius are required to be intercepted, and the front image of the adjacent correction image and the intercepted rear image of the adjacent correction image are subjected to seamless splicing to obtain a spliced second result image;

and taking the second result image as a reference image for the next loop stitching until a plurality of corrected images are stitched into a complete container panoramic image, so as to obtain the secondary stitching panoramic image.

10. A system for stitching panoramic images of a container, comprising:

the first processing module is used for acquiring a box surface image sequence of the container, wherein the box surface image sequence comprises a plurality of box surface images, and the box surface images are sorted according to time increment;

the second processing module is used for carrying out distortion correction on the box surface images in the box surface image sequence to obtain a box surface correction image sequence of the container, wherein the box surface correction image sequence comprises a plurality of correction images with the same size, and each correction image corresponds to one box surface image;

The third processing module is used for acquiring pixel displacement values when adjacent correction images in the box surface correction image sequence are spliced, and splicing a plurality of correction images into a container panoramic image based on the pixel displacement values to obtain a one-time spliced panoramic image;

the fourth processing module is used for acquiring the length-width ratio section of the primary spliced panoramic image and the length-width ratio section of the container panoramic image and judging whether the length-width ratio section of the primary spliced panoramic image is in the length-width ratio section of the container panoramic image or not;

a fifth processing module, configured to determine the primary stitched panoramic image as a final stitched container panoramic image if the aspect ratio of the primary stitched panoramic image is within the aspect ratio interval of the container panoramic image;

and a sixth processing module, configured to obtain a corrected pixel displacement value when adjacent corrected images in the sequence of case surface corrected images are stitched if the aspect ratio of the primary stitched panoramic image is not within the aspect ratio interval of the container panoramic image, stitch a plurality of corrected images into the container panoramic image based on the corrected pixel displacement value, obtain a secondary stitched panoramic image, and determine the secondary stitched panoramic image as a final stitched container panoramic image.