CN112184541A - Image splicing method, device and equipment and storage medium - Google Patents
Image splicing method, device and equipment and storage medium Download PDFInfo
- Publication number
- CN112184541A CN112184541A CN201910604336.1A CN201910604336A CN112184541A CN 112184541 A CN112184541 A CN 112184541A CN 201910604336 A CN201910604336 A CN 201910604336A CN 112184541 A CN112184541 A CN 112184541A
- Authority
- CN
- China
- Prior art keywords
- target
- seam
- point
- splicing
- position information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000010586 diagram Methods 0.000 claims description 15
- 230000008859 change Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000036544 posture Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Classifications
-
- G06T3/12—
-
- G06T3/08—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/32—Indexing scheme for image data processing or generation, in general involving image mosaicing
Abstract
The invention provides an image splicing method, an image splicing device, image splicing equipment and a storage medium, wherein the method comprises the following steps: when a target object has intersection with a splicing position in a preset panoramic splicing map, taking a point of the target object distributed on a target side after the target object is deviated along a position far away from the splicing position as a reference point, wherein the target side is one of two sides of the splicing position; determining a target seam position based on the position information of the reference point and the seam position information; and splicing the two sub-graphs of the target object at the position of the target splicing seam, so that the target object is distributed in one sub-graph of the spliced graph. The target object can be bypassed by the abutted seams, and the problem that the target object generates dislocation or ghost images is avoided.
Description
Technical Field
The present invention relates to the field of image technologies, and in particular, to an image stitching method, an image stitching device, an image stitching apparatus, and a storage medium.
Background
In some occasions such as vehicle driving occasions, sub-images collected from different directions of a vehicle need to be spliced, and compared with the sub-images, the obtained images have a larger field of view and a more complete scene. For example, sub-images collected from all directions around the vehicle can be spliced to obtain a panoramic image showing an omnidirectional scene.
In the related splicing mode, images are spliced by adopting a strategy of fixing the splicing seams, namely, no matter how the target objects are distributed in the global splicing map, the splicing seam positions in the global splicing map are fixed, and the related two sub-images are always spliced at the splicing positions.
However, when the relative postures of two adjacent cameras change (for example, when two cameras are installed on a vehicle and a vehicle body deforms by a certain amount due to an external force factor, the relative postures of two adjacent cameras change), the parallax of the two cameras changes, when a target object is distributed on two sides of a splicing position in a global splicing diagram, sub-images collected by the two cameras are spliced by adopting the splicing fixing strategy, and the target object is dislocated during splicing, and is distributed on two sides of the splicing diagram, so that the dislocation or ghost images are generated, and the visual experience is influenced.
Disclosure of Invention
In view of this, the present invention provides an image stitching method, an image stitching device, an image stitching apparatus, and a storage medium, which can make a seam bypass a target object and avoid the problem of dislocation or ghost of the target object.
The first aspect of the present invention provides an image stitching method, including:
when a target object has intersection with a splicing position in a preset panoramic splicing map, taking a point of the target object distributed on a target side after the target object is deviated along a position far away from the splicing position as a reference point, wherein the target side is one of two sides of the splicing position;
determining a target seam position based on the position information of the reference point and the seam position information;
and splicing the two sub-graphs of the target object at the position of the target splicing seam, so that the target object is distributed in one sub-graph of the spliced graph.
According to one embodiment of the invention, the reference points comprise at least points which are at a distance from the seam position greater than or equal to the distance from the pole to the seam position;
wherein the pole is a point which is farthest away from the seam splicing position in all points of the target object distributed on the target side.
In accordance with one embodiment of the present invention,
the reference point further includes a point of the target object distributed along the target object edge among all points of the target side different from the pole.
According to one embodiment of the present invention, determining a target seam position based on position information of a reference point and seam position information includes:
determining the position information of a target point at the target side according to the position information of the reference point, wherein the distance between the target point and the seam splicing position is greater than the distance between the reference point and the seam splicing position;
and determining the target seam splicing position according to the position information of the target point and the position information of the starting point and the stopping point of the seam splicing position.
According to an embodiment of the present invention, determining the position information of the target point at the target side according to the position information of the reference point includes:
checking whether the reference point is between the seam position and a specified boundary, wherein the specified boundary is the boundary of the splicing range positioned on the target side;
if so, determining the position information of one point between the reference point and the target boundary point on the specified boundary as the position information of the target point, wherein the target boundary point is the point which is the closest to the specified boundary from the reference point.
According to an embodiment of the present invention, determining a target seam position according to the position information of the target point and the position information of the start point and the end point of the seam position includes:
performing curve fitting on the target point and the start and stop points of the seam splicing positions according to the position information of the target point and the position information of the start and stop points of the seam splicing positions to obtain a fitting curve;
and determining the position of the fitted curve as the target seam splicing position.
According to an embodiment of the present invention, before the point at which the target object is distributed on the target side is shifted along the far-from-stitch position as the reference point, the method further includes:
calculating the distance between each point of the target object distributed in the global splicing diagram and the position of the splicing seam;
determining a first side where the farthest points of the two sides of the seam position are located, wherein the farthest points are the points with the largest distance from each point of the target object to the seam position;
and taking the other side of the first side in the two sides of the seam splicing position as the target side of the seam splicing.
According to an embodiment of the invention, the method further comprises:
acquiring the acquired position data of the target object;
determining a position information set corresponding to the position data of the target object in the global mosaic, wherein the position information set comprises position information of all points of the target object distributed in the global mosaic;
and checking whether an intersection exists between the position information set and the seam position in the global splicing map.
A second aspect of the present invention is an image stitching device, including:
the reference point determining module is used for taking a point of a target object distributed on a target side after the point deviates along a position far away from a splicing position as a reference point when the target object has an intersection with the splicing position in a preset panoramic splicing map, wherein the target side is one of two sides of the splicing position;
the target seam position determining module is used for determining the position of the target seam based on the position information of the reference point and the seam position information;
and the sub-graph splicing module is used for splicing the two sub-graphs of the target object at the position of the target splicing seam, so that the target object is distributed in one sub-graph of the spliced graph.
According to one embodiment of the invention, the reference points comprise at least points which are at a distance from the seam position greater than or equal to the distance from the pole to the seam position;
wherein the pole is a point which is farthest away from the seam splicing position in all points of the target object distributed on the target side.
In accordance with one embodiment of the present invention,
the reference point further includes a point of the target object distributed along the target object edge among all points of the target side different from the pole.
According to an embodiment of the present invention, the target seam position determining module is specifically configured to, when determining the target seam position based on the position information of the reference point and the seam position information:
determining the position information of a target point at the target side according to the position information of the reference point, wherein the distance between the target point and the seam splicing position is greater than the distance between the reference point and the seam splicing position;
and determining the target seam splicing position according to the position information of the target point and the position information of the starting point and the stopping point of the seam splicing position.
According to an embodiment of the present invention, when the target seam position determining module determines the position information of the target point on the target side according to the position information of the reference point, the target seam position determining module is specifically configured to:
checking whether the reference point is between the seam position and a specified boundary, wherein the specified boundary is the boundary of the splicing range positioned on the target side;
if so, determining the position information of one point between the reference point and the target boundary point on the specified boundary as the position information of the target point, wherein the target boundary point is the point which is the closest to the specified boundary from the reference point.
According to an embodiment of the present invention, when the target seam position determining module determines the target seam position according to the position information of the target point and the position information of the start point and the end point of the seam position, the target seam position determining module is specifically configured to:
performing curve fitting on the target point and the start and stop points of the seam splicing positions according to the position information of the target point and the position information of the start and stop points of the seam splicing positions to obtain a fitting curve;
and determining the position of the fitted curve as the target seam splicing position.
According to an embodiment of the present invention, before the point at which the target object is distributed on the target side is shifted along the far-from-stitch position as the reference point, the reference point determining module is further configured to:
calculating the distance between each point of the target object distributed in the global splicing diagram and the position of the splicing seam;
determining a first side where the farthest points of the two sides of the seam position are located, wherein the farthest points are the points with the largest distance from each point of the target object to the seam position;
and taking the other side of the first side in the two sides of the seam splicing position as the target side of the seam splicing.
According to an embodiment of the invention, the apparatus further comprises:
the position data acquisition module is used for acquiring the acquired position data of the target object;
a position information set determining module, configured to determine, in the global mosaic, a position information set corresponding to position data of the target object, where the position information set includes position information of all points of the target object distributed in the global mosaic;
and the checking module is used for checking whether an intersection exists between the position information set and the position of the joint in the global splicing map.
A third aspect of the invention provides an electronic device comprising a processor and a memory; the memory stores a program that can be called by the processor; wherein, when the processor executes the program, the image stitching method as described in the foregoing embodiments is implemented.
A fourth aspect of the present invention provides a machine-readable storage medium on which a program is stored, which when executed by a processor, implements the image stitching method as described in the foregoing embodiments.
The embodiment of the invention has the following beneficial effects:
in the embodiment of the invention, when the target object and the splicing position in the preset panoramic splicing map have intersection, the point of the target object distributed on the target side after the point is deviated along the position far away from the splicing position is taken as a reference point, the splicing position required by splicing two sub-images of the target object, namely the target splicing position, can be determined again based on the position information of the reference point and the splicing position information, the two sub-images of the target object are spliced at the target splicing position, so that the target object is distributed in one sub-image of the spliced map, the same target object is prevented from being distributed on two sides of the splicing in the spliced map, namely the splicing in the spliced map bypasses the target object, the dislocation or the ghost of the target object is avoided, and the visual effect is improved.
Drawings
FIG. 1 is a schematic flow chart of an image stitching method according to an embodiment of the present invention;
FIG. 2 is a block diagram of an image stitching apparatus according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of two-dimensional images spliced by a fixed seam splicing strategy;
FIG. 4 is a schematic diagram of two-dimensional images spliced according to an embodiment of the invention;
FIG. 5 is a schematic diagram of three-dimensional images spliced by a fixed-seam strategy;
FIG. 6 is a schematic diagram of three-dimensional images spliced according to an embodiment of the invention;
fig. 7 is a block diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the invention, as detailed in the appended claims.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It will be understood that, although the terms first, second, third, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one type of device from another. For example, a first device may also be referred to as a second device, and similarly, a second device may also be referred to as a first device, without departing from the scope of the present invention. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
The image splicing method provided by the embodiment of the invention can be applied to auxiliary driving occasions, such as a vehicle-mounted all-around system, the vehicle-mounted all-around system can comprise a plurality of camera devices arranged around a vehicle, each camera device acquires sub-images of different local scenes, a processor in the vehicle-mounted all-around system splices the acquired sub-images, and people on the vehicle can see the panorama around the vehicle through the spliced image so as to assist driving. Of course, there is an intersection of scenes in the fields of view of adjacent image pickup apparatuses.
Of course, the image stitching method according to the embodiment of the present invention may also be applied to other occasions where image stitching is required, and the driving assistance occasions are only examples and are not limitations on the application occasions of the image stitching method according to the present invention.
The image stitching method according to the embodiment of the present invention is described in more detail below, but should not be limited thereto. In one embodiment, referring to fig. 1, an image stitching method according to an embodiment of the present invention is shown, including:
s100: when a target object has intersection with a splicing position in a preset panoramic splicing map, taking a point of the target object distributed on a target side after the target object is deviated along a position far away from the splicing position as a reference point, wherein the target side is one of two sides of the splicing position;
s200: determining a target seam position based on the position information of the reference point and the seam position information;
s300: and splicing the two sub-graphs of the target object at the position of the target splicing seam, so that the target object is distributed in one sub-graph of the spliced graph.
The execution main body of the image splicing method is the electronic equipment. The electronic equipment can be computer equipment, embedded equipment and the like, can be connected with the camera equipment, and receives images collected by the camera equipment as subgraphs to be spliced. The electronic device may also be a camera device, and images acquired by itself and/or acquired from other camera devices are used as sub-images to be stitched. The specific type of electronic device is not limited as long as it has image processing capability.
In the method for obtaining the subgraph, a plurality of camera devices can be installed on a target object at different installation angles, and image acquisition is carried out from a plurality of directions (such as front, back, left, right, upper, lower and the like) around the target object to obtain the subgraph of a local scene in each direction. It is understood that the acquisition and splicing of the subgraphs may be performed continuously at a certain frequency.
For example, the target object is a vehicle, four image capturing devices are arranged on the vehicle in a surrounding manner, the four image capturing devices respectively capture images from the left, front, right and rear directions of the vehicle at the same time, the captured sub-images respectively comprise a vehicle left side scene, a vehicle front scene, a vehicle right side scene and a vehicle rear scene, and the sub-images are spliced to obtain a panoramic image showing the surrounding scenes of the vehicle.
The field of view of the adjacent camera devices has a certain coincidence, and if the target object is in the coincidence space of the field of view of the adjacent camera devices, the target object is (partially or completely) distributed in the sub-images acquired by the adjacent camera devices.
In this case, if the fixed seam strategy is adopted for splicing, a situation that a seam in a spliced graph passes through a target object may occur, that is, the target object may be distributed on two sides of the seam in the spliced graph, and when a relative posture of adjacent camera devices changes due to an external force action of a vehicle, the target object may be dislocated or ghosted on two sides of the seam in the spliced graph.
In step S100, when a target object intersects with a seam position in a preset panoramic stitching map, a point at which the target object is distributed on a target side, which is one of two sides of the seam position, is deviated along a direction away from the seam position and serves as a reference point.
Before splicing each time, whether a target object has an intersection with a splicing position in a preset panoramic splicing map can be checked, and if not, two sub-images needing to be spliced can be spliced at the splicing position; if so, it is stated that the target object is distributed in the two sub-graphs to be stitched, if stitching at the position of the seam may cause the target object to be distributed in the two sub-graphs of the stitched graph, i.e. the seam of the stitched graph may pass through the target object. Therefore, in the embodiment of the invention, when the two sub-graphs are spliced, a new splicing position is determined. The target object may be a pedestrian, a vehicle, a lane line, or the like, and is not limited in particular.
In the embodiment of the invention, the point and the seam position of the target object can be in the same coordinate system, such as the coordinate system applied by the panoramic spliced image, and correspondingly, the determined reference point and the target seam position information can also be in the coordinate system.
The reference point may be determined from the target side of the patchwork, the point at which the target object is distributed on the target side being offset along the position away from the patchwork as the reference point. The offset points on the target object may be points on the edge of the target object, and the number may be one or more than two, and correspondingly, the number of the reference points may also be one or more than two. The target side of the seam is one of two sides of the seam position, and the target side can be specified or determined according to a preset rule without limitation.
In step S200, a target seam position is determined based on the position information of the reference point and the seam position information.
The reference point and the seam position information are known, a line can be determined according to the known reference point and the known seam position information, and the position of the line is used as a target seam position. For example, the line may be defined by the start point and the end point of the seam as two end points and passing through the reference point, and the specific determination method is not limited.
Because the reference point is a point after the target object is distributed on the target side and the point is deviated along the direction far away from the seam position, the reference point farthest from the seam position and all the points before the target object is deviated are definitely not intersected, so the target seam position without intersection with all the points before the target object is deviated can be determined according to the position information of the reference point and the seam position information, and the seam in the seam map can bypass the target object when two sub-maps are spliced at the target seam position.
In step S300, the two sub-graphs of the target object are spliced at the target splicing position, so that the target object is distributed in one sub-graph of the spliced graph.
The target object and the joint position in the preset panoramic spliced graph have intersection, which shows that the target object enters the view field of two adjacent camera devices, the target object is distributed in the subgraphs acquired by the two camera devices, the two subgraphs of the target object are spliced at the target joint position, and the target object can be distributed in one subgraph of the spliced graph obtained by splicing.
The panoramic mosaic can be formed by splicing two or more subgraphs, and the steps of the embodiment of the invention are suitable for any two subgraphs to be spliced, which are distributed with the same target object.
Since there may be differences in luminance, chrominance, etc. between sub-images collected by different image capturing devices, there may be corresponding differences between two sub-images of a mosaic, so that a significant mosaic trace may appear. The splicing map can be processed by adopting modes of image fusion, color balance and the like so as to reduce splicing traces and enable two sub-maps of the splicing map to be in smooth transition. In addition, the driving trajectory line of the vehicle and the virtual radar obstacle plate (indicating the situation of the obstacle ahead) may be drawn in the mosaic image and displayed together.
With the acquisition and splicing of the sub-images, the panoramic spliced image is updated at a certain frequency, and the updated panoramic spliced image can be displayed on the vehicle in real time. The electronic equipment can transmit the panoramic mosaic to display equipment such as vehicle-mounted display equipment and a mobile phone display screen for displaying; alternatively, the panorama mosaic may be displayed on the electronic device if the display device is present on the electronic device.
In the embodiment of the invention, when the target object and the splicing position in the preset panoramic splicing map have intersection, the point of the target object distributed on the target side after the point is deviated along the position far away from the splicing position is taken as a reference point, the splicing position required by splicing two sub-images of the target object, namely the target splicing position, can be determined again based on the position information of the reference point and the splicing position information, the two sub-images of the target object are spliced at the target splicing position, so that the target object is distributed in one sub-image of the spliced map, the same target object is prevented from being distributed on two sides of the splicing in the spliced map, namely the splicing in the spliced map bypasses the target object, the dislocation or the ghost of the target object is avoided, and the visual effect is improved.
In one embodiment, the above method flow can be executed by the image stitching apparatus 100, as shown in fig. 2, the image stitching apparatus 100 mainly includes 3 modules: a reference point determining module 101, a target seam position determining module 102 and a sub-graph splicing module 103. The reference point determining module 101 is configured to perform the step S100, the target seam position determining module 102 is configured to perform the step S200, and the sub-graph stitching module 103 is configured to perform the step S300.
In one embodiment, the reference points include at least points that are at a distance from the seam location that is greater than or equal to the distance of the pole from the seam location;
wherein the pole is a point which is farthest away from the seam splicing position in all points of the target object distributed on the target side.
In other words, in this embodiment, only the pole of the target object may be shifted along the position away from the seam to obtain the corresponding reference point, and since the pole is the point farthest away from the seam position among all the points of the target object distributed on the target side, the obtained reference point is farther away from the seam position, and it is ensured that the determined target seam position can avoid all the points before the target object is shifted.
Optionally, the reference point further includes a point different from the pole, among all points of the target object distribution on the target side, along the target object edge.
In other words, the reference points may include all points of the target object distributed on all points of the target side along the edge of the target object, that is, a series of reference points obtained by offsetting the contour of the target object on the target side along the position far from the seam, so as to further ensure that the determined target seam position can avoid all points of the target object before offset.
In this embodiment, the reference points depict the contour of the target object in the target side, so that the corresponding target seam position determined according to the reference points can also surround the contour of the target object, and the target seam position is prevented from exceeding the splicing range.
In one embodiment, the step S200 of determining the target seam position based on the position information of the reference point and the seam position information may include the steps of:
s201: determining the position information of a target point at the target side according to the position information of the reference point, wherein the distance between the target point and the seam splicing position is greater than the distance between the reference point and the seam splicing position;
s202: and determining the target seam splicing position according to the position information of the target point and the position information of the starting point and the stopping point of the seam splicing position.
And determining the position information of the target point according to the position information of the reference point, wherein the reference point and the target point are both positioned at the target side, the distance between the target point and the seam position is greater than the distance between the reference point and the seam position, and the target point does not exceed the splicing range for splicing the two sub-images.
The target point and the reference point may correspond to each other, and when there are a plurality of reference points, there are a plurality of target points. The target point and the start and stop points of the seam position are known, and the target seam position can be determined according to the points, for example, the position of a connecting line of the points can be used as the target seam position, or the position of a line obtained by fitting the points can be used as the target seam position, and the specific mode is not limited.
The differences between the embodiments of the present invention and stitching with a fixed-stitch strategy are described below with reference to FIGS. 3-6.
Fig. 3 shows two-dimensional images spliced by a fixed seam strategy in the prior art, the seam positions are fixed and are always the positions of lines a1, B1, C1 and D1 shown in fig. 3, a1 is a straight line connecting M and N, if a target object S intersects with the seam positions in a preset panoramic spliced graph, the target object S is still spliced to obtain the panoramic spliced graph formed by sub-graphs of LEFT1, FRONT1, RIGHT1 and real 1, and the target object S is distributed on two sides of a seam a 1.
Fig. 4 shows two-dimensional images spliced in the embodiment of the present invention, where the position of the seam is adjustable according to the distribution of the target object, for example, the positions of lines a2, B2, C2, and D2 in fig. 4, and taking a2 as an example, when the target object S intersects with the seam position (the position of the straight line connecting M and N) in the preset panoramic spliced graph, the target object S is no longer spliced at the position of the straight line connecting M and N, but spliced at the position of a curve a2 adjusted according to the target object S, so as to obtain the panoramic spliced graph composed of subfigures of LEFT2, FRONT2, RIGHT2, and real 2, and the target object S is distributed only on one side of the seam a2 in the panoramic spliced graph, thereby ensuring that the target object S is completely represented in one subfigure.
Fig. 5 shows a three-dimensional image spliced by a fixed seam strategy in the prior art, the position of the seam is fixed and is always the positions of lines A3, B3, C3, and D3 shown in fig. 5, A3 is an arc line connected by M 'and N', if a target object S intersects with the seam position in a preset panoramic spliced graph, the target object S is still spliced to obtain the panoramic spliced graph formed by subfigures such as LEFT3, FRONT3, RIGHT3, and real 3, and the target object S is distributed on both sides of the seam A3.
Fig. 6 shows a three-dimensional image spliced in the embodiment of the present invention, where the position of the seam is adjustable according to the distribution of the target objects, such as the positions of lines a4, B4, C4, and D4 in fig. 4, for example, a4, when the target object S intersects with the seam position (the position of the arc connected by M 'and N') in the preset panoramic spliced image, the target object S is no longer spliced at the position of the arc connected by M 'and N', but spliced at the position of an irregular curve a4 that bypasses the target object S and takes M 'and N' as starting points, so as to obtain the panoramic spliced image composed of sub-images of LEFT4, FRONT4, RIGHT4, and real 4, and the target object S is only distributed on one side of the seam a4 in the panoramic spliced image, thereby ensuring that the target object S is completely represented in one sub-image.
The image stitching method of this embodiment, which is described below with reference to fig. 4, may include the following steps:
firstly, determining a reference point: taking the horizontal direction as the X axis and the vertical direction as the Y axis in FIG. 4 as an example, S1 and S2 are two points on the contour of the target object S, S1 is represented by (X1 ', Y1'), S2 is represented by (X2 ', Y2'), the maximum coordinate value of all points on the target object S on the X axis is X2 ', and the maximum coordinate value on the Y axis is Y1', and P (X2 ', Y1') is constructed, which can be used as a reference point required by the present embodiment (i.e., a point whose distance from the seam position is greater than or equal to the distance between the pole and the seam position);
next, the target point is determined: obtaining a farther P1(x ', y') from the line connecting M and N with P as a reference, where P1 can be used as a target point required by the present embodiment;
then, determining the position of the target seam: the position information of the start and stop points of the edge joint is M (x)0，y0)、N(x1，y1) Can be obtained by pairing M (x)0，y0)、N(x1，y1) Carrying out curve fitting on P1(x ', y') to obtain a target seam splicing position, namely the position of a line A2;
next, the two sub-graphs of the target object are stitched at a2, the stitch in the stitched graph bypassing the target object S.
In the above manner, only one reference point is used, but other reference points may be used, such as points along the edge of the target object, which are different from the poles, among all points where the target object is distributed on the target side.
As shown in fig. 6, the position of the seam in the global mosaic is, for example, an arc line connected by M 'and N', and the reference point may be a point around an edge of the target object, among all points on the target object S located on the target side of the straight line connected by M 'and N', and accordingly, a target point around the edge may be determined; with (x)i，yi，zi) To representThe position information of the reference point, i ranges from 1 to N, N is the number of the reference points, (x)i，yi，zi) The position information of the corresponding target point is (x'i，y′i，zi) And performing curve fitting on the M ', the N' and the target points to obtain a target seam position of the seam A4, wherein the seam A4 bypasses the target object S.
It is understood that the three-dimensional image or the two-dimensional image can be merged by the above method, which is only an example and not a limitation.
In one embodiment, the step S201 of determining the position information of the target point on the target side according to the position information of the reference point includes:
s2011: checking whether the reference point is between the seam position and a specified boundary, wherein the specified boundary is the boundary of the splicing range positioned on the target side;
s2012: if so, determining the position information of one point between the reference point and the target boundary point on the specified boundary as the position information of the target point, wherein the target boundary point is the point which is the closest to the specified boundary from the reference point.
And checking whether the reference point is positioned between the seam position and the specified boundary, if so, indicating that the reference point is positioned in the splicing range, determining a target point from the splicing range, and further determining a new seam position as a target seam position.
When determining the position information of the target point, the position information of one point between the reference point and the target boundary point on the designated boundary may be determined as the position information of the target point. The one point may be a point that is a specified distance from the reference point, and of course, may be any point, and is not limited specifically.
Therefore, the target point can be ensured to be in the splicing range, the final target splicing position can be in the splicing range, and normal splicing is realized.
When the number of the reference points is greater than or equal to two, the steps S2011 and S2012 can be performed for each reference point, the position information of one target point can be correspondingly determined according to the position information of each reference point, and when curve fitting is performed by using a plurality of target points, the fitting effect is better.
Of course, if the reference point is not located between the seam position and the specified boundary, it indicates that the reference point is not located within the spliceable range, and at this time, the two sub-images may still be spliced at the seam position.
In one embodiment, in step S202, determining a target seam position according to the position information of the target point and the position information of the start and stop points of the seam position includes the following steps:
s2021: performing curve fitting on the target point and the start and stop points of the seam splicing positions according to the position information of the target point and the position information of the start and stop points of the seam splicing positions to obtain a fitting curve;
s2022: and determining the position of the fitted curve as the target seam splicing position.
The number of the target points can be one or more than two, curve fitting is carried out on the target points and the starting and stopping points of the seams, the obtained fitting curve can bypass the target object, and the position of the fitting curve is determined as the position of the target seams.
The curve fitting can be realized by adopting algorithms such as a least square method, a B spline fitting method and the like, and the specific algorithm is not limited as long as the corresponding curve can be fitted according to a plurality of points.
In one embodiment, in step S100, before the point where the target object is distributed on the target side is shifted along the far-from-stitch position as the reference point, the method further includes:
s110: calculating the distance between each point of the target object distributed in the global splicing diagram and the position of the splicing seam;
s120: determining a first side where the farthest points of the two sides of the seam position are located, wherein the farthest points are the points with the largest distance from each point of the target object to the seam position;
s130: and taking the other side of the first side in the two sides of the seam splicing position as the target side of the seam splicing.
For convenience of explanation, portions of the global mosaic where the target objects are distributed on both sides of the mosaic are referred to as a first local object and a second local object, respectively.
If the point on the first local object farthest from the seam position is farther from the seam than the point on the second local object farthest from the seam position, that is, the farthest point is the point on the first local object, and the side where the first local object is located is the first side, it means that the first local object should be closer to the boundary of the joinable range than the second local object, so the side where the second local object is located is taken as the target side, and the target seam position is determined in the target side, so that the target seam position can be prevented from exceeding the joinable range as much as possible.
In one embodiment, the method further comprises:
s010: acquiring the acquired position data of the target object;
s020: determining a position information set corresponding to the position data of the target object in the global mosaic, wherein the position information set comprises position information of all points of the target object distributed in the global mosaic;
s030: and checking whether an intersection exists between the position information set and the seam position in the global splicing map.
In step S010, the position data of the target object may be acquired by sensing devices such as a camera device, a radar sensor, and a laser sensor, and the position data of the target object may represent relative positions of the target object and the sensing devices, and since the target object is a three-dimensional object, relative positions of each point on the target object and the sensing devices are different, and the position data acquired by the sensing devices may represent the different relative positions. The electronic equipment can be connected with the sensing equipment through the data transmission line and receives the position data transmitted by the sensing equipment, and the number of the sensing equipment connected with the electronic equipment can be more than or equal to 1.
In step S020, a position information set corresponding to the position data of the target object is determined in the global mosaic, where the position information set includes position information of all points of the target object distributed in the global mosaic.
Taking sensing equipment as the camera equipment as an example, a depth image containing a target object can be acquired, position data of the target object in a coordinate system applied by the camera equipment is determined according to the depth data in the image, and the position data is converted into the coordinate system applied by the global mosaic, so that position information of all points of the target object in the global mosaic can be obtained.
Of course, the correspondence between the position data and the position information is not limited thereto, and may be determined according to the actual situation of the sensing device.
In step S030, it is checked whether an intersection exists between the position information set and the position of the seam in the global mosaic.
If there is an intersection between the position information set and the position information of all the points on the seams in the global mosaic, it is indicated that the seams will pass through the target object, that is, it is determined that the target object has an intersection with the seam positions in the preset panoramic mosaic, step S100 may be executed.
Whether intersections exist between the position information set and the position information of all the points on the seams in the global mosaic can be checked in the following two ways:
first, traversing the position information of all points at the position of the joint in the global splicing map, searching the traversed position information in the position information set, and if the traversed position information is found, indicating that an intersection exists, ending the traversal. If the traversal is finished and the search is not found, the intersection does not exist.
And secondly, traversing the position information in the position information set, searching the traversed position information in the position information of all points at the position of the splicing seam in the global splicing map, and if the traversed position information is found, indicating that an intersection exists, and finishing the traversal. If the traversal is finished and the search is not found, the intersection does not exist.
The present invention also provides an image stitching apparatus, and referring to fig. 2, the image stitching apparatus 100 includes:
a reference point determining module 101, configured to, when a target object intersects a seam position in a preset panoramic seam map, use a point at which the target object is distributed on a target side and which is offset along a direction away from the seam position as a reference point, where the target side is one of two sides of the seam position;
a target seam position determining module 102, configured to determine a target seam position based on the position information of the reference point and the seam position information;
and the sub-graph splicing module 103 is used for splicing the two sub-graphs of the target object at the position of the target splicing seam, so that the target object is distributed in one sub-graph of the spliced graph.
In one embodiment, the reference points include at least points that are at a distance from the seam location that is greater than or equal to the distance of the pole from the seam location;
wherein the pole is a point which is farthest away from the seam splicing position in all points of the target object distributed on the target side.
In one embodiment, the reference point further comprises a point along the target object edge different from the pole, among all points where the target object is distributed on the target side.
In one embodiment, the target seam position determining module is specifically configured to, when determining the target seam position based on the position information of the reference point and the seam position information:
determining the position information of a target point at the target side according to the position information of the reference point, wherein the distance between the target point and the seam splicing position is greater than the distance between the reference point and the seam splicing position;
and determining the target seam splicing position according to the position information of the target point and the position information of the starting point and the stopping point of the seam splicing position.
In one embodiment, when the target seam position determining module determines the position information of the target point on the target side according to the position information of the reference point, the target seam position determining module is specifically configured to:
checking whether the reference point is between the seam position and a specified boundary, wherein the specified boundary is the boundary of the splicing range positioned on the target side;
if so, determining the position information of one point between the reference point and the target boundary point on the specified boundary as the position information of the target point, wherein the target boundary point is the point which is the closest to the specified boundary from the reference point.
In an embodiment, when the target seam position determining module determines the target seam position according to the position information of the target point and the position information of the start point and the stop point of the seam position, the target seam position determining module is specifically configured to:
performing curve fitting on the target point and the start and stop points of the seam splicing positions according to the position information of the target point and the position information of the start and stop points of the seam splicing positions to obtain a fitting curve;
and determining the position of the fitted curve as the target seam splicing position.
In one embodiment, the reference point determining module is further configured to, before taking as the reference point a point at which the target object is distributed on the target side after the point is shifted along the far-from-stitch position:
calculating the distance between each point of the target object distributed in the global splicing diagram and the position of the splicing seam;
determining a first side where the farthest points of the two sides of the seam position are located, wherein the farthest points are the points with the largest distance from each point of the target object to the seam position;
and taking the other side of the first side in the two sides of the seam splicing position as the target side of the seam splicing.
In one embodiment, the apparatus further comprises:
the position data acquisition module is used for acquiring the acquired position data of the target object;
a position information set determining module, configured to determine, in the global mosaic, a position information set corresponding to position data of the target object, where the position information set includes position information of all points of the target object distributed in the global mosaic;
and the checking module is used for checking whether an intersection exists between the position information set and the position of the joint in the global splicing map.
The implementation process of the functions and actions of each unit in the above device is specifically described in the implementation process of the corresponding step in the above method, and is not described herein again.
For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, wherein the units described as separate parts may or may not be physically separate, and the parts shown as units may or may not be physical units.
The invention also provides an electronic device, which comprises a processor and a memory; the memory stores a program that can be called by the processor; wherein, when the processor executes the program, the image stitching method as described in the foregoing embodiments is implemented.
The embodiment of the image splicing device can be applied to electronic equipment. Taking a software implementation as an example, as a logical device, the device is formed by reading, by a processor of the electronic device where the device is located, a corresponding computer program instruction in the nonvolatile memory into the memory for operation. From a hardware aspect, as shown in fig. 7, fig. 7 is a hardware structure diagram of an electronic device where the image stitching apparatus 100 is located according to an exemplary embodiment of the present invention, and except for the processor 510, the memory 530, the interface 520, and the nonvolatile memory 540 shown in fig. 7, the electronic device where the apparatus 100 is located in the embodiment may also include other hardware according to an actual function of the electronic device, which is not described again.
The present invention also provides a machine-readable storage medium having stored thereon a program which, when executed by a processor, implements the image stitching method as set forth in any one of the preceding embodiments.
The present invention may take the form of a computer program product embodied on one or more storage media including, but not limited to, disk storage, CD-ROM, optical storage, and the like, having program code embodied therein. Machine-readable storage media include both permanent and non-permanent, removable and non-removable media, and the storage of information may be accomplished by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of machine-readable storage media include, but are not limited to: phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technologies, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic tape storage or other magnetic storage devices, or any other non-transmission medium, may be used to store information that may be accessed by a computing device.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (18)
1. An image stitching method, comprising:
when a target object has intersection with a splicing position in a preset panoramic splicing map, taking a point of the target object distributed on a target side after the target object is deviated along a position far away from the splicing position as a reference point, wherein the target side is one of two sides of the splicing position;
determining a target seam position based on the position information of the reference point and the seam position information;
and splicing the two sub-graphs of the target object at the position of the target splicing seam, so that the target object is distributed in one sub-graph of the spliced graph.
2. The image stitching method of claim 1, wherein the reference points include at least a point that is at a distance from the seam location that is greater than or equal to a distance from a pole to the seam location;
wherein the pole is a point which is farthest away from the seam splicing position in all points of the target object distributed on the target side.
3. The image stitching method of claim 2,
the reference point further includes a point of the target object distributed along the target object edge among all points of the target side different from the pole.
4. The image stitching method of claim 2 or 3, wherein determining the target seam position based on the position information of the reference point and the seam position information comprises:
determining the position information of a target point at the target side according to the position information of the reference point, wherein the distance between the target point and the seam splicing position is greater than the distance between the reference point and the seam splicing position;
and determining the target seam splicing position according to the position information of the target point and the position information of the starting point and the stopping point of the seam splicing position.
5. The image stitching method according to claim 4, wherein determining the position information of the target point on the target side based on the position information of the reference point comprises:
checking whether the reference point is between the seam position and a specified boundary, wherein the specified boundary is the boundary of the splicing range positioned on the target side;
if so, determining the position information of one point between the reference point and the target boundary point on the specified boundary as the position information of the target point, wherein the target boundary point is the point which is the closest to the specified boundary from the reference point.
6. The image stitching method of claim 4, wherein determining a target seam position according to the position information of the target point and the position information of the start and stop points of the seam position comprises:
performing curve fitting on the target point and the start and stop points of the seam splicing positions according to the position information of the target point and the position information of the start and stop points of the seam splicing positions to obtain a fitting curve;
and determining the position of the fitted curve as the target seam splicing position.
7. The image stitching method according to claim 1, wherein, before the point at which the target object is distributed on the target side is shifted along the position away from the stitch position as a reference point, the method further comprises:
calculating the distance between each point of the target object distributed in the global splicing diagram and the position of the splicing seam;
determining a first side where the farthest points of the two sides of the seam position are located, wherein the farthest points are the points with the largest distance from each point of the target object to the seam position;
and taking the other side of the first side in the two sides of the seam splicing position as the target side of the seam splicing.
8. The image stitching method of claim 1, further comprising:
acquiring the acquired position data of the target object;
determining a position information set corresponding to the position data of the target object in the global mosaic, wherein the position information set comprises position information of all points of the target object distributed in the global mosaic;
and checking whether an intersection exists between the position information set and the seam position in the global splicing map.
9. An image stitching device, comprising:
the reference point determining module is used for taking a point of a target object distributed on a target side after the point deviates along a position far away from a splicing position as a reference point when the target object has an intersection with the splicing position in a preset panoramic splicing map, wherein the target side is one of two sides of the splicing position;
the target seam position determining module is used for determining the position of the target seam based on the position information of the reference point and the seam position information;
and the sub-graph splicing module is used for splicing the two sub-graphs of the target object at the position of the target splicing seam, so that the target object is distributed in one sub-graph of the spliced graph.
10. The image stitching device of claim 9, wherein the reference points include at least a point that is at a distance from the seam location that is greater than or equal to a distance from a pole to the seam location;
wherein the pole is a point which is farthest away from the seam splicing position in all points of the target object distributed on the target side.
11. The image stitching device of claim 10,
the reference point further includes a point of the target object distributed along the target object edge among all points of the target side different from the pole.
12. The image stitching device according to claim 10 or 11, wherein the target seam position determining module, when determining the target seam position based on the position information of the reference point and the seam position information, is specifically configured to:
determining the position information of a target point at the target side according to the position information of the reference point, wherein the distance between the target point and the seam splicing position is greater than the distance between the reference point and the seam splicing position;
and determining the target seam splicing position according to the position information of the target point and the position information of the starting point and the stopping point of the seam splicing position.
13. The image stitching device according to claim 12, wherein the target stitching position determining module, when determining the position information of the target point at the target side according to the position information of the reference point, is specifically configured to:
checking whether the reference point is between the seam position and a specified boundary, wherein the specified boundary is the boundary of the splicing range positioned on the target side;
if so, determining the position information of one point between the reference point and the target boundary point on the specified boundary as the position information of the target point, wherein the target boundary point is the point which is the closest to the specified boundary from the reference point.
14. The image stitching device according to claim 12, wherein the target seam position determining module, when determining the target seam position according to the position information of the target point and the position information of the start point and the stop point of the seam position, is specifically configured to:
performing curve fitting on the target point and the start and stop points of the seam splicing positions according to the position information of the target point and the position information of the start and stop points of the seam splicing positions to obtain a fitting curve;
and determining the position of the fitted curve as the target seam splicing position.
15. The image stitching apparatus according to claim 9, wherein the reference point determining module is further configured to, before a point at which the target object is distributed on the target side shifted along the far-from-stitch position is taken as the reference point:
calculating the distance between each point of the target object distributed in the global splicing diagram and the position of the splicing seam;
determining a first side where the farthest points of the two sides of the seam position are located, wherein the farthest points are the points with the largest distance from each point of the target object to the seam position;
and taking the other side of the first side in the two sides of the seam splicing position as the target side of the seam splicing.
16. The image stitching device of claim 9, further comprising:
the position data acquisition module is used for acquiring the acquired position data of the target object;
a position information set determining module, configured to determine, in the global mosaic, a position information set corresponding to position data of the target object, where the position information set includes position information of all points of the target object distributed in the global mosaic;
and the checking module is used for checking whether an intersection exists between the position information set and the position of the joint in the global splicing map.
17. An electronic device comprising a processor and a memory; the memory stores a program that can be called by the processor; wherein the processor, when executing the program, implements the image stitching method according to any one of claims 1 to 8.
18. A machine-readable storage medium, having stored thereon a program which, when executed by a processor, implements the image stitching method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910604336.1A CN112184541A (en) | 2019-07-05 | 2019-07-05 | Image splicing method, device and equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910604336.1A CN112184541A (en) | 2019-07-05 | 2019-07-05 | Image splicing method, device and equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112184541A true CN112184541A (en) | 2021-01-05 |
Family
ID=73915765
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910604336.1A Pending CN112184541A (en) | 2019-07-05 | 2019-07-05 | Image splicing method, device and equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112184541A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090022422A1 (en) * | 2007-07-18 | 2009-01-22 | Samsung Electronics Co., Ltd. | Method for constructing a composite image |
| CN101859433A (en) * | 2009-04-10 | 2010-10-13 | 日电（中国）有限公司 | Image mosaic device and method |
| CN106981048A (en) * | 2017-03-31 | 2017-07-25 | 联想(北京)有限公司 | A kind of image processing method and device |
| CN107038686A (en) * | 2017-04-21 | 2017-08-11 | 成都观界创宇科技有限公司 | A kind of method and apparatus of image mosaic processing |
| US20180061006A1 (en) * | 2016-08-26 | 2018-03-01 | Multimedia Image Solution Limited | Method for ensuring perfect stitching of a subject's images in a real-site image stitching operation |
| CN108074216A (en) * | 2016-11-14 | 2018-05-25 | 北京航天长峰科技工业集团有限公司 | Video-splicing method based on manual planning splicing seams algorithm |
| US20180253875A1 (en) * | 2017-03-02 | 2018-09-06 | Qualcomm Incorporated | Systems and methods for content-adaptive image stitching |
| CN109146782A (en) * | 2018-07-23 | 2019-01-04 | 深圳积木易搭科技技术有限公司 | A kind of full-view image joining method and system |
-
2019
- 2019-07-05 CN CN201910604336.1A patent/CN112184541A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20090022422A1 (en) * | 2007-07-18 | 2009-01-22 | Samsung Electronics Co., Ltd. | Method for constructing a composite image |
| CN101859433A (en) * | 2009-04-10 | 2010-10-13 | 日电（中国）有限公司 | Image mosaic device and method |
| US20180061006A1 (en) * | 2016-08-26 | 2018-03-01 | Multimedia Image Solution Limited | Method for ensuring perfect stitching of a subject's images in a real-site image stitching operation |
| CN108074216A (en) * | 2016-11-14 | 2018-05-25 | 北京航天长峰科技工业集团有限公司 | Video-splicing method based on manual planning splicing seams algorithm |
| US20180253875A1 (en) * | 2017-03-02 | 2018-09-06 | Qualcomm Incorporated | Systems and methods for content-adaptive image stitching |
| CN106981048A (en) * | 2017-03-31 | 2017-07-25 | 联想(北京)有限公司 | A kind of image processing method and device |
| CN107038686A (en) * | 2017-04-21 | 2017-08-11 | 成都观界创宇科技有限公司 | A kind of method and apparatus of image mosaic processing |
| CN109146782A (en) * | 2018-07-23 | 2019-01-04 | 深圳积木易搭科技技术有限公司 | A kind of full-view image joining method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11102405B2 (en) | Rear-stitched view panorama for rear-view visualization | |
| CN113554698B (en) | Vehicle pose information generation method and device, electronic equipment and storage medium | |
| JP5208203B2 (en) | Blind spot display device | |
| JP6197388B2 (en) | Distance measuring device, distance measuring method, and program | |
| US11748907B2 (en) | Object pose estimation in visual data | |
| CN106331527B (en) | A kind of image split-joint method and device | |
| CN104539925B (en) | The method and system of three-dimensional scenic augmented reality based on depth information | |
| JP6176541B2 (en) | Information display device, information display method, and program | |
| US20070009137A1 (en) | Image generation apparatus, image generation method and image generation program | |
| US11783443B2 (en) | Extraction of standardized images from a single view or multi-view capture | |
| JP2009017462A (en) | Driving support system and vehicle | |
| CN102792333A (en) | Image processing device, method, program and storage medium | |
| JP2011215063A (en) | Camera attitude parameter estimation device | |
| CN111462503A (en) | Vehicle speed measuring method and device and computer readable storage medium | |
| JP6820561B2 (en) | Image processing device, display device, navigation system, image processing method and program | |
| JP2003044996A (en) | Obstacle detecting device | |
| CN110689476A (en) | Panoramic image splicing method and device, readable storage medium and electronic equipment | |
| JP5405809B2 (en) | Vehicle periphery monitoring device and camera position / posture determination method | |
| CN107209930A (en) | Look around image stability method and device | |
| JP2004265396A (en) | Image forming system and image forming method | |
| JP6802999B2 (en) | Compartment line detection system | |
| Kinzig et al. | Real-time seamless image stitching in autonomous driving | |
| KR102003387B1 (en) | Method for detecting and locating traffic participants using bird's-eye view image, computer-readerble recording medium storing traffic participants detecting and locating program | |
| JP2020135866A (en) | Object detection method, detection device and electronic apparatus | |
| CN112184541A (en) | Image splicing method, device and equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |