TWI653625B - Method and device for seeing through obstacles - Google Patents

Abstract

The invention discloses an obstacle perspective method. The obstacle perspective method includes the steps of: collecting at least two images partially obscured by the predetermined obstacle from at least two different viewing angles; deleting the portion of the at least two images that is obscured by the obstacle, generating at least two reservations An image; determining a stitching position of the at least two reserved images; and stitching the at least two reserved images. The present invention further discloses a corresponding obstacle fluoroscopy device. The obstacle fluoroscopy method and obstacle fluoroscopy apparatus disclosed by the present invention are intended to reduce or eliminate blind spots of vision caused by obstacles.

Description

Obstacle perspective method and obstacle perspective device

The present invention relates to an obstacle seeing method and an obstacle seeing device.

In the presence of an obstacle, it is often easy to form a blind spot of the visual field, and the observer or the photographing device cannot obtain the field of view of the blocking portion of the obstruction, so that a complete picture cannot be obtained. For example, the column between the windshield and the front door of the car plays a supporting role called the A-pillar. The A-pillar blocks the driver's partial line of sight during driving, thus causing a blind spot. The existence of the blind spot of the A-pillar is one of the important causes of traffic accidents. The inconvenience of the occlusion of the line of sight is especially obvious during the driver's turn or lane change. Therefore, the A-column is narrower and better, and the A-pillar supports the car. The weight of the part requires a certain degree of strength, thus causing a contradiction between the obstacle cannot be eliminated and the requirement of visual integrity.

Taiwan Patent Publication No. TWI267061 discloses a method for processing multi-layer image overlay, the method comprising the steps of: detecting whether a mask weight of a first picture material is within a predetermined range; and when the first picture When the mask weight of the data is within the predetermined range, a third screen data is generated according to the first screen data, a second screen data, and a mask weight of the second screen data. The disclosure of TWI267061 provides a method of superimposing a multi-layer picture in a translucent form, yet it has not been used to reduce or eliminate blind spots of vision caused by obstacles.

In view of this, it is necessary to provide an obstacle fluoroscopy method and an obstacle fluoroscopy device to reduce or eliminate blind spots of vision caused by obstacles.

An obstacle perspective method comprising the steps of: collecting at least two images partially obscured by a predetermined obstacle from at least two different viewing angles; deleting at least two portions of the at least two images that are obscured by the obstacle, generating at least two pre- An image is retained; a stitching position of the at least two reserved images is determined; and the at least two reserved images are stitched.

Preferably, of the at least two different viewing angles, the adjacent two viewing angles have overlapping regions.

Preferably, the step of determining the splicing position of the at least two reserved images comprises: determining the at least two images according to the at least two different viewing angles and preset sizes of the at least two images Position information; compares the position information of the adjacent image, and calculates the angle conversion amount and the translation transformation amount with reference to one of the images.

Preferably, the step of determining the splicing position of the at least two reserved images comprises: performing feature extraction on the adjacent reserved images; comparing the extracted features and performing image registration.

Preferably, the step of comparing the extracted features and performing image registration specifically comprises: determining a transformation relationship between the two images according to the position of the overlapping portion in the adjacent image, and finding a mapping between the images to be stitched. A transformation model of relationships; registration of multiple images using one or more registration methods.

Preferably, before the step of splicing the at least two reserved images, the method further comprises the steps of: restoring the other of the at least two reserved images with reference to one of the reserved images Make a size change.

Preferably, before the step of splicing the at least two reserved images, the method further comprises the steps of: restoring the other of the at least two reserved images with reference to one of the reserved images Perform resolution conversion.

Preferably, the step of splicing the at least two reserved images comprises: weighting and merging corresponding regions of adjacent reserved images, wherein the weighted fusion is implemented by a fusion algorithm of a primitive level.

An obstacle fluoroscopy device includes an image capturing device and a processing unit, the camera device includes at least two groups of cameras, the at least two groups of cameras have different orientations; and signal outputs of the at least two groups of cameras are connected to the processing After receiving the at least two sets of camera information of the at least two groups of cameras, the processing unit fuses the at least two sets of camera information to form panoramic information of a predetermined area.

Preferably, among the at least two sets of cameras, the viewing angles of the adjacent two sets of cameras have overlapping regions.

In the obstacle observing method and the obstacle fluoroscopy device, after at least two images partially obscured by the predetermined obstacle are collected from at least two different viewing angles, the portion of the at least two images that is blocked by the obstacle is deleted, Generating at least two reserved images and splicing the at least two reserved images, wherein at least two of the reserved images are not occluded by the obstacles, thereby obtaining a partial or complete Eliminating the image of the occlusion portion achieves the purpose of seeing the obstacle, thereby reducing or eliminating the blind spot of the visual field caused by the obstacle.

100A‧‧‧First obstacle see-through device

100B‧‧‧Second obstacle see-through device

10‧‧‧First camera

20‧‧‧second camera

30‧‧‧ obstacles

40‧‧‧ bullseye chart

50‧‧‧ third camera

60‧‧‧4th camera

101‧‧‧First preset boundary

102‧‧‧ second preset boundary

103‧‧‧First amended boundary

201‧‧‧ third preset boundary

202‧‧‧ fourth preset boundary

203‧‧‧ second revised boundary

A1‧‧‧ first field of vision

A2‧‧‧ second field of view

B1‧‧‧ third field of view

401‧‧‧First Reserved Image

402‧‧‧First occlusion image

403‧‧‧Second reserved image

404‧‧‧second occlusion image

405‧‧‧ fusion image

406‧‧‧Residual occlusion image

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is a schematic view showing the structure of an obstacle seeing device in a preferred embodiment.

FIG. 2 is a schematic view showing the working principle of the obstacle seeing device of FIG. 1. FIG.

Figure 3 is a schematic illustration of the operation of the obstacle see-through device in another preferred embodiment.

As shown in Figures 1 and 2, the obstacle see-through device 100 includes an imaging device (not shown) and a processing unit (not shown). The camera device includes at least two groups of cameras, and the orientations of the at least two groups of cameras are different. The first camera 10 and the second camera 20 in the obstacle fluoroscopy apparatus 100 will be described in the following embodiments.

The viewing angles of the first camera 10 and the second camera 20 have coincident regions. The signal outputs of the first camera 10 and the second camera 20 are connected to a processing unit, which may optionally use a microprocessor. After receiving the two sets of imaging information of the first camera 10 and the second camera 20, the processing unit fuses the two sets of imaging information to form panoramic information of a predetermined area. The panoramic information herein can be understood as the sum of the portions of the two sets of camera information that are not blocked by the obstacle 30, that is, an image that partially or completely eliminates the occlusion portion.

The operation of the obstacle fluoroscopy apparatus 100 will be described in detail below in conjunction with the obstacle fluoroscopy method.

For example, the camera device is intended to acquire a complete image of the bull's eye chart image 40. However, due to the presence of the obstacle 30, the fields of view of the first camera 10 and the second camera 20 are partially blocked by the obstacle 30, so that neither the first camera 10 nor the second camera 20 can completely acquire the complete image of the complete bull's eye chart image 40. image. At this time, since the viewing angles of the first camera 10 and the second camera 20 are different, and the viewing angles of the first camera 10 and the second camera 20 have overlapping regions, the two sets of imaging information of the first camera 10 and the second camera 20 can mutually Fusion.

The first image is acquired by the first camera 10 and the second image is acquired by the second camera 20.

As shown in the upper left part of FIG. 2, the theoretical field of view of the first camera 10 is defined by the first preset boundary 101 and the second preset boundary 102, and is stitched along the first preset boundary 101 and the second preset boundary 102. By. Due to the presence of the obstacle 30, the actual field of view of the first camera 10 is the intermediate area between the first preset boundary 101 and the first modified boundary 103; therefore, the first camera 10 can only collect the target image of the bullseye image 40. The first reserved image 401 is located between the first preset boundary 101 and the first modified boundary 103, and the bull's-eye chart image 40 is located outside the first preset boundary 101 and the first modified boundary 103. The portion is referred to as the first occlusion image 402.

Similarly, as shown in the upper right portion of FIG. 2, the theoretical field of view of the second camera 20 is defined by the third predetermined boundary 201 and the fourth predetermined boundary 202. Due to the presence of the obstacle 30, the actual field of view of the second camera 20 is the middle area of the fourth preset boundary 202 and the second correction boundary 203. Therefore, when the bull's-eye chart image 40 is acquired, only the bull's-eye chart image 40 is located. a second reserved image 403 between the four preset boundaries 202 and the second modified boundary 203, and a portion of the bull's eye chart image 40 that is outside the fourth preset boundary 202 and the second modified boundary 203 is recorded as a second occlusion map. Like 404.

Neither the first reserved image 401 nor the second reserved image 403 constitute a complete image of the bull's eye chart image 40. The first occlusion image 402 and the second occlusion image 404 form a blind spot of the first camera 10 and the second camera 20. In practical applications, it is necessary to reduce or eliminate blind spots of vision.

In order to achieve the purpose of reducing or eliminating the blind spot of the visual field, the first reserved image 401 and the second reserved image 403 may be superimposed by splicing along the first modified boundary 103 and the second modified boundary 203.

That is, the splicing position of the first reserved image 401 and the second reserved image 403 can be implemented by the following methods: according to the viewing angles of the first camera 10 and the second camera 20, and the first image and the second image. The preset size determines the position information of the first image and the second image, compares the position information of the first image and the second image, and calculates the image of the other image relative to the reference image by using one of the images as a reference. Angle transformation amount and translation transformation amount.

The splicing position of the first reserved image 401 and the second reserved image 403 may also be implemented by performing feature extraction on the first reserved image 401 and the second reserved image 403; The registration of the first reserved image 401 and the second reserved image 403 is performed. The step of aligning the extracted features and performing image registration may include: determining the first reserved image 401 and the second according to the positions of the overlapping portions in the first reserved image 401 and the second reserved image 403. Reserve the transformation relationship of image 403 to find out A transformation model of the mapping relationship between the first reserved image 401 and the second reserved image 403; and registration of multiple images by one or more registration methods. The method of image registration can be implemented by the method in the prior art. Since it is a mature prior art, it will not be described here.

After the splicing position of the first reserved image 401 and the second reserved image 403 is determined, the first reserved image 401 and the second reserved image 403 may be spliced by a processing unit, such as a microprocessor. . The splicing of the first reserved image 401 and the second reserved image 403 can be implemented by a method of weighted fusion. The weighted fusion can be implemented by a fusion algorithm at the primitive level. The fused image 405 obtained by fusing the first reserved image 401 and the second reserved image 403 is as shown in the lower part of FIG. In a specific implementation, due to the size of the obstacle 30, the boundary, and the angle of view of the first camera 10 and the second camera 20, the fused image 405 may be a complete image of the bull's-eye chart image 40, or there may be residual occlusion. Image 406, but ultimately can achieve the goal of reducing or eliminating blind spots of vision caused by obstacles 30.

The portion of the spliced residual occlusion image 406 can be processed in a variety of ways. One of the embodiments is to use a mirroring process to take a reserved image adjacent to the modified boundary as a pattern to generate a mirror image to fill in the residue. The occlusion image 406 is affixed; another embodiment is to attach a predetermined image, such as a preset good trademark, product image, other predetermined image, etc., to fill the residual occlusion image 406.

Before the step of splicing the first reserved image 401 and the second reserved image 403, the method further includes the step of: using the reserved image as a reference, the first reserved image 401 and the second reserved The other reserved images in the image 403 are size-converted. In addition, the first reserved image 401 and the other reserved images in the second reserved image 403 may be subjected to resolution conversion according to one of the reserved images as needed, so as to implement subsequent weighting. Fusion.

In the present embodiment, only the first camera 10 and the second camera 20 are taken as an example for description. It is easy to understand that in other embodiments, more than two cameras may be used to acquire images and perform fusion as needed.

As shown in FIG. 3, in another application scenario, there are multiple obstacle seeing devices. Taking two obstacle seeing devices as an example, a third camera 50 of a first obstacle seeing device 100A will be in the shooting when it is photographed. A second obstacle seeing device 100B appears in the theoretical field of view (ie, the first field of view A1 in FIG. 3), causing the third camera 50 to have an occlusion field of view (ie, the second field of view A2 in FIG. 3), but if the user does not It is desirable that the second obstacle fluoroscopy device 100B appears in the image captured by the third camera 50, and the partial image in the third field of view B1 captured by the fourth camera 60 of the second obstacle illuminating device 100B can be replaced. The image of the second field of view A2 is taken to obtain an image that is not obscured by the second obstacle viewing device 100B.

In the above obstacle observing method and obstacle obscuring device, after at least two images captured by at least two different angles of view and partially obscured by the obstacle, the at least two images are blocked by the obstacle And generating at least two reserved images and splicing the at least two reserved images, wherein at least two of the reserved images are not occluded by the obstacle 30, thereby obtaining a portion Or completely eliminating the image of the occluded portion, the purpose of seeing the obstacle 30 is achieved, thereby reducing or eliminating the blind spot of the visual field caused by the obstacle 30. Moreover, the two lenses may exist in the same obstacle seeing device or in different obstacle seeing devices depending on the embodiment.

The above is only the preferred embodiment of the present invention, and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection of the present invention. Within the scope.

Claims (10)

An obstacle perspective method comprising the steps of: collecting at least two images partially obscured by a predetermined obstacle from at least two different viewing angles; deleting at least two portions of the at least two images that are obscured by the obstacle, generating at least two pre- Determining a splicing position of the at least two reserved images, comprising: determining a position of the at least two images according to the at least two different viewing angles and preset sizes of the at least two images Information; comparing position information of adjacent images to determine a stitching position; stitching the at least two reserved images. The obstacle perspective method of claim 1, wherein of the at least two different viewing angles, the adjacent two viewing angles have overlapping regions. The obstacle perspective method of claim 1, wherein the determining the splicing position of the at least two reserved images further comprises calculating an angle conversion amount and a translation transformation amount with reference to one of the images. The obstacle fluoroscopy method of claim 1, wherein the determining the splicing position of the at least two reserved images comprises: performing feature extraction on the adjacent reserved images; comparing the extracted features Perform image registration. The obstacle perspective method of claim 1, wherein the step of comparing the extracted features and performing image registration specifically comprises: determining a transformation of two images according to positions of overlapping portions in adjacent images Relationship, find the transformation model of the mapping relationship between the images to be stitched; use one or more registration methods to achieve registration of multiple images. The obstacle perspective method of claim 1, wherein before the step of splicing the at least two reserved images, the method further comprises the step of: referring to one of the reserved images as a reference to the at least two The other reserved images in the reserved image are used for size conversion. The obstacle perspective method of claim 1 or 6, wherein before the step of splicing the at least two reserved images, the method further comprises the step of: referring to one of the reserved images, The other reserved images in the two reserved images are subjected to resolution conversion. The obstacle perspective method of claim 1, wherein the step of splicing the at least two reserved images comprises: weighting and merging corresponding regions of adjacent reserved images, the weighted fusion adopting a primitive level The fusion algorithm is implemented. An obstacle fluoroscopy device includes a camera device and a processing unit, wherein the camera device includes at least two groups of cameras, the at least two groups of cameras have different orientations; and the signal outputs of the at least two groups of cameras are connected to the camera Processing unit; after receiving the at least two sets of camera information of the at least two groups of cameras, the processing unit splicing and merging the at least two sets of camera information by using an obstacle perspective method according to claim 1 to form a predetermined area Panoramic information. The obstacle fluoroscopy device of claim 9, wherein, among the at least two groups of cameras, the viewing angles of the adjacent two groups of cameras have coincident regions.