CN110458753B - Adaptive segmentation and undistorted unfolding system and method for panoramic girdle image - Google Patents
Adaptive segmentation and undistorted unfolding system and method for panoramic girdle image Download PDFInfo
- Publication number
- CN110458753B CN110458753B CN201910740468.7A CN201910740468A CN110458753B CN 110458753 B CN110458753 B CN 110458753B CN 201910740468 A CN201910740468 A CN 201910740468A CN 110458753 B CN110458753 B CN 110458753B
- Authority
- CN
- China
- Prior art keywords
- image
- undistorted
- rectangular
- coordinates
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 22
- 230000011218 segmentation Effects 0.000 title claims abstract description 16
- 230000003044 adaptive effect Effects 0.000 title claims description 6
- 238000012545 processing Methods 0.000 claims abstract description 12
- 238000001514 detection method Methods 0.000 claims abstract description 11
- 238000013500 data storage Methods 0.000 claims abstract description 8
- 238000003709 image segmentation Methods 0.000 claims abstract description 4
- 238000013507 mapping Methods 0.000 claims description 9
- 230000003287 optical effect Effects 0.000 claims description 4
- 238000003384 imaging method Methods 0.000 claims description 3
- 238000007781 pre-processing Methods 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 8
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000002860 competitive effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000011895 specific detection Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
- G06T3/047—Fisheye or wide-angle transformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a self-adaptive segmentation and undistorted unfolding system and method for panoramic girdle images, wherein the system comprises a PAL camera, a data storage module and a data processing module, and the method comprises the following steps: acquiring PAL camera internal parameters; acquiring an annulus image by using a PAL camera; expanding the girdle image into a rectangular image by using a rectangular expansion algorithm; detecting pixel coordinates of a human face in a rectangular image by using a human face detection algorithm; calculating the pixel coordinates corresponding to the girdle image according to the pixel coordinates in the rectangular image; image segmentation is carried out by utilizing the obtained face pixel coordinates; and carrying out undistorted expansion on the image by utilizing the pixel positions of the image of the ring belt after internal reference and segmentation of the camera. The invention adopts the self-adaptive segmentation algorithm to intelligently identify the faces and segment the participants, and then removes distortion according to the projection relationship, so that each person of the participant can be intelligently displayed, and the problems of unsatisfactory display effect and overlarge distortion when the panoramic girdle camera is applied to the field of video conferences are solved.
Description
Technical Field
The invention relates to the field of panoramic images and video conferences, in particular to a self-adaptive segmentation and distortion-free unfolding system and method based on panoramic girdle images.
Background
With the continuous improvement of the computing capability of computer hardware and the continuous maturity of image processing technology, especially the advent of miniaturization, light weight, high computing capability and low power consumption computing hardware, many products are miniaturized and light weight. The panoramic girdle camera-based video conference system can realize 360-degree panoramic shooting only by a single lens, has a light structure when used as a video conference, occupies no calculation cost, is easy to realize real-time display, has low transmission cost, and is one of choices with high competitive power under the requirement of medium and low image quality.
In the conventional video conference scheme, if panoramic requirements exist, a multi-camera stitching technology is generally used, and the method needs to align and stitch a plurality of acquired images, has a complex structure, needs to perform synchronous alignment triggering, and has high hardware complexity.
Disclosure of Invention
The invention provides a panoramic annular image-based adaptive segmentation and undistorted unfolding system and method, which are mainly used for a video conference system, can adaptively detect the face position in an annular image, segment and extract the position of the face, correct the original annular image with distortion into a normal undistorted image, and are more suitable for human eyes to observe.
The technical scheme adopted by the invention is as follows:
a first object of the present invention is to provide an adaptive segmentation and distortion-free unfolding system for panoramic annular images, comprising:
the PAL camera is used for acquiring panoramic girdle images;
the data storage module is used for storing parameters calibrated by the PAL camera;
the data processing module is used for expanding the ring belt image into a rectangular image by utilizing a rectangular expansion algorithm, detecting the pixel coordinates of a human face in the rectangular image by utilizing a face detection algorithm, calculating the pixel coordinates corresponding to the ring belt image according to the pixel coordinates in the rectangular image, dividing the ring belt image into a plurality of required parts by utilizing the obtained pixel coordinates of all the human faces, and carrying out undistorted expansion on the image by utilizing the pixel positions of the ring belt image after internal reference and division of the camera, wherein the undistorted expansion comprises the following specific steps:
establishing a unit sphere model, making a tangent plane along the optical axis direction, positioning an undistorted image on the tangent plane, and determining the height of the undistorted unfolded image according to the vertical field angle of the lens;
determining the width of the undistorted unfolded image according to the width of the segmented image;
determining the position of the image to be unfolded without distortion according to the position of the segmented image;
and remapping the part of the girdle image needing undistorted expansion by utilizing the camera internal parameters, and projecting the part onto a tangent plane according to the height and the width to finish undistorted expansion.
The second object of the present invention is to provide a method for adaptively segmenting and distortionless unfolding a panoramic annular image, comprising the steps of:
the method comprises the steps of (1) obtaining PAL camera internal parameters by using a camera calibration algorithm, and storing calibrated data into a data storage module;
step (2), acquiring an annulus image currently being photographed by using a PAL camera;
expanding the girdle image into a rectangular image by using a rectangular expansion algorithm;
step (4) detecting the pixel coordinates of the human face in the rectangular image by using a human face detection algorithm;
step (5) calculating the pixel coordinates corresponding to the girdle image according to the pixel coordinates in the rectangular image;
step (6), image segmentation is carried out by utilizing the obtained face pixel coordinates, and the girdle image is segmented into a plurality of needed parts;
the method comprises the following specific steps of (7) performing undistorted unfolding on an image by utilizing pixel positions of an internal reference and segmented annular image of a camera:
establishing a unit sphere model, making a tangent plane along the optical axis direction, positioning an undistorted image on the tangent plane, and determining the height of the undistorted unfolded image according to the vertical field angle of the lens;
determining the width of the undistorted unfolded image according to the width of the segmented image;
determining the position of the image to be unfolded without distortion according to the position of the segmented image;
and remapping the part of the girdle image needing undistorted expansion by utilizing the camera internal parameters, and projecting the part onto a tangent plane according to the height and the width to finish undistorted expansion.
Further, in step (1), the following is specifically mentioned:
calibrating a PAL camera to be used by using an open-source OCamCalib toolbox, fitting an imaging model into a curved surface by using 4 Taylor series, and giving the central position of an image to obtain the mapping from the pixel coordinates of the image to the real world coordinates, wherein the mapping from the real world to the pixel coordinates is represented as the pixel coordinates of a return image, and the mapping from the pixel coordinates to the real world is represented as a vector; the corresponding functions in the toolbox are World2Cam and Cam2World.
Further, the step (2) specifically includes the following steps:
the PAL camera forms a ring-shaped ring belt image on the photosensitive chip, the central part of the image is a dead zone due to the unique structural characteristics, the left side and the right side of the ring belt image are irrelevant pixels, and all pixels in the ring are really and effectively utilized. When the PAL camera shoots, the PAL camera shoots horizontally upwards, the imaging is horizontally 360 degrees, the vertical view angle is determined by design, and the maximum vertical view angle is 90 degrees or more based on the vertical upwards of 0 degree.
Further, in the step (3), the Remap function used in OpenCV is needed to convert the ring graph into the rectangular graph, which is specifically as follows:
(3.1) acquiring the length and height of the target image, and recording as w and h;
(3.2) acquiring the pixel coordinates of the center of the original image, which are given in the calibration file and denoted as x c And y c ；
(3.3) marking a certain point on the target image as (i, j), wherein the connecting line of the point corresponding to the point on the original image and the center of the image forms an angle with the right upper part of the image, and is marked as theta;
(3.4) recording the inner diameter and the outer diameter of the annulus image as R respectively min And R is max ；
(3.5) calculating the correspondence between the pixels of the annulus image and the rectangular image as follows:
(x, y) is the pixel coordinates of the annulus image;
(3.6) writing the mapping relation into a cvRemap function, namely remapping the girdle image into a rectangular image.
Further, in the step (4), the occurrence of the face needs to be preprocessed and detected, and finally the face coordinates are obtained, which is as follows:
(4.1) preprocessing the image, converting the RGB image into a gray scale image, and then performing histogram equalization operation;
(4.2) transmitting the processed image in the step (4.1) to a HaarDetectObjects function to obtain a set of faces;
and (4.3) acquiring pixel coordinates of all the faces in the rectangular image.
Further, the step (5) specifically comprises the following steps:
setting a detected face image face 0 The center pixel coordinate of (i) is (i) 0 ，j 0 ) The corresponding angles in the annulus image are:and similarly, calculating the coordinates of the pixels at the edge of the face image.
Further, the step (6) specifically includes the following steps:
in two adjacent face images, setting the corresponding angle of the right edge of the first face image as theta 1 The corresponding angle of the left edge of the second face image is theta 2 If theta 2 -θ 1 ≤θ e Two faces are considered to exist on the same image without segmentation; if theta is 2 -θ 1 ≥θ e Consider that two faces should be displayed separately at θ 1 +θ e And theta 2 -θ e Each of which is divided, i.e. a common image region can appear, where θ e Is the minimum spacing angle of the two face images.
Further, the step (7) specifically includes the following steps:
let a certain segment be theta a ～θ b The opening angle of the image to be expanded with respect to the center of the image is theta b -θ a The expansion center of the expansion section corresponds to an angle (theta b -θ a )/2；
The World2Cam function maps three-dimensional coordinates in the real World to pixel coordinates of an image, a vertical plane is established in the real World, and the three-dimensional coordinates of a certain point on the plane are sent into the function, so that the color of the returned pixel point is the real color of the current plane point.
A sphere with radius of 1 is established by taking the camera as the sphere center, a rectangular surface tangent to the spherical surface is formed in the same direction as the lens, namely in the vertical upward direction, and the segment theta is aimed at a ～θ b Let the width W of the plane be 2 tan ((θ) b -θ a ) 2), the height H is tan (90 degrees to A degrees) plus tan (B degrees to 90 degrees) (B is more than or equal to 90 degrees), A degrees is the minimum vertical field angle, and B degrees is the maximum vertical field angle; the height of the rectangular surface is determined by the angle of view of the PAL lens and is a fixed value, and the maximum and minimum angles of view of the lens are always unchanged, so that the value of W can be changed only when the image is segmented; establishing a right-hand coordinate system by taking a connecting line of the tangent point and the sphere center as an x-axis and taking a connecting line of the lens forward direction and the sphere center as a z-axis, wherein the rectangular surface is perpendicular to the yOz plane and tangential to the spherical surface (1, 0), and the aspect ratio of the image after distortion removal can be set according to the aspect ratio of the rectangular surface; let the length of the undistorted image be W 0 High is H 0 Then the coordinates of each point on the rectangular surface areBecause the rectangular surface is not at the position of the segment theta a ～θ b Towards position, it is therefore necessary to rotate it in coordinates, the z-axis coordinates remain unchanged, and the x-y-axis coordinates are given by the following formula:
wherein θ r ＝(θ b -θ a )/2；
And sending the coordinates of each point on the rotated rectangular surface into a World2Cam function, so that the true color representation of each point can be obtained, and the obtained image is a distortion-free image.
Compared with the prior art, the invention has the following beneficial effects:
the invention adopts the self-adaptive segmentation algorithm to intelligently identify the faces and segment the participants, and then removes distortion according to the projection relationship, so that each person of the participant can be intelligently displayed, and the problems of unsatisfactory display effect and overlarge distortion when the panoramic girdle camera is applied to the field of video conferences are solved.
Drawings
FIG. 1 is a schematic diagram of an exemplary architecture of a system of the present invention;
FIG. 2 is a schematic view of a panoramic annular lens structure;
FIG. 3 is a flow chart of the method of the present invention;
FIG. 4 is an example of the invention used in the example of the re-projection into three-dimensional space after calibration;
FIG. 5 is a schematic view of a face taken by an example of the present invention;
FIG. 6 is a rectangular development view of the sample taken in FIG. 5;
FIG. 7 is a schematic diagram of the undistorted expansion range required for the sample taken in FIG. 5;
FIG. 8 is an effect diagram of undistorted expansion of a selected portion of the swatch taken in FIG. 5;
fig. 9 is a schematic diagram of a shooting effect under a multi-person condition;
FIG. 10 is an effect diagram of undistorted expansion of the portion of interest of FIG. 9;
in the drawings, the reference numerals and corresponding part names:
1-catadioptric lens 2-refractive lens group
3-photosensitive chip
Detailed Description
The present invention will be described in more detail with reference to examples and drawings for the purpose of making the objects, technical solutions and some of the present invention more clear, and the illustrative embodiments of the present invention and the descriptions thereof are only for explaining the present invention and are not limiting the present invention.
The example system comprises a PAL camera, a data storage module and a data processing module, the structure of the example system is shown in figure 1, the structure of the panoramic annular lens is shown in figure 2, the PAL camera is connected with the data processing module and is responsible for collecting the image of the current environment, and the circular annular image is transmitted to the data processing module; the data storage module is connected with the data processing module, is responsible for storing calibrated PAL lens parameters and provides the calibrated PAL lens parameters for the data processing module; the data processing module is responsible for specific implementation of algorithms such as unfolding an annular image, face detection, distortion-free unfolding and the like.
The flow of the adaptive segmentation and undistorted unfolding method of the panoramic girdle image is shown in fig. 3, before the system operates, an OCamCalib tool box is required to be used for calibrating a PAL camera, and the angle of view of the PAL lens used in the embodiment is 360 degrees horizontally and 30-90 degrees vertically. The calibration picture can be re-projected to the three-dimensional space after calibration, as shown in fig. 4, and the generated TXT file is stored in the data storage module.
When the system is running, the PAL camera module receives image data in real time and transmits the image data to the data processing module, and the data processing module receives one frame of image and then performs the following operations:
(1) Expanding the girdle image into a rectangular image by using a rectangular expansion algorithm;
(2) Calculating the coordinates of face pixels in the rectangular image by using a face detection algorithm;
(3) Calculating the pixel coordinates corresponding to the ring belt graph according to the face pixel coordinates in the rectangular image;
(4) Image segmentation is carried out by utilizing the obtained face pixel coordinates, and the girdle image is segmented into a plurality of needed parts;
(5) The image is unfolded in a distortion-free way by utilizing the pixel positions of the internal reference and the segmented girdle image of the camera;
(6) Repeating the above steps.
In the operation of (1): as shown in fig. 5, the image captured by the example system needs to use the Remap function in OpenCV to convert the ring-band diagram into a rectangular diagram, specifically:
(1.1) acquiring the length and height of a target image, wherein the PAL lens used in the example is 2/3", the matched CMOS size is 2/3",2448×2048 pixels are adopted, and then the width w=2448 and the height h=2048 of the image are adopted;
(1.2) acquiring the pixel coordinates of the center of the original image, which is given in the calibration document, for this example, x c ＝1022.72，y c ＝1223.16；
(1.3) recording a certain point on the target image as (i, j), wherein the angle corresponding to the point on the original image is theta;
(1.4) obtaining the inner diameter and the outer diameter of the annular band image in the calibration file as R respectively min ＝360，R max ＝1020；
(1.5) calculating the correspondence between the pixels of the annulus image and the rectangular image as follows:
(x, y) is the pixel coordinates of the annulus image;
(1.6) calling the cvRemap function, writing the mapping relation into the function, and obtaining a rectangular image, namely 1318×208 in this example
As shown in fig. 6.
In the operation of (2): before face detection, the image needs to be preprocessed, the Cvtcolor in the OpenCV is used for converting the RGB image into a gray scale image, then the EqualizeHist function in the OpenCV is used for carrying out histogram equalization operation, and then the HaarDetectObjects function is used for carrying out Haar feature detection, so that the face is extracted.
Because the face detection is only a means for assisting in dividing the image, which is not original, the invention replaces the face on the real conference with the schematic diagram, and the specific detection method is not repeated.
After the face detection is successful, the pixel coordinates of the face part are returned, in this example, the center point of the face is (650, 87), and the edge points are (608, 85) and (689, 89).
In the operation of (3): the opening angle corresponding to the face center is calculated as follows:
the angles corresponding to the edge points are 166 degrees and 188 degrees.
In the operation of (4): the angle corresponding to the center point of the image is 177 degrees, and theta is taken e At this time, only one face is present on the image, and the angle corresponding to the ring belt image is 141 ° to 213 °, so the angle of the required segmentation is 72 °, as shown in fig. 7.
In the operation of (5): for the present example, the vertical field angle of the PAL lens used is 30 ° to 90 °, and substituting the ring belt image angle 141 ° to 213 ° in (4) into 2×tan ((213 ° -141 °)/2) ≡1.45) is calculated, the corresponding image width is 1.45, and the image height is calculated according to the vertical field angle: therefore, the aspect ratio of the displayed image is 1.45:1.73, namely the true and correct image, and the display effect of the image fed into the World2Cam function is shown in figure 8.
In the case of multiple persons, similar to a single person, the deployment is shown in fig. 9 after no distortion. At this time, the included angle between three persons is not more than 25 degrees, and the three persons are unfolded in the same image as shown in fig. 10.
The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the invention, and is not meant to limit the scope of the invention, but to limit the invention to the particular embodiments, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (8)
1. An adaptive segmentation and undistorted unfolding system for panoramic annular images, comprising:
the PAL camera is used for acquiring panoramic girdle images;
the data storage module is used for storing parameters calibrated by the PAL camera;
the data processing module is used for expanding the ring belt image into a rectangular image by utilizing a rectangular expansion algorithm, detecting the pixel coordinates of a human face in the rectangular image by utilizing a face detection algorithm, calculating the pixel coordinates corresponding to the ring belt image according to the pixel coordinates in the rectangular image, dividing the ring belt image into a plurality of required parts by utilizing the obtained pixel coordinates of all the human faces, and carrying out undistorted expansion on the image by utilizing the pixel positions of the ring belt image after internal reference and division of the camera, wherein the undistorted expansion comprises the following specific steps:
establishing a unit sphere model, making a tangent plane along the optical axis direction, positioning an undistorted image on the tangent plane, and determining the height of the undistorted unfolded image according to the vertical field angle of the lens;
determining the width of the undistorted unfolded image according to the width of the segmented image;
determining the position of the image to be unfolded without distortion according to the position of the segmented image;
remapping a part of the girdle image needing undistorted expansion by utilizing the camera internal parameter, and projecting the part onto a tangent plane according to the height and the width to finish the undistorted expansion;
let a certain segment be theta a ～θ b The opening angle of the image to be expanded with respect to the center of the image is theta b -θ a The expansion center of the expansion section corresponds to an angle (θ b -θ a )/2；
A sphere with radius of 1 is established by taking the camera as the sphere center, a rectangular surface tangent to the spherical surface is formed in the same direction as the lens, namely in the vertical upward direction, and the segment theta is aimed at a ～θ b Let the width W of the plane be 2 tan ((θ) b -θ a ) 2), the height H is tan (90 degrees to A degrees) plus tan (B degrees to 90 degrees) (B is more than or equal to 90 degrees), A degrees is the minimum vertical field angle, and B degrees is the maximum vertical field angle; the height of the rectangular surface is determined by the field angle of the PAL lens and is a fixed value because the maximum and minimum field angles of the lens beginThe value of W is only changed when the segmentation image is unchanged; establishing a right-hand coordinate system by taking a connecting line of the tangent point and the sphere center as an x-axis and taking a connecting line of the lens forward direction and the sphere center as a z-axis, wherein the rectangular surface is perpendicular to the yOz plane and tangential to the spherical surface (1, 0), and the aspect ratio of the image after distortion removal can be set according to the aspect ratio of the rectangular surface; let the length of the undistorted image be W 0 High is H 0 Then the coordinates of each point on the rectangular surface areBecause the rectangular surface is not at the position of the segment theta a ～θ b Towards position, it is therefore necessary to rotate it in coordinates, the z-axis coordinates remain unchanged, and the x-y-axis coordinates are given by the following formula:
wherein θ r ＝(θ b -θ a )/2；
And sending the coordinates of each point on the rotated rectangular surface into a World2Cam function, so that the true color representation of each point can be obtained, and the obtained image is a distortion-free image.
2. The self-adaptive segmentation and undistorted unfolding method of the panoramic girdle image is characterized by comprising the following steps of:
the method comprises the steps of (1) obtaining PAL camera internal parameters by using a camera calibration algorithm, and storing calibrated data into a data storage module;
step (2), acquiring an annulus image currently being photographed by using a PAL camera;
expanding the girdle image into a rectangular image by using a rectangular expansion algorithm;
step (4) detecting the pixel coordinates of the human face in the rectangular image by using a human face detection algorithm;
step (5) calculating the pixel coordinates corresponding to the girdle image according to the pixel coordinates in the rectangular image;
step (6), image segmentation is carried out by utilizing the obtained face pixel coordinates, and the girdle image is segmented into a plurality of needed parts;
the method comprises the following specific steps of (7) performing undistorted unfolding on an image by utilizing pixel positions of an internal reference and segmented annular image of a camera:
establishing a unit sphere model, making a tangent plane along the optical axis direction, positioning an undistorted image on the tangent plane, and determining the height of the undistorted unfolded image according to the vertical field angle of the lens;
determining the width of the undistorted unfolded image according to the width of the segmented image;
determining the position of the image to be unfolded without distortion according to the position of the segmented image;
remapping a part of the girdle image needing undistorted expansion by utilizing the camera internal parameter, and projecting the part onto a tangent plane according to the height and the width to finish the undistorted expansion;
let a certain segment be theta a ～θ b The opening angle of the image to be expanded with respect to the center of the image is theta b -θ a The expansion center of the expansion section corresponds to an angle (θ b -θ a )/2；
A sphere with radius of 1 is established by taking the camera as the sphere center, a rectangular surface tangent to the spherical surface is formed in the same direction as the lens, namely in the vertical upward direction, and the segment theta is aimed at a ～θ b Let the width W of the plane be 2 tan ((θ) b -θ a ) 2), the height H is tan (90 degrees to A degrees) plus tan (B degrees to 90 degrees) (B is more than or equal to 90 degrees), A degrees is the minimum vertical field angle, and B degrees is the maximum vertical field angle; the height of the rectangular surface is determined by the angle of view of the PAL lens and is a fixed value, and the maximum and minimum angles of view of the lens are always unchanged, so that the value of W can be changed only when the image is segmented; establishing a right-hand coordinate system by taking a connecting line of the tangent point and the sphere center as an x-axis and taking a connecting line of the lens forward direction and the sphere center as a z-axis, wherein the rectangular surface is perpendicular to the yOz plane and tangential to the spherical surface (1, 0), and the aspect ratio of the image after distortion removal can be set according to the aspect ratio of the rectangular surface; let the length of the undistorted image be W 0 High is H 0 Then the coordinates of each point on the rectangular surface areBecause the rectangular surface is not at the position of the segment theta a ～θ b Towards position, it is therefore necessary to rotate it in coordinates, the z-axis coordinates remain unchanged, and the x-y-axis coordinates are given by the following formula:
wherein θ r ＝(θ b -θ a )/2；
And sending the coordinates of each point on the rotated rectangular surface into a World2Cam function, so that the true color representation of each point can be obtained, and the obtained image is a distortion-free image.
3. The method according to claim 2, characterized in that in step (1) it is specified as follows:
calibrating a PAL camera to be used by using an open-source OCamCalib toolbox to obtain a mapping from image pixel coordinates to real world coordinates, wherein the mapping from the real world to the pixel coordinates is represented as pixel coordinates of a return image, and the mapping from the pixel coordinates to the real world is represented as a vector; the corresponding functions in the toolbox are World2Cam and Cam2World.
4. The method according to claim 2, wherein the step (2) is specifically as follows:
when the PAL camera shoots, the PAL camera shoots horizontally upwards, the imaging is horizontally 360 degrees, the maximum vertical field angle is more than or equal to 90 degrees based on the condition that the vertical upwards is 0 degree.
5. The method according to claim 2, wherein the step (3) is specifically as follows:
(3.1) acquiring the length and height of the target image, and recording as w and h;
(3.2) acquiring the pixel coordinates of the center of the original image, which is in the calibration fileGiven, denoted as x c And y c ；
(3.3) marking a certain point on the target image as (i, j), wherein the connecting line of the point corresponding to the point on the original image and the center of the image forms an angle with the right upper part of the image, and is marked as theta;
(3.4) recording the inner diameter and the outer diameter of the annulus image as R respectively min And R is max ；
(3.5) calculating the correspondence between the pixels of the annulus image and the rectangular image as follows:
(x, y) is the pixel coordinates of the annulus image;
(3.6) writing the mapping relation into the cvRemap function, namely remapping the girdle image into a rectangular image.
6. The method according to claim 2, wherein the step (4) is specifically as follows:
(4.1) preprocessing the image, converting the RGB image into a gray scale image, and then performing histogram equalization operation;
(4.2) transmitting the processed image in the step (4.1) to a HaarDetectObjects function to obtain a set of faces;
and (4.3) acquiring pixel coordinates of all the faces in the rectangular image.
7. The method according to claim 2, wherein the step (5) is specifically as follows:
setting a detected face image face 0 The center pixel coordinate of (i) is (i) 0 ，j 0 ) The corresponding angles in the annulus image are:and similarly, calculating the coordinates of the pixels at the edge of the face image.
8. The method according to claim 2, wherein the step (6) is specifically as follows:
in two adjacent face images, setting the corresponding angle of the right edge of the first face image as theta 1 The corresponding angle of the left edge of the second face image is theta 2 If theta 2 -θ 1 ≤θ e Two faces are considered to exist on the same image without segmentation; if theta is 2 -θ 1 ≥θ e Consider that two faces should be displayed separately at θ 1 +θ e And theta 2 -θ e Each of which is divided, i.e. a common image region can appear, where θ e Is the minimum spacing angle of the two face images.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910740468.7A CN110458753B (en) | 2019-08-12 | 2019-08-12 | Adaptive segmentation and undistorted unfolding system and method for panoramic girdle image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910740468.7A CN110458753B (en) | 2019-08-12 | 2019-08-12 | Adaptive segmentation and undistorted unfolding system and method for panoramic girdle image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110458753A CN110458753A (en) | 2019-11-15 |
| CN110458753B true CN110458753B (en) | 2023-08-11 |
Family
ID=68485898
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910740468.7A Active CN110458753B (en) | 2019-08-12 | 2019-08-12 | Adaptive segmentation and undistorted unfolding system and method for panoramic girdle image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110458753B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111504331B (en) * | 2020-04-29 | 2021-09-14 | 杭州环峻科技有限公司 | Method and device for positioning panoramic intelligent vehicle from coarse to fine |
| CN112858328B (en) * | 2020-12-31 | 2022-11-08 | 青岛大学 | Method, device, equipment and storage medium for detecting copper exposure area of electric connector |
| CN113570751A (en) * | 2021-07-16 | 2021-10-29 | 杭州环峻科技有限公司 | Vehicle event data recorder system of panorama clitellum camera |
| CN116958164A (en) * | 2022-04-08 | 2023-10-27 | 影石创新科技股份有限公司 | Image processing method, device, electronic equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010217984A (en) * | 2009-03-13 | 2010-09-30 | Tottori Univ | Image detector and image detection method |
| CN109615663A (en) * | 2018-12-12 | 2019-04-12 | 深圳市至高通信技术发展有限公司 | Panoramic video bearing calibration and terminal |
-
2019
- 2019-08-12 CN CN201910740468.7A patent/CN110458753B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010217984A (en) * | 2009-03-13 | 2010-09-30 | Tottori Univ | Image detector and image detection method |
| CN109615663A (en) * | 2018-12-12 | 2019-04-12 | 深圳市至高通信技术发展有限公司 | Panoramic video bearing calibration and terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110458753A (en) | 2019-11-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110458753B (en) | Adaptive segmentation and undistorted unfolding system and method for panoramic girdle image | |
| CN108600576B (en) | Image processing apparatus, method and system, and computer-readable recording medium | |
| US9325899B1 (en) | Image capturing device and digital zooming method thereof | |
| JP6467787B2 (en) | Image processing system, imaging apparatus, image processing method, and program | |
| JP5450739B2 (en) | Image processing apparatus and image display apparatus | |
| US20060120712A1 (en) | Method and apparatus for processing image | |
| US10841485B2 (en) | Photographing method and photographing apparatus for terminal, and terminal | |
| CN105141841B (en) | Picture pick-up device and its method | |
| JP2010088105A (en) | Imaging apparatus and method, and program | |
| JP2017208619A (en) | Image processing apparatus, image processing method, program and imaging system | |
| JP2007336411A (en) | Imaging apparatus, auto-bracketing photographing method, and program | |
| JP2009123081A (en) | Face detection method and photographing apparatus | |
| US20200364832A1 (en) | Photographing method and apparatus | |
| TW201342303A (en) | Three-dimensional image obtaining system and three-dimensional image obtaining method | |
| US7683935B2 (en) | Imaging device | |
| CN109218602B (en) | Image acquisition device, image processing method and electronic device | |
| US7835552B2 (en) | Image capturing apparatus and face area extraction method | |
| JP6696596B2 (en) | Image processing system, imaging device, image processing method and program | |
| JPH1118007A (en) | Omnidirectional image display system | |
| JPH08305841A (en) | Distorted image correcting display device | |
| TWI824321B (en) | Image controller, image processing system and image modifying method | |
| CN210670250U (en) | Camera device, camera and terminal | |
| US20230007182A1 (en) | Image controller, image processing system and image correcting method | |
| JP2001266152A (en) | Face image recognizing device and method | |
| CN115293971B (en) | Image splicing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |