CN116757928A - Panoramic image processing method, panoramic image processing system, electronic equipment and storage medium - Google Patents
Panoramic image processing method, panoramic image processing system, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN116757928A CN116757928A CN202310637167.8A CN202310637167A CN116757928A CN 116757928 A CN116757928 A CN 116757928A CN 202310637167 A CN202310637167 A CN 202310637167A CN 116757928 A CN116757928 A CN 116757928A
- Authority
- CN
- China
- Prior art keywords
- image
- pinhole
- camera
- panoramic image
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 15
- 238000012545 processing Methods 0.000 title claims description 14
- 238000003860 storage Methods 0.000 title claims description 8
- 238000000034 method Methods 0.000 claims abstract description 32
- 230000003287 optical effect Effects 0.000 claims abstract description 28
- 238000010606 normalization Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 238000007781 pre-processing Methods 0.000 claims description 8
- 238000013507 mapping Methods 0.000 claims description 6
- 238000005516 engineering process Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 14
- 238000004422 calculation algorithm Methods 0.000 description 5
- 230000000007 visual effect Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 101100459256 Cyprinus carpio myca gene Proteins 0.000 description 2
- 238000005520 cutting process Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 241000251468 Actinopterygii Species 0.000 description 1
- 238000012935 Averaging Methods 0.000 description 1
- 108010001267 Protein Subunits Proteins 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Processing (AREA)
Abstract
The application relates to a panoramic image processing method, wherein the method comprises the following steps: determining the resolution of a pinhole image to be generated and camera model parameters corresponding to the pinhole image in the panoramic image; determining a reference plane corresponding to the initial pinhole image according to the camera sphere model and the resolution of the panoramic image; determining an optical axis offset between a camera model of the target pinhole image and a camera model of the initial pinhole image according to the camera model parameters; a target pinhole image is generated based on the reference plane and the camera optical axis offset. The application solves the problem of poor efficiency of the method for generating the pinhole image according to the panoramic image in the related technology, and can automatically calculate the number and the position of the detachable pinhole images of one panoramic image, the internal parameters and external parameters corresponding to the camera model, recover the pinhole images of any angle and the like, so that the mode of converting the panoramic image into the pinhole image is more flexible and efficient.
Description
Technical Field
The present application relates to the field of image processing, and in particular, to a panoramic image processing method, a panoramic image processing system, an electronic device, and a storage medium.
Background
Pinhole images have a deeper depth of field and less distortion than panoramic images, and therefore vision algorithms typically select as input pinhole images with simpler camera models and less distortion. However, because the observable range of the pinhole camera is limited, in order to meet the requirement of a visual algorithm with a large visual field, multiple-angle repeated acquisition is often required to be performed on the same scene, and the acquisition steps are complex; in addition, a smaller field of view of the pinhole image may reduce the proportion of effective information, and may also result in a decrease in algorithm stability.
In the related art, a partial region of a panoramic image may be converted into a pinhole image through a camera spherical model, but the current manner of converting the panoramic image into a pinhole image can only support a specific region, and a pinhole image of a plurality of arbitrary regions in succession cannot be generated.
At present, no effective solution has been proposed for the problem of poor efficiency of a method for generating a pinhole image from a panoramic image.
Disclosure of Invention
The embodiment of the application provides a scenery image processing method, a scenery image processing system, electronic equipment and a storage medium, which at least solve the problem of poor efficiency of a method for generating a pinhole image according to a panoramic image in the related art.
In a first aspect, an embodiment of the present application provides a panoramic image processing method, including:
determining the resolution of a pinhole image to be generated and camera model parameters corresponding to the pinhole image based on the panoramic image;
determining a reference plane corresponding to an initial pinhole image according to a camera sphere model of the panoramic image and the resolution;
determining an optical axis offset between a camera model of a target pinhole image and a camera model of the initial pinhole image according to the camera model parameters;
the target pinhole image is generated based on the reference plane and the camera optical axis offset.
In some of these embodiments, determining a reference plane corresponding to an initial pinhole image from a camera sphere model of the panoramic image and the resolution includes:
converting the panoramic image into a sphere camera model;
acquiring an intersection point of a coordinate axis of the spherical camera model and the outer surface of the spherical camera model;
and carrying out image normalization according to the intersection point and the resolution to obtain the reference plane.
In some of these embodiments, normalizing the image according to the intersection point and the resolution, obtaining the reference plane includes:
taking the intersection point as a tangent point to obtain a target plane tangent to the camera sphere model;
and carrying out image normalization on the target plane according to the resolution ratio to obtain the reference plane, wherein the reference plane is a three-dimensional point set of the pixel points of the initial pinhole image, which corresponds to the camera sphere model coordinate system.
In some of these embodiments, generating the target pinhole image based on the reference plane and the optical axis offset includes:
acquiring initial three-dimensional point coordinates corresponding to the initial pinhole image in the reference plane;
obtaining a target three-dimensional point coordinate of the target pinhole image according to the initial three-dimensional point coordinate and the optical axis offset;
mapping the target three-dimensional point to the panoramic image through the camera sphere model to obtain a pixel position of the target three-dimensional point in the panoramic image;
and acquiring a pixel value corresponding to the pixel position in the panoramic image, and generating the target pinhole image based on the pixel value.
In some of these embodiments, the method further comprises:
and under the condition that the pixel position is a non-integer, acquiring a pixel value corresponding to the pixel position through a bilinear difference value, and generating the target pinhole image based on the pixel value.
In some of these embodiments, the three-dimensional points of the target pinhole image are mapped to the panoramic image by the following formula:
wherein, (x) c 、y c 、z c ) Is the target three-dimensional point coordinates, (p) w ，p h ) Is the resolution, beta, r, delta are intermediate parameters,is the pixel location.
In a second aspect, the present embodiment provides a panoramic image processing system, the system comprising: a preprocessing module and a pinhole image generation module, wherein,
the preprocessing module is used for determining the resolution of a pinhole image to be generated and camera model parameters corresponding to the pinhole image based on the panoramic image,
determining a reference plane corresponding to the initial pinhole image according to the camera sphere model of the panoramic image and the resolution;
the pinhole image generating module is configured to determine an optical axis offset between a camera model of a target pinhole image and a camera model of the initial pinhole image according to the camera model parameters, and generate the target pinhole image based on the reference plane and the camera optical axis offset.
In some of these embodiments, the preprocessing module determines a reference plane corresponding to an initial pinhole image based on a camera sphere model of the panoramic image and the resolution, including:
determining a sphere camera model corresponding to the panoramic image;
acquiring an intersection point of a coordinate axis of the spherical camera model and the outer surface of the spherical camera model;
and carrying out normalization processing according to the intersection point and the resolution to obtain the reference plane.
In a third aspect, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the method according to the first aspect when executing the computer program.
In a fourth aspect, embodiments of the present application provide a computer readable storage medium having stored thereon a computer program which when executed by a processor implements a method as described in the first aspect above.
Compared with the related art, the panoramic image processing method provided by the embodiment of the application comprises the following steps: the method is based on a panoramic image, and the resolution of a pinhole image to be generated and camera model parameters corresponding to the pinhole image are determined; determining a reference plane corresponding to the initial pinhole image according to the camera sphere model and the resolution of the panoramic image; determining an optical axis offset between a camera model of the target pinhole image and a camera model of the initial pinhole image according to the camera model parameters; a target pinhole image is generated based on the reference plane and the camera optical axis offset. The application solves the problem of poor efficiency of the method for generating the pinhole image according to the panoramic image in the related technology, and can automatically calculate the number and the position of the detachable pinhole images of one panoramic image, the internal parameters and external parameters corresponding to the camera model, recover the pinhole images of any angle and the like, so that the mode of converting the panoramic image into the pinhole image is more flexible and efficient.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
fig. 1 is a schematic view of an application environment of a panoramic image processing method according to an embodiment of the present application;
fig. 2 is a flowchart of an image processing method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a camera sphere model according to an embodiment of the application;
FIG. 4 is a schematic illustration of a panoramic image derived based on a camera sphere model in accordance with an embodiment of the present application;
FIG. 5 is a schematic diagram of another camera sphere model according to an embodiment of the application;
FIG. 6 is a schematic diagram based on determining a reference plane from a camera sphere model, according to an embodiment of the application;
fig. 7 is a block diagram of a panoramic image processing system according to an embodiment of the present application;
fig. 8 is a schematic diagram of an internal structure of an electronic device according to an embodiment of the present application.
Detailed Description
The present application will be described and illustrated with reference to the accompanying drawings and examples in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application. All other embodiments, which can be made by a person of ordinary skill in the art based on the embodiments provided by the present application without making any inventive effort, are intended to fall within the scope of the present application.
It is apparent that the drawings in the following description are only some examples or embodiments of the present application, and it is possible for those of ordinary skill in the art to apply the present application to other similar situations according to these drawings without inventive effort. Moreover, it should be appreciated that while such a development effort might be complex and lengthy, it would nevertheless be a routine undertaking of design, fabrication, or manufacture for those of ordinary skill having the benefit of this disclosure, and thus should not be construed as having the benefit of this disclosure.
Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is to be expressly and implicitly understood by those of ordinary skill in the art that the described embodiments of the application can be combined with other embodiments without conflict.
Unless defined otherwise, technical or scientific terms used herein should be given the ordinary meaning as understood by one of ordinary skill in the art to which this application belongs. The terms "a," "an," "the," and similar referents in the context of the application are not to be construed as limiting the quantity, but rather as singular or plural. The terms "comprising," "including," "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to only those steps or elements but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. The terms "connected," "coupled," and the like in connection with the present application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The term "plurality" as used herein means two or more. "and/or" describes an association relationship of an association object, meaning that there may be three relationships, e.g., "a and/or B" may mean: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship. The terms "first," "second," "third," and the like, as used herein, are merely distinguishing between similar objects and not representing a particular ordering of objects.
The image processing method provided by the application can be applied to an application environment shown in fig. 1, fig. 1 is a schematic diagram of an application environment of a panoramic image processing method according to an embodiment of the application, and as shown in fig. 1, the method is applied to a server 10, a user inputs an operation instruction to the server 10 through external equipment or remote communication, the server 10 operates the image processing method provided by the embodiment, and after receiving the operation instruction input by the user, a pinhole image of any specific area can be automatically generated from the panoramic image. The server 10 in the present embodiment may be, but not limited to, a PC computer, a personal server, or a cloud service.
Fig. 2 is a flowchart of an image processing method according to an embodiment of the present application, as shown in fig. 2, the flowchart including the steps of:
s201, determining the resolution of a pinhole image to be generated and camera model parameters corresponding to the pinhole image based on the panoramic image;
different types of cameras can generate different types of images, and devices such as mobile phones and the like often adopt smaller diaphragms and lenses, so that an imaging principle is close to a classical pinhole model, and a pinhole image is generated. This type of image typically has a deeper depth of field and less distortion, but a relatively narrow viewing angle. The monitoring camera and other devices have a larger visual field range, and can shoot objects at a long distance. The camera can realize wider visual field range such as wide angle, fish eyes and the like by using the fisheye lens or lens group for matching, and the generated image distortion is larger.
The panoramic camera composed of a plurality of lenses can shoot a plurality of pictures, the complete image obtained by splicing the pictures can be called a panoramic image through a special image processing algorithm, and particularly, how to splice the plurality of images into the panoramic image has no influence on the application, so that the description is omitted in the embodiment.
Alternatively, by means of a camera spherical model, a real object in a camera coordinate system may be associated with a sphere coordinate system, fig. 3 is a schematic diagram of the camera spherical model according to an embodiment of the present application, and as shown in fig. 3, coordinate points of the real object may be represented by sphere coordinates, after which coordinate points in the sphere coordinate system may be mapped into an image by equidistant columnar projection to form a planar image. FIG. 4 is a schematic view of a panoramic image obtained based on a camera sphere model according to an embodiment of the present application, as shown in FIG. 4, the side of a cylinder outside the sphere is the projected panoramic image, wherein the panoramic image may be obtained by cutting line e a e b And cutting to obtain the panoramic camera shown in fig. 4.
In this embodiment, the resolution of the pinhole image to be generated is preset by the user according to the service requirement and the attribute of the panoramic image. Further, any one pinhole image corresponds to a camera model having its corresponding camera model parameters including, but not limited to: angle of field, internal and external parameters, etc.
In this embodiment, the internal parameters and the field angles of the pinhole camera models are equal, and the external parameters are the relative transformation relations between the plurality of cameras, and in this embodiment, the external parameters are constrained to one rotation amount.
The user sets the resolution to (p) according to the service requirement and the actual condition of the panoramic image w ，p h ) The angle of view of the corresponding pinhole camera model is alpha, and the optical axes of two adjacent pinhole cameras are offset by theta 0 The overlap region is or=α - θ 0 。
Specifically, the pinhole camera model internal reference k= (f, c), where the camera internal reference focal length f is related to the camera FOV and imaging resolution, the internal reference of the pinhole camera model can be obtained by the following equation 1.
Equation 1:
in equation 1, f is the reference focal length and c is the camera reference center.
Further, after splitting the panoramic image into a plurality of pinhole images, an external parameter exists between the pinhole cameras corresponding to each pinhole image, the external parameter is a rotation offset, and the rotation angle is a preset offset θ of the optical axis of the pinhole camera 0 Optionally, the rotation axis is the y-axis u of the camera coordinate system y ＝[0，1，0]. FIG. 5 is a schematic diagram of another camera sphere model according to an embodiment of the present application, as shown in FIG. 5, assuming that the camera coordinate system is referenced to the z-axis direction, the corresponding pinhole camera is cam0, and rotates clockwise by θ with the y-axis as the rotation axis 0 Obtaining a second pinhole camera model cam1, wherein the camera external parameters of cam1 relative to cam0 can be obtained by the following formula 2:
equation 2:
further, the external parameters of the camera model cam which are extended to any pinhole camera model cam0 can be obtained by the following formula 3:
equation 3:
s202, determining a reference plane corresponding to an initial pinhole image according to a camera sphere model and resolution of the panoramic image;
FIG. 6 is a schematic view of a reference plane based on a camera sphere model according to an embodiment of the present application, as shown in FIG. 6, the center of the camera coordinate system is located at the center of the sphere, there is a tangent plane to the Z-axis of the camera coordinate system at point P, the tangent plane is determined by taking the point P as the plane center, further, the resolution (P w ，p h ) And carrying out image normalization to obtain a reference plane. And storing three-dimensional points of the camera coordinate system corresponding to all pixel points of the initial pinhole image under the panoramic camera sphere model in the reference plane.
It should be noted that, in this embodiment, through image normalization, the pixel value of the image may be mapped into a specific range (such as a percentage) to obtain a normalized image plane, so that calculation and processing may be facilitated, the pixel value of the image may be more controllable and predictable in value, and the comparability of the image and the robustness of the algorithm may be improved through a unified quantization range.
Specifically, assume that a normalized plane corresponding to a z-axis intersection tangent point of the camera coordinate system is a reference plane S 0 The generated pinhole image is I 0 The corresponding pinhole camera is cam0. Reference plane S 0 Any one of three-dimensional points P i The coordinates in the camera coordinate system can be obtained by the following equation 4:
equation 4:
in addition, the reference plane is determined by a plane tangential to the Z axis is only a specific example, and it is to be understood that in the present embodiment, the reference plane may be determined by an X axis or a Y axis of the sphere camera model, or may be determined by any one of rays from the center of the sphere to the surface of the sphere.
S203, determining the optical axis offset between the camera model of the target pinhole image and the camera model of the initial pinhole image according to the camera model parameters, and generating the target pinhole image based on the reference plane and the camera optical axis offset;
in this embodiment, the reference plane S is obtained due to the rotation relationship of the pinhole camera optical axis offset corresponding to the normalized plane 0 Then, by combining the offset angle theta of the optical axis of the pinhole camera, a normalized plane S under any angle can be obtained i Coordinates of the three-dimensional point:
specifically, the three-dimensional point of any one normalized plane can be calculated based on the sum of the reference planes and the optical axis shift by the following equation 5:
equation 5:
further, mapping the three-dimensional point to the panoramic image through a camera sphere model, determining a corresponding pixel position, and obtaining a corresponding pixel value according to the pixel position to obtain a corresponding target pinhole image.
Compared with the method for acquiring the pinhole images from the panoramic images through manual operation in the related art, the method provided by the embodiment of the application can acquire the number and the position of the detachable pinhole images from any panoramic image, the internal parameters and the external parameters corresponding to the camera model and recover the pinhole images at any angle, so that the mode of converting the pinhole images by the panoramic images is more flexible, the panoramic images are not required to be acquired again through manual adjustment of the parameters, and the generation efficiency of the pinhole images is greatly improved.
In some of these embodiments, determining the reference plane of the initial pinhole image based on the camera sphere model resolution of the panoramic image includes:
step1, determining a coordinate system of a camera sphere model, and acquiring an intersection point of a coordinate axis in the coordinate system and an outer layer of the camera sphere model;
in this embodiment, preferably, an intersection point between the z axis of the camera sphere model and the outer surface of the camera sphere model is taken as the intersection point;
step2, taking the intersection point as a tangent point, and acquiring a target plane tangent to the camera sphere model;
step3, normalizing the target plane according to the resolution ratio to obtain a reference plane,
after determining a plane in Step2, performing image normalization in combination with a preset pinhole image resolution to obtain the reference plane. It can be understood that the reference plane is a three-dimensional point set of the initial target image under the camera coordinate system corresponding to all the pixel points under the panoramic camera sphere model.
Through the steps, any pinhole image is taken as an initial pinhole image, a reference plane corresponding to the initial pinhole image can be acquired, and basic data is provided for subsequent acquisition of pinhole images at other positions.
In some of these embodiments, generating the target pinhole image based on the reference plane and the optical axis offset includes:
step1, acquiring a three-dimensional point corresponding to an initial pinhole image in a reference plane, and acquiring a three-dimensional point of a target pinhole image according to the three-dimensional point in the initial pinhole image and the offset of the optical axis of the camera;
the specific implementation manner of this step may be that the target pinhole image may be a pinhole image at any position, and the camera optical axis offset is a rotation angle between two camera models through the above formula 5.
Step2, mapping the three-dimensional point of the target pinhole image to the panoramic image through a camera sphere model to obtain a pixel position corresponding to the three-dimensional point;
the three-dimensional point is required to be converted into spherical coordinates, and then the spherical coordinates are projected to the panoramic image to obtain the pixel position. It should be noted that, in the present embodiment, projection methods that may be applied include, but are not limited to: equidistant columnar projections (equirectangular 1ar projections), cube map projections (Cube Map Projection) and equidistant azimuthal projections (Azimuthal Equidistant Projection), in this embodiment the pixel positions are preferably obtained by equidistant columnar projections.
Specifically, the projection process of equidistant columnar projection is to convert longitude and latitude coordinates of a sphere into plane coordinates, wherein the longitude corresponds to a transverse position on a plane, and the latitude corresponds to a longitudinal position on the plane. The projection method keeps the area of each area equal by uniformly distributing the warp of the sphere on the cylindrical surface and then expanding the warp into a plane.
The three-dimensional point P in the normalized plane can be calculated using the following equation 6 i Mapping to panoramic image to obtain pixel locations
Equation 6:
step3, obtaining pixel values corresponding to pixel positions in the panoramic image, and generating a target pinhole image based on the pixel values.
It can be understood that this step obtains the corresponding pixel value on the panoramic image according to the pixel position, and further obtains the required target pinhole image,
through the steps, the pinhole images at any angles can be restored through the set external parameters and the normalization reference plane, so that the panoramic image pinhole image conversion mode is more flexible and efficient.
In some embodiments, the pixel values corresponding to the pixel positions may be obtained by bilinear interpolation or the like, in consideration that the pixel positions obtained in the above steps may not be integers. Wherein the pixel value of the non-integer pixel position can be calculated by weighted averaging of the four nearest pixels to the target position by bilinear interpolation. The method comprises the following specific steps:
step1, four nearest neighbor pixels, typically four adjacent integer pixel positions, of the target pixel position are determined.
Step2, calculating the horizontal and vertical offset of the target pixel position relative to the nearest neighbor pixel.
Step3, weighting the pixel values of the nearest neighbor pixels respectively, and determining weights according to the offset and the distance, wherein the weights are generally calculated by using linear interpolation in the horizontal and vertical directions and the distance weights.
Step4, taking the weighted average of the four nearest neighbor pixels as the pixel value of the target pixel position.
Through the embodiment, under the condition that the pixel position obtained by mapping is a non-integer, the pixel value closest to the actual situation can be obtained, so that the accuracy of the pinhole image is further ensured.
It should be noted that the steps illustrated in the above-described flow or flow diagrams of the figures may be performed in a computer system, such as a set of computer-executable instructions, and that, although a logical order is illustrated in the flow diagrams, in some cases, the steps illustrated or described may be performed in an order other than that illustrated herein.
The embodiment also provides a panoramic image processing system, which is used for implementing the above embodiments and preferred embodiments, and is not described in detail. As used below, the terms "module," "unit," "sub-unit," and the like may be a combination of software and/or hardware that implements a predetermined function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.
Fig. 7 is a block diagram of a panoramic image processing system according to an embodiment of the application, as shown in fig. 7, the system comprising: a preprocessing module 70, and a pinhole image generation module 71, wherein,
a preprocessing module 70, configured to determine, based on the panoramic image, a resolution of a pinhole image to be generated, and a camera model parameter corresponding to the pinhole image, and determine, according to a camera sphere model and the resolution of the panoramic image, a reference plane corresponding to an initial pinhole image;
the pinhole image generating module 71 is configured to determine an optical axis offset between the camera model of the target pinhole image and the camera model of the initial pinhole image according to the camera model parameters, and generate the target pinhole image based on the reference plane and the camera optical axis offset.
By the system, the number and the position of the detachable pinhole images and the internal and external parameters corresponding to the camera model can be obtained from any panoramic image, and the pinhole images at any angles can be recovered, so that the panoramic image can be converted more flexibly, the panoramic image can be obtained again without manually adjusting the parameters, and the generation efficiency of the pinhole images is greatly improved.
In one embodiment, fig. 8 is a schematic diagram of an internal structure of an electronic device according to an embodiment of the present application, and as shown in fig. 8, an electronic device, which may be a server, is provided, and an internal structure diagram thereof may be as shown in fig. 8. The electronic device includes a processor, a network interface, an internal memory, and a non-volatile memory connected by an internal bus, where the non-volatile memory stores an operating system, computer programs, and a database. The processor is used for providing computing and control capabilities, the network interface is used for communicating with an external terminal through a network connection, the internal memory is used for providing an environment for the operation of an operating system, and the computer program is executed by the processor in a panoramic image processing method, and the database is used for storing data.
It will be appreciated by those skilled in the art that the structure shown in fig. 8 is merely a block diagram of a portion of the structure associated with the present inventive arrangements and is not limiting of the electronic device to which the present inventive arrangements are applied, and that a particular electronic device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The foregoing examples illustrate only a few embodiments of the application, which are described in detail and are not to be construed as limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of protection of the present application is to be determined by the appended claims.
Claims (10)
1. A panoramic image processing method, the method comprising:
determining the resolution of a pinhole image to be generated and camera model parameters corresponding to the pinhole image based on the panoramic image;
determining a reference plane corresponding to an initial pinhole image according to a camera sphere model of the panoramic image and the resolution;
determining an optical axis offset between a camera model of a target pinhole image and a camera model of the initial pinhole image according to the camera model parameters;
the target pinhole image is generated based on the reference plane and the camera optical axis offset.
2. The method of claim 1, wherein determining a reference plane corresponding to an initial pinhole image based on a camera sphere model of the panoramic image and the resolution comprises:
converting the panoramic image into a sphere camera model;
acquiring an intersection point of a coordinate axis of the spherical camera model and the outer surface of the spherical camera model;
and carrying out image normalization according to the intersection point and the resolution to obtain the reference plane.
3. The method of claim 2, wherein normalizing the image based on the intersection point and the resolution to obtain the reference plane comprises:
taking the intersection point as a tangent point to obtain a target plane tangent to the camera sphere model;
and carrying out image normalization on the target plane according to the resolution ratio to obtain the reference plane, wherein the reference plane is a three-dimensional point set of the pixel points of the initial pinhole image, which corresponds to the camera sphere model coordinate system.
4. A method according to any one of claims 1 or 3, wherein generating the target pinhole image based on the reference plane and the optical axis offset comprises:
acquiring initial three-dimensional point coordinates corresponding to the initial pinhole image in the reference plane;
obtaining a target three-dimensional point coordinate of the target pinhole image according to the initial three-dimensional point coordinate and the optical axis offset;
mapping the target three-dimensional point to the panoramic image through the camera sphere model to obtain a pixel position of the target three-dimensional point in the panoramic image;
and acquiring a pixel value corresponding to the pixel position in the panoramic image, and generating the target pinhole image based on the pixel value.
5. The method according to claim 4, wherein the method further comprises:
and under the condition that the pixel position is a non-integer, acquiring a pixel value corresponding to the pixel position through a bilinear difference value, and generating the target pinhole image based on the pixel value.
6. The method according to any one of claims 4 or 5, wherein the three-dimensional point of the target pinhole image is mapped to the panoramic image by the following formula:
wherein, (x) c 、y c 、z c ) Is the target three-dimensional point coordinates, (p) w ，p h ) Is the resolution, beta, r, delta are intermediate parameters,is the pixel location.
7. A panoramic image processing system, said system comprising: a preprocessing module and a pinhole image generation module, wherein,
the preprocessing module is used for determining the resolution of a pinhole image to be generated and camera model parameters corresponding to the pinhole image based on the panoramic image,
determining a reference plane corresponding to the initial pinhole image according to the camera sphere model of the panoramic image and the resolution;
the pinhole image generating module is configured to determine an optical axis offset between a camera model of a target pinhole image and a camera model of the initial pinhole image according to the camera model parameters, and generate the target pinhole image based on the reference plane and the camera optical axis offset.
8. The method of claim 7, wherein the preprocessing module determining a reference plane corresponding to an initial pinhole image based on a camera sphere model of the panoramic image and the resolution comprises:
determining a sphere camera model corresponding to the panoramic image;
acquiring an intersection point of a coordinate axis of the spherical camera model and the outer surface of the spherical camera model;
and carrying out normalization processing according to the intersection point and the resolution to obtain the reference plane.
9. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method according to any of claims 1 to 6 when executing the computer program.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310637167.8A CN116757928A (en) | 2023-05-31 | 2023-05-31 | Panoramic image processing method, panoramic image processing system, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310637167.8A CN116757928A (en) | 2023-05-31 | 2023-05-31 | Panoramic image processing method, panoramic image processing system, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116757928A true CN116757928A (en) | 2023-09-15 |
Family
ID=87956292
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310637167.8A Pending CN116757928A (en) | 2023-05-31 | 2023-05-31 | Panoramic image processing method, panoramic image processing system, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116757928A (en) |
-
2023
- 2023-05-31 CN CN202310637167.8A patent/CN116757928A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102227583B1 (en) | Method and apparatus for camera calibration based on deep learning | |
| CN111052176B (en) | Seamless image stitching | |
| CN106875451B (en) | Camera calibration method and device and electronic equipment | |
| US9436973B2 (en) | Coordinate computation device and method, and an image processing device and method | |
| CN104778656B (en) | Fisheye image correcting method based on spherical perspective projection | |
| Ha et al. | Panorama mosaic optimization for mobile camera systems | |
| CN111784585B (en) | Image splicing method and device, electronic equipment and computer readable storage medium | |
| US10565803B2 (en) | Methods and apparatuses for determining positions of multi-directional image capture apparatuses | |
| CN112215880B (en) | Image depth estimation method and device, electronic equipment and storage medium | |
| CN113643414B (en) | Three-dimensional image generation method and device, electronic equipment and storage medium | |
| CN111598777A (en) | Sky cloud image processing method, computer device and readable storage medium | |
| CN111161173A (en) | Image correction information acquisition method, image correction information acquisition device, image correction information model construction method, image correction information model construction device, and medium | |
| CN114511447A (en) | Image processing method, device, equipment and computer storage medium | |
| EP4266239A1 (en) | Image splicing method, computer-readable storage medium, and computer device | |
| CN116757928A (en) | Panoramic image processing method, panoramic image processing system, electronic equipment and storage medium | |
| CN111353945A (en) | Fisheye image correction method, fisheye image correction device and storage medium | |
| CN113436247B (en) | Image processing method and device, electronic equipment and storage medium | |
| WO2018150086A2 (en) | Methods and apparatuses for determining positions of multi-directional image capture apparatuses | |
| CN112203023B (en) | Billion pixel video generation method and device, equipment and medium | |
| WO2018100230A1 (en) | Method and apparatuses for determining positions of multi-directional image capture apparatuses | |
| US20130343636A1 (en) | Image processing apparatus, control method of the same and non-transitory computer-readable storage medium | |
| CN114926371B (en) | Vertical correction and vanishing point detection method and device for panorama and storage medium | |
| CN117422650B (en) | Panoramic image distortion correction method and device, electronic equipment and medium | |
| CN113763530B (en) | Image processing method, device, computing equipment and storage medium | |
| CN115660959B (en) | Image generation method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |