CN107862649B - GPU acceleration multi-channel fusion method and system based on simulation visual system - Google Patents
GPU acceleration multi-channel fusion method and system based on simulation visual system Download PDFInfo
- Publication number
- CN107862649B CN107862649B CN201710943690.8A CN201710943690A CN107862649B CN 107862649 B CN107862649 B CN 107862649B CN 201710943690 A CN201710943690 A CN 201710943690A CN 107862649 B CN107862649 B CN 107862649B
- Authority
- CN
- China
- Prior art keywords
- fusion
- value
- picture
- gpu
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004088 simulation Methods 0.000 title claims abstract description 50
- 230000000007 visual effect Effects 0.000 title claims abstract description 39
- 238000007500 overflow downdraw method Methods 0.000 title claims abstract description 22
- 230000001133 acceleration Effects 0.000 title claims description 13
- 230000004927 fusion Effects 0.000 claims abstract description 78
- 238000000034 method Methods 0.000 claims abstract description 26
- 238000007499 fusion processing Methods 0.000 claims abstract description 25
- 238000012937 correction Methods 0.000 claims abstract description 24
- 238000012545 processing Methods 0.000 claims abstract description 22
- 230000008569 process Effects 0.000 claims abstract description 17
- 230000000694 effects Effects 0.000 claims abstract description 12
- 239000003086 colorant Substances 0.000 claims description 10
- 238000009792 diffusion process Methods 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000012544 monitoring process Methods 0.000 claims description 4
- 238000005562 fading Methods 0.000 claims description 2
- 210000003746 feather Anatomy 0.000 claims description 2
- 230000001105 regulatory effect Effects 0.000 claims description 2
- 230000008901 benefit Effects 0.000 abstract description 6
- 230000004075 alteration Effects 0.000 abstract description 4
- 230000002708 enhancing effect Effects 0.000 abstract description 2
- 230000006872 improvement Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 4
- 238000002156 mixing Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000001788 irregular Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 241000872198 Serjania polyphylla Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004040 coloring Methods 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
-
- G06T5/70—
-
- G06T5/80—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Abstract
The invention discloses a GPU (graphics processing Unit) accelerated multi-channel fusion method and a system based on a simulation visual system, belonging to the related technical field of simulation, wherein the rapid processing of a multi-channel projection picture is effectively realized by carrying out processes such as chromatic aberration adjustment, geometric correction, picture fusion and the like on a multi-channel fusion picture, the functions are integrated with the simulation visual system, and the GPU is utilized to rapidly process the multi-channel picture fusion process. The GPU accelerated multi-channel fusion method and system based on the simulation visual system, utilize the GPU to process the ability fast, the operation is flexible, the regulation mode is various, and does not need the hardware debugging interface, the debugging is convenient, greatly reduce the input cost of the hardware while obviously enhancing the display effect, have obvious economic benefits.
Description
Technical Field
The invention belongs to the technical field of simulation correlation, and particularly relates to a GPU (graphics processing unit) acceleration multi-channel fusion method and system based on a simulation vision system.
Background
With the high-speed development of economy in China, the rail transit industry is also in the high-speed development stage, and diversified and multi-form rail transit is also rapidly developed, updated and applied. The application of the simulation technology in the field of rail transit is mature day by day, and various energy consumption analysis or equipment type selection analysis simulation systems of subsystems such as driving organizations, operation scheduling, power supply, station yards and the like are widely applied. The computer simulation technology combines two methods of experiment and analysis, adopts a simulation model to replace an entity for experiment, and has the advantages of economy, safety, short experiment period and the like. Particularly in the aspect of driving training, various train simulation systems play more and more important roles in training train drivers with the outstanding advantages of vividness, flexibility, effectiveness, economy and the like.
In a train driving simulation training system, a large circular screen, a large spliced screen and a large irregular screen become very important components, but in the application process of the screens, a single projector is difficult to meet the resolution requirement of the screens, a plurality of projectors are required to output pictures simultaneously, the output pictures need to be spliced and fused, but the process of splicing and fusing the output pictures is very complicated, and the difficulty is high.
In the prior art, in order to realize splicing and fusion of output pictures by a plurality of projectors, a hardware fusion machine customized by a manufacturer of a purchased professional fusion machine is generally adopted, a plurality of paths of video signals are input into the hardware fusion machine, and then an image is processed by using an FPGA chip inside the hardware fusion machine and then output to a projection screen. Although the method can realize splicing and fusion of images output by a plurality of projectors to a certain extent, the method usually takes a long time when the images are processed in the hardware fusion machine, so that the images fused by hardware generally have the problem of delay, the image refreshing rate is not high, in addition, a large amount of capital cost is required for purchasing the fusion machine, and in the debugging process, the fusion machine manufacturer is required to carry out technical support to complete the splicing of the scenes. And once problems arise during the course of the project operation, the fuser manufacturer is required to dispatch technicians for technical support multiple times, which undoubtedly results in additional economic costs and requires extensive coordination of time periods between the customer and the fuser manufacturer. In addition, because the hardware fusion machine collects image signals for the second time for processing, not only time delay exists, but also the requirement on the pixel proportion of a fusion belt is high, more pixels can be lost, and then an output picture is not clear, and the picture is blocked.
Disclosure of Invention
Aiming at the defects or improvement requirements of the prior art, the invention provides a GPU (graphics processing Unit) accelerated multi-channel fusion method based on a simulation visual system, wherein the rapid processing of a multi-channel projection picture is effectively realized by performing processes such as chromatic aberration adjustment, geometric correction, picture fusion and the like on a multi-channel fusion picture, the functions are integrated with the simulation visual system, the multi-channel picture fusion process is rapidly processed by utilizing the GPU, the economic cost for realizing picture fusion is effectively reduced, the speed of the picture processing process is improved, and the synchronism of the projection picture is ensured.
In order to achieve the above object, one aspect of the present invention provides a GPU accelerated multi-channel fusion method based on a simulation view system, which is used for splicing and fusing multi-channel projection images on a view screen in an application process of the simulation view system, and comprises the following steps:
s1: a resolution adjustment step of adjusting a resolution of the view screen according to resolutions of projectors used for projecting the picture on the view screen and a resolution of an instructor display for monitoring a forward view;
s2: adjusting color difference, namely adjusting the output colors of the projectors of all channels to be consistent;
s3: the method comprises the following steps of performing picture geometric correction, namely constructing a plurality of virtual grids corresponding to the number of projectors in a display card of the simulation visual system, enabling each projector to correspond to one virtual grid, adjusting the virtual grids according to the shape of a visual screen, projecting rendered pictures onto the virtual grids, wherein the data of the virtual grids comprise vertex data, texture coordinate data and color attribute data, and changing the vertex data of the virtual grids to achieve the distortion effect on the pictures so as to realize the geometric correction on the multichannel pictures;
s4: the image fusion processing is carried out, after the image is geometrically corrected, the fusion processing is carried out on the overlapped and staggered parts of each projection, the image brightness of a fusion zone formed when two or more projectors are combined, projected and spliced into one image is adjusted, so that the image brightness of the fusion zone is consistent with the image brightness of two sides of the fusion zone, namely the brightness of the whole image of the visual screen is consistent, and the fusion processing of the multi-channel projection image is realized;
s5: and dark field regulation, namely after the picture fusion processing is finished, regulating the brightness of the dark fields on the two sides of the fusion zone on the visual screen to make the brightness of the whole screen tend to be consistent, and then outputting a multi-channel fusion picture.
As a further improvement of the present invention, the vertex data in step S3 includes three values, x, y and z, where the x value and the y value determine the position of the vertex in the viewing screen, the z value is used to store an alpha value corresponding to the vertex, and the alpha value refers to a brightness value and/or a transparency value of the vertex.
As a further improvement of the present invention, in step S3, the geometric correction of the picture is implemented by applying a cubic spline curve function, and the cubic spline curve function can automatically perform smooth interpolation on the grid lines of the virtual grid to implement smooth and stepless jaggies of the grid lines of the whole picture.
As a further refinement of the present invention, the cubic spline function is defined as follows:
defining: function S (x) epsilon C2[ b, C]And in each cell [ x ]j,xj+1]Above is a cubic polynomial wherein
b＝x0<x1<…<xnGiven node c, then s (x) is node x0,x1,...xnThe cubic spline function of (a) above,
if at node xjGiven the function value yj＝f(xj) And S (x)j)＝yjWhere j is 0,1, …, n, n is a natural number, and s (x) is a cubic spline interpolation function.
As a further improvement of the present invention, in the step S4, the blending processing of the picture is performed in a way of accelerating the GPU hardware, a fade-in and fade-out processing function in the Shader code is executed at the GPU end to perform fade-in and fade-out processing on the blending band, and a multi-segment bezier luminance curve is used to perform feathering processing on the blending band, so as to eliminate the bright band.
As a further improvement of the present invention, the Shader code includes a fade function for fading the blend zone, wherein the fade function is as follows:
and the value of z is equal to an alpha value, the GPU of the simulation visual system interpolates the alpha value according to the vertex data, and then the z value is smoothed, wherein the value a and the value p are respectively a fusion band brightness ratio and a fusion band diffusion factor.
As a further improvement of the present invention, the Shader code includes a luminance calculation function and a bezier luminance curve, and is configured to calculate a luminance value of the fusion band and use the luminance value by the bezier luminance curve to further feather the fusion band, where the luminance calculation function is as follows:
float brightness＝(z>0.5)？(1-(1-a)×pow(2×(1-z),p)):(a×pow(2×z,p))
wherein the value of z is equal to the alpha value, and the a value and the p value are the fusion band luminance ratio and the fusion band diffusion factor, respectively.
In another aspect of the present invention, a GPU-accelerated multi-channel fusion system based on a simulation view system is provided, which is used for splicing and fusing multi-channel projection images on a view screen in an application process of the simulation view system, and the GPU-accelerated multi-channel fusion system is characterized by comprising:
a resolution adjustment module for adjusting a resolution of a view screen according to a resolution of each projector employed to project a picture on the view screen and a resolution of an instructor display for monitoring a forward view;
the color difference adjusting module is used for adjusting the output colors of the projectors of all the channels to be consistent;
the picture geometric correction module is used for constructing a plurality of virtual grids corresponding to the number of the projectors in a display card of the simulation visual system, enabling each projector to correspond to one virtual grid, adjusting the virtual grids according to the shape of the visual screen, projecting rendered pictures onto the virtual grids, wherein the data of the virtual grids comprise vertex data, texture coordinate data and color attribute data, and the distortion effect on the pictures is achieved by changing the vertex data of the virtual grids, so that the geometric correction of the multichannel pictures is realized;
the image fusion processing module is used for performing fusion processing on the overlapped and staggered parts of the projections after the geometric correction of the image is finished, and adjusting the image brightness of a fusion band formed when two or more projectors are combined, projected and spliced into one image, so that the image brightness of the fusion band is consistent with the image brightness of two sides of the fusion band, namely the brightness of the whole image of the visual screen is consistent, and the fusion processing of the multi-channel projection image is realized; and
and the dark field adjusting module is used for adjusting the brightness of the dark fields on the two sides of the fusion zone on the visual scene screen after the image fusion processing is finished, so that the brightness of the whole screen tends to be consistent, and then outputting a multi-channel fusion image.
As a further improvement of the present invention, a function of "simulating full screen" is integrated in the simulation view system program to implement the resolution adjustment process in step S1.
As a further improvement of the present invention, the dark field adjusting module is integrated in the artificial vision system to realize the dark field adjusting process.
Generally, compared with the prior art, the above technical solution conceived by the present invention has the following beneficial effects:
(1) according to the GPU acceleration multi-channel fusion method based on the simulation view system, smooth interpolation is automatically carried out on the constructed virtual projection grid by applying the cubic spline function, the geometric correction of a multi-channel output picture is effectively realized, and the effect that the whole picture curve grid is smooth and has no stepped sawteeth is achieved;
(2) according to the GPU acceleration multi-channel fusion method based on the simulation visual system, the shadow code executed at the GPU end is used for performing fade-in and fade-out processing on the fusion zone of the projection picture in a GPU hardware acceleration mode, and a Bessel brightness curve is used for performing feathering processing on the fusion zone, so that the fusion processing of the multi-channel projection picture is effectively realized, and the brightness of the fused picture tends to be consistent;
(3) the GPU acceleration multi-channel fusion method based on the simulation visual system is realized based on the simulation visual system, can tightly combine various adjusting functions with visual programs, has extremely high stability and compatibility, completes the processing process of pictures in the GPU of the display card, fully utilizes the quick processing capacity of the display card, reduces the delay of projected pictures and ensures the real-time output of the simulation pictures;
(4) the GPU accelerated multi-channel fusion method based on the simulation visual system, disclosed by the invention, is based on the simulation visual system, utilizes the GPU rapid processing capability, is flexible to operate, has various adjusting modes, does not need a hardware debugging interface, is convenient to debug, greatly reduces the investment cost of hardware while remarkably enhancing the display effect, and has remarkable economic benefit.
Drawings
FIG. 1 is a schematic diagram of the working principle of a GPU-accelerated multi-channel fusion method and system based on a simulation vision system according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a projector and a display in the GPU-accelerated multi-channel fusion method and system based on a simulation vision system according to the embodiment of the invention;
fig. 3 is a schematic diagram of a GPU accelerated multi-channel fusion method and system based on a simulation vision system according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
The working principle schematic diagram of the GPU acceleration multi-channel fusion method based on the simulation view system in the preferred embodiment is shown in FIG. 1, the GPU based on the simulation view system is used for realizing the rapid splicing and fusion of multi-channel output pictures, the method utilizes the hardware acceleration capability of a graphics card GPU to efficiently realize the multi-channel large-screen display output effect, and therefore various determinations of the traditional hardware fusion display are eliminated.
The implementation steps of the GPU acceleration multi-channel fusion method based on the simulation vision system in a preferred embodiment comprise resolution adjustment, chromatic aberration adjustment, picture geometric correction, picture fusion processing, dark field adjustment and the like. Further specifically:
s1: resolution adjustment: before the GPU of the simulation visual system is used for multi-channel image fusion, the resolution ratio of a visual program needs to be set. In a preferred embodiment, the number of the projectors projected together is three, the positional relationship between the three projectors and the position relationship between the projectors and the instructor display are shown in fig. 2, wherein, the instructor monitors the forward view, i.e. the view screen is located right in front of the projectors, if the resolution of each projector is 1440 × 1050 and the resolution of the instructor display is 1024 × 768, the resolution of the view program should be set to 1440 × 3 horizontally and 1050+768 vertically, i.e. the resolution of the view program should be set to 4320 × 1818; further preferably, a function of simulating a full screen is integrated in the simulation view system program in a preferred embodiment, and resolution adjustment can be effectively realized by simulating the full screen, so that the limitation that the simulation view system cannot realize a full screen program under a win7 operating system is solved.
S2: and (3) color difference adjustment: after the resolution adjustment is completed, it is necessary to determine whether the colors output by the projectors of each channel are consistent, and if the output colors are not consistent, color difference adjustment is necessary, and in a preferred embodiment, the method for performing color difference adjustment on the colors output by the projectors of each channel is as follows: and switching the background image to a gray background, observing whether the colors of each projector are consistent, and if not, solving the problem of projection chromatic aberration of different channels by using a hardware adjusting function of the projectors or by using a software fused self-contained color balancing function.
S3: and (3) correcting the picture geometry: because the view screen of the simulation system is mostly a circular screen, and the projection directions of the projectors form a certain angle, the original output picture of the multi-channel projection is usually distorted, geometric correction is needed to achieve the correct display effect, and any distortion deformation needs to be carried out on the rendered picture. In a preferred embodiment, the method for geometrically correcting the original output frames of the projectors of each channel comprises the following steps: the method comprises the steps of constructing virtual projection grids in a display card, adjusting the virtual grids according to the shape of a visual screen, projecting rendered pictures onto the virtual grids, enabling each projector to correspond to one virtual grid, enabling data of the grids to comprise vertex data, texture coordinate data and color attribute data, enabling the vertex data to determine the overall position of the pictures, enabling the texture coordinate data to determine the position of the pictures mapped to the vertex coordinates, enabling the color attributes not to be used for coloring the vertexes, and enabling the display card GPU to be used for storing data for balancing colors of three channels of red, green and blue due to the fact that the number of the attribute data which can be transmitted into the display card GPU is limited. The vertex data comprises three values of x, y and z, wherein the x value and the y value determine the position in the screen, the z value is used for storing an alpha value corresponding to each vertex, and the alpha value is used for referring to a brightness value and/or a transparency value of the vertex, so that the distortion effect on the picture can be achieved by changing the vertex value of the grid.
Further, the simulation system in a preferred embodiment integrates a "grid adjustment" interface, which can adjust any projector, can adjust one grid line, and can also adjust a single grid point, so as to implement rotation, movement, or deletion of the grid line. In the preferred embodiment, the cubic spline curve function is adopted to realize the automatic smoothing of the curve, so as to realize the rapid adjustment of the grid lines, and the cubic spline curve automatically carries out smooth interpolation on the geometric correction grid lines, so as to achieve the purpose that the curve grid of the whole picture is smooth and has no stepped saw teeth.
Further, the cubic spline function in a preferred embodiment is defined as follows:
defining: function S (x) epsilon C2[ b, C]And in each cell [ x ]j,xj+1]Above is a cubic polynomial wherein
b＝x0<x1<…<xnGiven node c, then s (x) is node x0,x1,...xnCubic spline function of (a).
If at node xjAbove given function value Yj＝f(xj) (j ═ 0,1, …, n), and S (x)j)＝yjIf (j) is 0,1, …, and n is a natural number, s (x) is a cubic spline interpolation function.
S4: and (3) picture fusion processing: after the geometric correction of the picture is completed, the parts of the projection which are overlapped and staggered need to be fused. More specifically, when two or more projectors are combined, projected and spliced into a frame, a part of image lights are overlapped to form a fusion zone, so that the brightness of the frame at the overlapped part is not consistent with the brightness of the frames outside the fusion zone, and generally, the brightness of the frame at the fusion zone is greater than the brightness of the frames at two sides of the fusion zone, so that the brightness of the frame at the overlapped part needs to be gradually reduced to make the brightness of the whole frame consistent, that is, the fusion processing of the frames is realized. In a preferred embodiment, the image fusion processing is performed in a GPU hardware acceleration mode, a Shader code executed at a GPU end is used for performing fade-in fade-out processing on a fusion band, a multi-segment Bessel brightness curve is used for performing feathering processing on the fusion band, and through GPU mixing, irregular correction is performed on gamma to eliminate a bright band. The fade processing function selected in a preferred embodiment is as follows:
wherein z is the alpha value in step S3, and the graphics card interpolates the alpha value according to the value of the vertex Shader to smooth the z value, and the a value and the p value are variables respectively indicating the fusion band luminance ratio and the fusion band diffusion factor.
In a preferred embodiment, the Shader code can calculate the brightness of the fusion band, and combine the calculated brightness value with a multi-segment bezier brightness curve to implement feathering of the fusion band, where the brightness calculation function is as follows:
float brightness＝(z>0.5)？(1-(1-a)×pow(2×(1-z),p)):(a×pow(2×z,p))
wherein the value of z is equal to the alpha value, and the a value and the p value are a fusion band luminance ratio and a fusion band diffusion factor, respectively.
Further, in a preferred embodiment, when a is 0.5, the luminance graph is shown in fig. 3, and then the obtained luminance brightness is used for RGB colors, a part of the fusion band appears to be brighter or darker, because the luminance response of the projector itself is non-linear, so gamma correction needs to be added. Therefore, in the preferred embodiment, a function of curve adjustment of luminance response is provided, that is, the curve adopts a bezier curve, more control points are provided, after adjustment, the curve is sampled and then converged into a texture picture, the texture has four channels RGBA, and RGB three channels are filled, that is, the three channels of red, green and blue can be independently subjected to independent luminance curve correction. Each part of the fusion band of each grid has a single texture image, after the GPU obtains the brightness value brightness, the texture is sampled according to the brightness, and the sampled value is the new brightness value of RGB. And then multiplying the brightness value by the last RGB value to obtain the RGB value finally output by the display card, thereby realizing the fusion processing of the overlapped pictures.
S5: dark field adjustment: since the projector is black with no real sense, but gray with very low brightness. Therefore, when the scene is dark, the intersected part of the two projectors is brighter than the non-intersected part, and a bright band is relatively obvious. In the preferred embodiment, a dark field adjusting function is added, a dark field threshold (capable of being adjusted externally) is set, the GPU samples the fused image, for each image value, a color value lower than the threshold is obtained according to the threshold, then the color value is interpolated, and finally the color value is superimposed on the fused image, so that the brightness adjustment of the dark field is realized, the brightness of the whole screen tends to be consistent, and then the multi-channel fused image is output.
Compared with the existing traditional hardware fusion display system, the technical scheme in the preferred embodiment of the invention has obvious advantages, the related functions in the steps can be integrated into related functional modules, the integrated functions are tightly combined with the simulation visual system, the compatibility and the stability are good, the processing process of the picture is finished in the graphics card GPU, and the picture output is not delayed. Furthermore, the multi-channel fusion method in the preferred embodiment adopts a software built-in debugging interface to adjust in the visual program, so that what you see is what you get, a hardware debugging interface is not needed, debugging is convenient, different fusion effects can be selected at different terminals according to different configuration file settings, and the fusion effects are output by the video card at the same time.
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (8)
1. A GPU acceleration multi-channel fusion method based on a simulation visual system is used for splicing and fusing multi-channel projection images on a visual screen in the application process of the simulation visual system, and comprises the following steps:
s1: a resolution adjustment step of adjusting a resolution of the view screen according to resolutions of projectors used for projecting the picture on the view screen and a resolution of an instructor display for monitoring a forward view;
s2: adjusting color difference, namely adjusting the output colors of the projectors of all channels to be consistent;
s3: the method comprises the following steps of performing picture geometric correction, namely constructing a plurality of virtual grids corresponding to the number of projectors in a display card of the simulation visual system, enabling each projector to correspond to one virtual grid, adjusting the virtual grids according to the shape of a visual screen, projecting rendered pictures onto the virtual grids, wherein the data of the virtual grids comprise vertex data, texture coordinate data and color attribute data, and changing the vertex data of the virtual grids to achieve the distortion effect on the pictures so as to realize the geometric correction on the multichannel pictures; and is
The geometric correction of the picture is realized by applying a cubic spline curve function, and the cubic spline curve function can automatically carry out smooth interpolation on the grid lines of the virtual grid so as to realize the smooth step-free sawtooth of the grid lines of the whole picture; and the cubic spline function is defined as follows:
defining: function S (x) epsilon C2[ b, C]And in each cell [ x ]j,xj+1]Above is a cubic polynomial wherein,
b＝x0<x1<…<xngiven node c, then s (x) is node x0,x1,...xnCubic spline function ofThe number of the first and second groups is,
if at node xjGiven the function value yj＝f(xj) And S (x)j)＝yjWhere j is 0,1, …, n, n is a natural number, and s (x) is a cubic spline interpolation function;
s4: the image fusion processing is carried out, after the image is geometrically corrected, the fusion processing is carried out on the overlapped and staggered parts of each projection, the image brightness of a fusion zone formed when two or more projectors are combined, projected and spliced into one image is adjusted, so that the image brightness of the fusion zone is consistent with the image brightness of two sides of the fusion zone, namely the brightness of the whole image of the visual screen is consistent, and the fusion processing of the multi-channel projection image is realized;
s5: and dark field regulation, namely after the picture fusion processing is finished, regulating the brightness of the dark fields on the two sides of the fusion zone on the visual screen to make the brightness of the whole screen tend to be consistent, and then outputting a multi-channel fusion picture.
2. The GPU-accelerated multi-channel fusion method based on the simulation vision system of claim 1, wherein the vertex data in the step S3 comprises three values of x, y and z, wherein the x value and the y value determine the position of the vertex in the vision screen, the z value is used for saving an alpha value corresponding to the vertex, and the alpha value refers to a brightness value and/or a transparency value of the vertex.
3. The GPU-accelerated multi-channel fusion method based on the artificial vision system according to claim 1 or 2, wherein the fusion processing of the picture in step S4 adopts the GPU hardware-accelerated mode, and performs fade-in and fade-out processing on the fusion band by executing Shader code at the GPU end, and performs feathering processing on the fusion band by using multi-segment bezier luminance curves, so as to eliminate the bright band.
4. The simulated vision system-based GPU-accelerated multi-channel fusion method of claim 3, wherein said Shader code comprises a fade function for fading said fusion band, wherein said fade function is as follows:
and the value of z is equal to an alpha value, the GPU of the simulation visual system interpolates the alpha value according to the vertex data, and then the z value is smoothed, wherein the value a and the value p are respectively a fusion band brightness ratio and a fusion band diffusion factor.
5. The GPU-accelerated multi-channel fusion method based on artificial vision system of claim 3, wherein the Shader code comprises a luminance calculation function and a bezier luminance curve for calculating the luminance value of the fusion band and using the luminance value by the bezier luminance curve to further feather the fusion band, and the luminance calculation function is as follows:
float brightness＝(z>0.5)？(1-(1-a)×pow(2×(1-z),p)):(a×pow(2×z,p))
wherein the value of z is equal to the alpha value, and the a value and the p value are the fusion band luminance ratio and the fusion band diffusion factor, respectively.
6. A GPU acceleration multi-channel fusion system based on a simulation view system is used for splicing and fusing multi-channel projection images on a view screen in the application process of the simulation view system and is characterized by comprising the following steps:
a resolution adjustment module for adjusting a resolution of a view screen according to a resolution of each projector employed to project a picture on the view screen and a resolution of an instructor display for monitoring a forward view;
the color difference adjusting module is used for adjusting the output colors of the projectors of all the channels to be consistent;
the picture geometric correction module is used for constructing a plurality of virtual grids corresponding to the number of the projectors in a display card of the simulation visual system, enabling each projector to correspond to one virtual grid, adjusting the virtual grids according to the shape of the visual screen, projecting rendered pictures onto the virtual grids, wherein the data of the virtual grids comprise vertex data, texture coordinate data and color attribute data, and the distortion effect on the pictures is achieved by changing the vertex data of the virtual grids, so that the geometric correction of the multichannel pictures is realized; and is
The picture geometric correction module is used for realizing geometric correction on the picture by applying a cubic spline curve function; the cubic spline curve function is used for automatically carrying out smooth interpolation on the grid lines of the virtual grid so as to realize the smooth step-free sawtooth of the grid lines of the whole picture; and the cubic spline function is defined as follows:
defining: function S (x) epsilon C2[ b, C]And in each cell [ x ]j,xj+1]Above is a cubic polynomial wherein,
b＝x0<x1<…<xngiven node c, then s (x) is node x0,x1,...xnThe cubic spline function of (a) above,
if at node xjGiven the function value yj＝f(xj) And S (x)j)＝yjWhere j is 0,1, …, n, n is a natural number, and s (x) is a cubic spline interpolation function;
the image fusion processing module is used for performing fusion processing on the overlapped and staggered parts of the projections after the geometric correction of the image is finished, and adjusting the image brightness of a fusion band formed when two or more projectors are combined, projected and spliced into one image, so that the image brightness of the fusion band is consistent with the image brightness of two sides of the fusion band, namely the brightness of the whole image of the visual screen is consistent, and the fusion processing of the multi-channel projection image is realized; and
and the dark field adjusting module is used for adjusting the brightness of the dark fields on the two sides of the fusion zone on the visual scene screen after the image fusion processing is finished, so that the brightness of the whole screen tends to be consistent, and then outputting a multi-channel fusion image.
7. The simulated vision system-based GPU-accelerated multichannel fusion system of claim 6, wherein said resolution adjustment module is integrated in said simulated vision system, achieving resolution adjustment through "simulated full screen".
8. The simulated vision system-based GPU accelerated multi-channel fusion system of claim 6 or 7, wherein said dark-field scaling module is integrated in said simulated vision system to implement a dark-field scaling process.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710943690.8A CN107862649B (en) | 2017-10-11 | 2017-10-11 | GPU acceleration multi-channel fusion method and system based on simulation visual system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710943690.8A CN107862649B (en) | 2017-10-11 | 2017-10-11 | GPU acceleration multi-channel fusion method and system based on simulation visual system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107862649A CN107862649A (en) | 2018-03-30 |
| CN107862649B true CN107862649B (en) | 2021-04-30 |
Family
ID=61698454
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710943690.8A Active CN107862649B (en) | 2017-10-11 | 2017-10-11 | GPU acceleration multi-channel fusion method and system based on simulation visual system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107862649B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111586385B (en) * | 2020-05-29 | 2022-04-01 | 燕山大学 | Projected image color correction method based on B spline curve |
| CN111586384B (en) * | 2020-05-29 | 2022-02-11 | 燕山大学 | Projection image geometric correction method based on Bessel curved surface |
| CN113409196B (en) * | 2021-07-07 | 2023-02-24 | 安徽水天信息科技有限公司 | High-speed global chromatic aberration correction method for real-time video splicing |
| CN114416365B (en) * | 2022-01-18 | 2022-09-27 | 北京拙河科技有限公司 | Ultra-clear image quality image data processing method and device based on GPU fusion processing |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101571663A (en) * | 2009-06-01 | 2009-11-04 | 北京航空航天大学 | Distributed online regulating method for splicing multiple projectors |
| EP2249560A1 (en) * | 2009-05-06 | 2010-11-10 | Christie Digital Systems USA, Inc. | DLP edge blending artefact reduction |
| CN104282014A (en) * | 2013-07-13 | 2015-01-14 | 哈尔滨点石仿真科技有限公司 | Multichannel geometric correction and edge blending method based on NURBS curved surfaces |
| CN104954715A (en) * | 2015-07-06 | 2015-09-30 | 山东大学 | GPU (graphics processing unit) acceleration based video display method adopting multi-projector splicing fusion on special-shaped screens |
| CN105430365A (en) * | 2015-11-06 | 2016-03-23 | 深圳市创凯智能股份有限公司 | Projection fusion dark field processing method and fusion equipment |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101500172B (en) * | 2009-02-20 | 2012-11-07 | 四川华控图形科技有限公司 | Projection automatic geometric correction method based on optical sensor |
| JP6456086B2 (en) * | 2014-09-25 | 2019-01-23 | キヤノン株式会社 | Projection type image display apparatus and control method thereof, projector and control method thereof |
| CN106897976B (en) * | 2015-12-23 | 2019-09-24 | 中国人民解放军海军航空大学 | Single video card triple channel solid what comes into a driver's projection software based on GPU corrects fusion method |
-
2017
- 2017-10-11 CN CN201710943690.8A patent/CN107862649B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2249560A1 (en) * | 2009-05-06 | 2010-11-10 | Christie Digital Systems USA, Inc. | DLP edge blending artefact reduction |
| CN101571663A (en) * | 2009-06-01 | 2009-11-04 | 北京航空航天大学 | Distributed online regulating method for splicing multiple projectors |
| CN104282014A (en) * | 2013-07-13 | 2015-01-14 | 哈尔滨点石仿真科技有限公司 | Multichannel geometric correction and edge blending method based on NURBS curved surfaces |
| CN104954715A (en) * | 2015-07-06 | 2015-09-30 | 山东大学 | GPU (graphics processing unit) acceleration based video display method adopting multi-projector splicing fusion on special-shaped screens |
| CN105430365A (en) * | 2015-11-06 | 2016-03-23 | 深圳市创凯智能股份有限公司 | Projection fusion dark field processing method and fusion equipment |
Non-Patent Citations (4)
| Title |
|---|
| A Method for Multi-projectSeamless Tiled Display Systemor;Yan Han 等;《2015 2nd International Conference on Signal Processing and Integrated Networks》;20150427;第837-842页 * |
| Large Screen Stereo Projection Splice Based on Parametric Curve;Hong Zeng 等;《2012 IEEE fifth International Conference on Advanced Computational Intelligence》;20130219;第1200-1203页 * |
| 多投影沉浸式教学系统的无缝拼接技术研究;陶婷婷;《中国优秀硕士学位论文全文数据库 信息科技辑》;20150315(第3期);第I138-2292页 * |
| 快速多投影画面几何校正与边缘融合方法;曾鸿 等;《计算机工程与设计》;20130516;第34卷(第5期);第1846-1850页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107862649A (en) | 2018-03-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107862649B (en) | GPU acceleration multi-channel fusion method and system based on simulation visual system | |
| CN104954715B (en) | Method based on the GPU special-shaped screen multi-projection system fusion video playbacks accelerated | |
| CN104954769B (en) | A kind of immersion ultra high-definition processing system for video and method | |
| CN103037189B (en) | A kind of method realizing multiple projections overall output large-size screen monitors frame video image | |
| CN105574918A (en) | Material adding method and apparatus of 3D model, and terminal | |
| CN101082992A (en) | Drawing of real time high dynamic range image and display process | |
| CN102929572A (en) | Method for realizing large-screen multi-projection seamless splicing and splicing fusion device thereof | |
| CN107087123A (en) | It is a kind of that image space method is scratched based on the real-time high-definition that high in the clouds is handled | |
| CN206819048U (en) | A kind of ball curtain projection system | |
| CN102819855B (en) | The generation method of two dimensional image and device | |
| CN108986232A (en) | A method of it is shown in VR and AR environment picture is presented in equipment | |
| CN105933692A (en) | Method Used For Preparing 3d Images | |
| CN104157005A (en) | Image-based HDR (high-dynamic range) illumination rendering method | |
| US6462743B1 (en) | Pipeline processing system and method | |
| CN114071104B (en) | Method for realizing multi-projector projection gradual change fusion based on shader | |
| CN102541488A (en) | Image processing method and system for realizing seamless alignment of projection screen | |
| GB2390950A (en) | Video wipe generation based on the distance of a display position between a wipe origin and a wipe destination | |
| CN104469226B (en) | Project the method and fusion device of fusion | |
| CN102737410B (en) | Display methods and device of a kind of three-dimensional pel in two-dimentional picture and text compile broadcast system | |
| CN106951204A (en) | Image synchronization method based on computer cluster visualization system | |
| CN104836970B (en) | More projection fusion methods and system based on GPU real time video processings | |
| CN113093903B (en) | Image display method and display equipment | |
| CN101290763A (en) | Image brightness dynamically regulating method | |
| US6268860B1 (en) | Imaging method and apparatus for executing the method | |
| CN115484446A (en) | Multi-projector fusion color soft calibration system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |