CN116112802A

CN116112802A - Concentric splicing method and device

Info

Publication number: CN116112802A
Application number: CN202211415510.6A
Authority: CN
Inventors: 刘锡林; 郭西越; 杨玉峰; 赵兴国
Original assignee: Shanghai Sailian Information Technology Co ltd
Current assignee: Shanghai Sailian Information Technology Co ltd
Priority date: 2022-11-11
Filing date: 2022-11-11
Publication date: 2023-05-12

Abstract

The invention provides a common optical center splicing method and a device thereof. Wherein the method comprises: adjusting the co-light heart image acquisition array by using a physical splicing adjusting mechanism to obtain an adjusted co-light heart image acquisition array; acquiring an image by using the adjusted co-optical heart image acquisition array to obtain a co-optical heart image with a mark point; roughly determining a first splicing line according to mark points on the common optical center image; acquiring an image by using a panoramic camera to obtain a panoramic image; comparing the co-optical center image with the panoramic image to obtain an image coincidence ratio; accurately determining a second stitching line around the first stitching line based on the image overlap ratio; and cutting the spliced edge of the co-optical center image according to the second splicing line to obtain a cut co-optical center image, and splicing the cut co-optical center image according to the cut edge to obtain a spliced co-optical center image. The co-optical center splicing method effectively realizes the low-cost accurate determination of the image splicing position of the co-optical center, and solves the problem of eclosion dip-dyeing of the image splicing position.

Description

Concentric splicing method and device

Technical Field

The invention relates to the technical field of image acquisition and processing, in particular to a common optical center splicing method and a device thereof.

Background

With the wide use of high-definition video conference solutions, people are increasingly urgent to demand for remote offices and video conferences, and video effects reach 720P, 1080P and even 4K, but the whole conference experience still cannot meet the demands of clients. On the premise that large-size screens are gradually popularized, manufacturers in the video conference industry push out far-really schemes in a dispute mode, so that the experience of participants is comprehensively improved from a pure video conference to the size of a real person 1:1 image, and face-to-face immersive communication experience is realized by matching with a sound positioning technology. However, since image acquisition is limited by the physical structure and optical field angle of the camera, a single camera cannot obtain a seamless continuous and undistorted panoramic image; the problem of discontinuous splicing caused by parallax problems can be solved by adopting a plurality of cameras for splicing, and particularly, when a large conference room is used, a large depth of field exists for participants sitting in the front row and the rear row, and the problem of discontinuous splicing can be highlighted.

Most of the existing remote real systems adopt a plurality of cameras to shoot scene images in a parallel or convergent arrangement mode. Wherein the mainstream camera solution is shown in fig. 6a and 6 b.

To illustrate why the cameras in the current tele-real system cannot obtain panoramic seamless images, we examine a two-camera system consisting of parallel cameras, as shown in fig. 7. O1 and O2 are the optical centers of the two cameras, respectively. The relation between parallax and focal length, optical center and depth of field can be obtained according to the optical imaging principle: d=fb/Z;

where d is the parallax of the overlapping parts of the two images, f is the focal length of the camera, B is the optical center distance of the camera, Z is the depth of field of the shot, and the relation of the converging camera system is similar. It can be seen that the parallax of the image is related to the optical center spacing and depth of field. Because the optical centers of the cameras are arranged inside the cameras, the optical center distance B of the two cameras cannot be 0 only by arranging the positions of the cameras, and therefore, a panoramic image formed by seamlessly splicing a plurality of camera images cannot be obtained by using the cameras in the existing remote system.

In the prior art, the common optical center stitching is a main scheme for solving the problem that a plurality of camera images stitch panoramic images seamlessly, but due to the influence of optical parameters of a lens, glass refraction and object surface reflection, the image stitching position is easy to generate a eclosion dip-dye band phenomenon, and the dip-dye band cannot be used in stitching, so that the stitching effect is poor, and a high-quality panoramic image cannot be obtained. The method adopted in the prior art is a method of attaching a polaroid to a reflector surface to filter stray light or adding a heterochromatic baffle at the junction, and then cutting off the edge by a subsequent software algorithm, so that partial images are lost, and the cost is high.

In addition, the problem of how to completely reserve a speaker in a far-end picture when a person stands for speaking in a conference is solved. In the prior art, a mode of cutting after an image is acquired by a large-size image sensor is generally adopted, or a zoom camera is adopted to reduce the image in a certain proportion so as to obtain more image range, but the cost of the two schemes is higher.

Disclosure of Invention

In order to solve the problems in the prior art, the invention provides a common-light center stitching method and a device thereof, wherein mark points are determined on stitching lines among stitching reflectors, a stitching camera acquires a common-light center image with the mark points, and the image stitching position is roughly determined; and then the panoramic image acquired by the panoramic camera is combined to accurately calibrate the image splicing position, so that the low-cost accurate determination of the image splicing position of the co-light heart is effectively realized, the problem of eclosion dip-dyeing of the image splicing position is solved, and the high-quality co-light heart panoramic image is obtained.

In a first aspect, the invention provides a co-optical center stitching method, which is characterized in that a co-optical center stitching device is provided, the co-optical center stitching device comprises a co-optical center image acquisition array and a physical stitching adjustment mechanism for adjusting the co-optical center image acquisition array, wherein the co-optical center image acquisition array comprises at least 3 stitching cameras, at least 1 panoramic camera and at least 3 stitching reflectors which are in one-to-one correspondence with the at least 3 stitching cameras, and mark points marked by colors are arranged on the stitching reflectors; the method comprises the following steps:

Based on the azimuth and the rotation angle of the acquisition area, the relative positions between the spliced cameras and the corresponding spliced reflectors are adjusted by using the physical spliced adjusting mechanism, so that an adjusted co-optical heart image acquisition array is obtained;

acquiring an image by using the adjusted co-optical heart image acquisition array to obtain a co-optical heart image with a mark point;

roughly determining a first splicing line according to the mark points on the common optical center image;

acquiring an image by using the panoramic camera to obtain a panoramic image;

comparing the co-illuminated heart image with the panoramic image to obtain an image coincidence degree;

accurately determining a second stitching line in the vicinity of the first stitching line based on the image overlap ratio;

and cutting the spliced edge of the co-optical center image according to the second splicing line to obtain a cut co-optical center image, and splicing the cut co-optical center image according to the cut edge to obtain a spliced co-optical center image.

In a second aspect, the present invention further provides a co-optical center stitching device, which is characterized in that the device includes a co-optical center image acquisition array and a physical stitching adjustment mechanism, wherein the co-optical center image acquisition array includes at least 3 stitching cameras, at least 1 panoramic camera, and at least 3 stitching mirrors in one-to-one correspondence with the at least 3 stitching cameras, and the stitching mirrors have marking points marked with colors thereon;

The physical splicing adjusting mechanism is used for adjusting the relative positions between the splicing cameras and the corresponding splicing reflectors based on the azimuth and the rotation angle of the acquisition area, so as to obtain an adjusted co-optical heart image acquisition array;

the device also comprises a rough determination unit, a contact ratio comparison unit, a precise determination unit and a cutting and splicing unit; wherein the method comprises the steps of

The adjusted co-optical heart image acquisition array is used for carrying out image acquisition to obtain a co-optical heart image with mark points;

the rough determination unit is used for roughly determining a first splicing line according to the mark points on the common optical center image;

the panoramic camera is used for collecting images to obtain panoramic images;

the coincidence ratio comparing unit is used for comparing the coaxial image with the panoramic image to obtain an image coincidence ratio;

the accurate determination unit is used for accurately determining a second stitching line in the vicinity of the first stitching line based on the image overlapping ratio;

and the cutting and splicing unit is used for cutting the spliced edges of the co-optical center images according to the second splicing line to obtain cut co-optical center images, and splicing the cut co-optical center images according to the cut edges to obtain spliced co-optical center images.

According to the method and the device for splicing the co-optical centers, the mark points are determined on the splicing lines among the spliced reflectors, the spliced camera acquires the co-optical center images with the mark points, and the image splicing positions are roughly determined; and then the panoramic image acquired by the panoramic camera is combined to accurately calibrate the image splicing position, so that the low-cost accurate determination of the image splicing position of the co-light heart is effectively realized, the problem of eclosion dip-dyeing of the image splicing position is solved, and the high-quality co-light heart panoramic image is obtained. In addition, besides determining the mark point on the spliced lines between the spliced reflectors, the image spliced position, namely the first spliced line, can be determined by sticking dark films on other spliced reflectors adjacent to the selected spliced reflectors or setting a mark at the corner of the acquisition area, so as to accurately determine the second spliced line reduction range for the follow-up. In addition, the invention utilizes the flexibility of convenient replacement of the FA industrial lens, and aims at the problem that standing staff cannot go out of the frame in a conference, and adopts the FA industrial lens with the same series but different focal lengths, namely, the 12mm focal length lens is replaced by the 8mm focal length lens, so that a larger visual field range is obtained. According to the head position of a participant, particularly the head position of a standing person, the spliced co-illuminated heart image is partially sheared, so that the head position of the participant is kept within the range of 1/3-2/3 of the longitudinal upper part of a picture in a ratio of 1:1 to a real person, and the spliced co-illuminated heart image is smoothly transited and displayed on a receiving end screen; in addition, the at least 3 spliced reflectors can be respectively rotated through an electromagnet or other mechanical structures, so that the head position of a participant is positioned in the middle of the co-illuminated heart image, and the problem that standing staff cannot go out of a frame is better solved at lower cost.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a method for splicing co-optical centers provided by an embodiment of the invention;

fig. 2a and fig. 2b are block diagrams of a common optical center splicing device according to an embodiment of the present invention;

FIG. 3 is a flow chart of a method for determining a first patch cord according to an embodiment of the present invention;

FIG. 4 is a flow chart of another method for determining a first patch cord provided by an embodiment of the present invention;

FIG. 5 is a flow chart of a method for solving the problem of display of conference standing personnel according to an embodiment of the present invention;

FIGS. 6a and 6b are schematic diagrams of parallel and convergent placement of cameras in the prior art;

FIG. 7 is a schematic diagram of a prior art parallel camera acquisition principle;

FIG. 8 is a block diagram of a physical splice adjustment mechanism provided by an embodiment of the present invention;

FIG. 9 is a schematic diagram of a co-optical image acquisition array according to an embodiment of the present invention;

Fig. 10 is a graph of a co-optical center stitching collection effect provided by an embodiment of the present invention;

FIG. 11 is a schematic diagram of a common optical center splicing principle according to an embodiment of the present invention;

FIG. 12a is a schematic illustration of a co-illuminated heart image with marker points captured by an intermediate stitching camera according to an embodiment of the present invention;

FIG. 12b is a schematic view of a panoramic image captured by a panoramic camera provided by an embodiment of the present invention;

FIG. 12c is a schematic view of a cut co-optical center image captured by an intermediate stitching camera according to an embodiment of the present invention;

fig. 13 is a view showing the effect of a conference stand person according to the embodiment of the present invention;

fig. 14 is a block diagram of a common optical center splicing device according to an embodiment of the present invention.

Detailed Description

The technical scheme of the invention is further described in detail through the drawings and the embodiments.

Summary of The Invention

As described above, the invention provides a method and a device for splicing co-optical centers, which effectively realize the low-cost accurate determination of the splicing position of the co-optical center images, solve the problem of eclosion dip-dyeing of the image splicing position and obtain high-quality co-optical center panoramic images.

Exemplary method

Fig. 1 is a flowchart of a method for splicing a co-optical center, and provides a device for splicing a co-optical center, as shown in fig. 2, which includes a co-optical center image acquisition array, a physical splicing adjustment mechanism for adjusting the co-optical center image acquisition array, and an image codec processor, wherein the co-optical center image acquisition array includes at least 3 splicing cameras, at least 1 panoramic camera, and at least 3 splicing mirrors corresponding to the at least 3 splicing cameras one by one.

As shown in fig. 8, the physical splicing adjustment mechanism specifically includes: 3 groups of XYZ axial rotation holders and 1 group of Y-direction linear lifting holders. The physical splicing adjusting mechanism realizes the precise adjustment of XYZ 3 axial rotation and Y-direction linear lifting.

XZ axis angle adjustment: the XYZ axial rotation cradle head sequentially comprises an XZ axis cradle head, a middle cradle head and a Y axis cradle head from top to bottom, wherein 1 steel ball is arranged at the corner between the XZ axis cradle head and the middle cradle head, 2 groups of belleville springs are pre-fixed above the XZ axis cradle heads on two adjacent sides of the steel ball, and the belleville springs are in L-shaped layout. And penetrating the precise thread pair through the XZ-axis holder along the L-shaped direction, and screwing in the middle holder. Thus, steel balls, belleville springs and precise thread pairs are arranged in a straight line on two sides of the XZ axis holder. When the precise screw thread pair is rotated by hand, the middle cradle head can rotate under the action of the elasticity of the belleville spring by taking the steel ball as the rotation axis, and the rotation angle can be within +/-3 degrees due to the limited deformation space of the belleville spring, so that the requirement of angle adjustment of a camera can be met. After the screw thread is adjusted in place, a locking nut is arranged on the precise screw thread pair, and the screw thread pair is reversely screwed and locked by hands.

Y-axis angle adjustment: the round platform protruding below the middle holder penetrates through a round hole in the center of the Y-axis holder and is locked to a floating disc in the center of the Y-axis holder, the precise thread pair penetrates through the Y-axis holder from the right side and props against the floating disc, and the spring anti-loose screw rod penetrates through the Y-axis holder from the left side and props against the other end of the floating disc. When the precise screw pair is rotated by hand, the floating disc drives the middle rotary table to rotate in the Y axis, and the movable gap for adjusting the XZ axis is reserved between the floating disc and the inner side of the Y axis holder, so that the Y axis rotation is not influenced even if the middle rotary table is inclined.

Y-direction linear adjustment: the XZ axis holder is fastened with the sliding block by a screw, and the sliding block is locked with the dovetail groove type track by the screw after the sliding block is lifted and adjusted in place on the dovetail groove type track.

Specifically, in order to achieve the precision and flexibility of the physical splice adjustment mechanism, the pitch precision (pitch) of the precision screw pair responsible for XYZ axis rotation is 0.24mm to 0.26mm, preferably the pitch precision of the precision screw pair responsible for XYZ axis rotation is 0.25mm.

Under the lever action of the axle center of the steel ball and the belleville springs, the inclination angle of the rotary holder deflects by 0.47-0.49 degrees every time the screw pair rotates, and preferably, the inclination angle of the rotary holder deflects by 0.48 degrees every time the screw pair rotates, so that precise angle adjustment is realized.

Each XYZ axial rotation cloud platform is an independent module, and X, Y, Z axes are controlled by a hand-rotation thread pair and a reverse anti-loosening structure respectively, so that any rotating shaft is regulated to not move in other directions, and the multi-dimensional flexible regulation can be realized by matching with a Y-direction lifting platform.

As shown in fig. 9, the co-optical image acquisition array sequentially comprises, from front to back: high perspective window glass, a panoramic camera fixing seat, a spliced camera image Sensor circuit board (called Sensor board for short), a spliced camera bayonet seat, a spliced camera, a front coating spliced reflector and a reflector sealing box body; in order to not shelter from the concatenation camera and can complete shooting whole meeting place, preferably place the below of middle concatenation camera in the centre with the panoramic camera to be located the shooting blind area department of middle concatenation camera, the panoramic camera just is to high perspective window glass and adopts short burnt wide angle lens, in order to realize the shooting of meeting place panorama.

The embodiment shown in fig. 1 comprises the following steps:

s101: based on the azimuth and the rotation angle of the acquisition area, the relative positions between the spliced cameras and the corresponding spliced reflectors are adjusted by using the physical spliced adjusting mechanism, so that the adjusted co-optical heart image acquisition array is obtained.

The collecting area is an area needing to be photographed in the conference process, as shown in fig. 10, and the collecting area is three directions of the left, the middle and the right of the conference table in the figure. After the meeting acquisition area is determined, firstly setting the position of the co-optical center splicing device; secondly, the physical splicing adjusting mechanism is utilized to further adjust the image acquisition array of the co-light center, namely, the relative positions and angles between the 3 splicing cameras and the relative positions and angles between the splicing cameras and the corresponding splicing reflectors are adjusted, so that the shooting ranges of the 3 splicing cameras are exactly three directions on the left, middle and right of the conference table respectively, the shooting view angle edges of the 3 splicing cameras fall on the corner line of the conference table, namely, the 3 splicing cameras are respectively responsible for shooting the conference table with one direction, and finally, 3 virtual light centers are converged on 1 point, and the aim of the co-light center is achieved, as shown in fig. 11.

S102: at least 4 marker points are determined on the stitching line between the stitching mirrors.

As shown in fig. 11, the mark points on the splice line are point 110, point 120, point 130, and point 140, respectively.

S103: and carrying out color marking on the mark points on the stitching line so as to facilitate image recognition.

For example, the marker points are marked with red.

S104: and acquiring an image by using the adjusted co-optical heart image acquisition array to obtain a co-optical heart image with the mark points.

For example, fig. 12a is a co-centered image with a marker point taken by an intermediate stitching camera, where the stitching camera may take a portion of the image reflected by an adjacent stitching mirror, i.e., the area shown outside the dashed line, which is severely distorted by the image reflected by the adjacent stitching mirror, due to the 150 degree angle between the adjacent stitching mirror and the intermediate stitching camera. The dashed line in fig. 12a is the line of the marker points on the stitching mirror, which serves as the first stitching line for intercepting the intermediate co-centered image.

Preferably, the splicing camera adopts an FA industrial lens which has the characteristics of small image distortion and high consistency of the parameters of the angle of view, aperture and focal length, the horizontal angle of view of the 3 FA industrial lenses is 25-35 degrees, preferably, the splicing camera is respectively responsible for collecting local images with the horizontal angle of 30 degrees and the aspect ratio of 16:9, and the local images are physically spliced into global images with the aspect ratio of 48:9. The lens direction is downward opposite to the front coating reflecting mirror, is arrayed in a delta shape, and is screwed on a panoramic camera fixing seat connected with the Y-axis holder. The inclined upward viewing angles of the 3 front coating reflectors are 45 degrees, and the three front coating reflectors are closely adjacent and are distributed at an included angle of 150 degrees. The image is horizontally incident on the front coating film reflecting mirror through the high perspective window glass, is totally reflected to the lens through 45 degrees, and is projected to the Sensor plate for processing, so that a virtual optical center is formed on the rear side of the front coating film reflecting mirror through reflection of a physical optical center of the lens, and 3 virtual optical centers can be converged on 1 point through the adjustment of the position and the angle of the splicing camera through the physical splicing adjusting mechanism assembly, thereby realizing the purpose of sharing the optical center, as shown in fig. 10 and 11.

Preferably, in order to avoid the influence of the inside of the mirror enclosure on the light rays irradiated to the spliced mirrors, the inside of the mirror enclosure of the co-optical heart image acquisition array adopts a black low-reflection coating with the reflectivity lower than 2% on all the surfaces of the devices.

Preferably, the high-perspective window glass and the infrared cut-off filter IRCF of the co-light heart image acquisition array are made of high-transmittance materials and are plated with antireflection films, and after stray light in an infrared band is filtered, the visible light transmittance is up to more than 95%.

S105: and roughly determining the first splicing line according to the mark points on the co-optical center image.

And connecting the mark points on the same longitudinal direction on the common optical center image by using a straight line to obtain a first splicing line. The first stitching line is a rough cutting stitching line.

S106: and acquiring an image by using the panoramic camera to obtain a panoramic image, as shown in fig. 12 b.

Preferably, the panoramic camera adopts M12 monitoring lens, is fixed to the top of the reflector sealing box body through the panoramic camera fixing seat, is small in size and size, does not shield the image of the spliced camera, and is placed in the middle of the fixed angle in the forward direction of the lens. The panoramic camera collects a horizontal included angle of 85-95 degrees, preferably, the panoramic camera collects a panoramic image with a horizontal included angle of 90 degrees and an aspect ratio of 16:9, which just covers the horizontal view angles of 3 splicing cameras, and the panoramic image can be used as fusion splicing verification of an image software algorithm to obtain a more accurate seamless splicing effect.

S107: and comparing the co-illuminated heart image with the panoramic image to obtain the image coincidence degree.

Specifically, the image processing software may be used to perform the contrast of the coincidence therebetween, for example, the contrast of the coincidence between fig. 12a and fig. 12 b.

S108: a second stitching line is precisely determined in the vicinity of the first stitching line based on the image overlap ratio.

And precisely searching a second stitching line with high image overlap ratio on one side and non-overlapping other side near the first stitching line by using image processing software. The second splice line is an accurate cut splice line.

S109: and cutting the spliced edge of the co-optical center image according to the second splicing line to obtain a cut co-optical center image, as shown in fig. 12 c. And splicing the three cut co-optical center images according to the edges of the three cut co-optical center images to obtain the spliced co-optical center images.

And precisely cutting off the edge dip-dyed strips which are acquired in the co-optical center images according to the second stitching lines, then performing seamless stitching, and finally obtaining the stitched co-optical center images, namely the panoramic co-optical center images with high quality.

In a word, the mark points are determined on the spliced lines between the spliced reflectors, and the spliced camera acquires the co-optical heart image with the mark points to roughly determine the splicing position of the images; and then the panoramic image acquired by the panoramic camera is combined to accurately calibrate the image splicing position, so that the low-cost accurate determination of the co-light heart image splicing position is effectively realized. For example, after the second stitching lines are respectively determined in the common-light center images shot by each stitching camera, whether the second stitching lines determined by the adjacent common-light center images are overlapped or not is determined on the panoramic image, if not, fine adjustment is performed to keep that the cut common-light center image stitching lines have no missing images and no overlapped dip-dyeing strips, so that the problem of the eclosion dip-dyeing strips at the image stitching positions is solved, and the high-quality common-light center panoramic image is obtained.

In addition to the embodiment of the first patch cord obtained in steps S102 to S105 in the example shown in fig. 1, as shown in fig. 3, as another alternative embodiment, the embodiment includes:

s301: one of the spliced reflectors is selected, and other spliced reflectors adjacent to the selected spliced reflector are respectively stuck with dark films.

For example, when the middle split joint mirror in fig. 11 is selected, the dark films are attached to the surfaces of the left and right split joint mirrors adjacent to the middle split joint mirror, and it should be noted that the dark films are to be completely attached to the split joint mirrors, especially the edges are to be completely clamped on the split joint lines, and the purpose of this step is to determine the boundary between the selected split joint mirror and the adjacent split joint mirrors for precise control.

S302: and acquiring an image by using the adjusted co-optical heart image acquisition array to obtain a co-optical heart image with the mark line.

The mark line is located at the position of the selected spliced reflecting mirror and the boundary line between the selected spliced reflecting mirror and the adjacent spliced reflecting mirrors.

S303: and roughly determining the first splicing line according to the mark line on the common optical center image.

The first splicing line is the position of the marking line.

After the first splicing line of the middle splicing camera is determined, determining the first splicing lines for the splicing cameras on the left side and the right side, namely, attaching a dark color film on the surface of the middle splicing reflecting mirror, removing the dark color films of the splicing reflecting mirrors on the left side and the right side, and determining the first splicing lines of the splicing cameras on the left side and the right side according to a similar method. Compared with the method of sticking the polarizing film on the spliced reflecting mirror in the prior art, the cost is lower, and the light flux is greatly reduced due to the polarizing film, so that the high-quality imaging is not facilitated.

As shown in fig. 4, as a further alternative embodiment, the embodiment for obtaining the first patch cord includes:

s401: and setting a marker at the corner of the acquisition region.

S402: and acquiring by using the adjusted co-optical heart image acquisition array to obtain the co-optical heart image with the marker.

S403: and roughly determining the first splicing line according to the marker on the co-optical center image.

Markers are arranged at the collecting areas, namely, the

points

180, 160, 150 and 170 at the corners of the conference table, as shown in fig. 10, such as 4 wood sticks or 4 light sources are vertically inserted; acquiring the range of the conference table by using a spliced camera to obtain a co-illuminated heart image with a marker stick/light source; the first splice line is obtained by connecting points of the same longitudinal wood sticks/light sources.

In short, in addition to determining the mark point on the spliced lines between the spliced reflectors, the invention roughly determines the image spliced position, namely the first spliced line, the invention can also determine the first spliced line by sticking dark films on other spliced reflectors adjacent to the selected spliced reflectors or setting a mark at the corner of the acquisition area, so as to accurately determine the second spliced line shrinkage range for the follow-up.

In addition, in order to solve the problem that a standing person cannot go out of a frame when a person stands in a conference, as shown in fig. 5, the method of the present invention further includes:

s501: and acquiring the spliced co-optical center image with a larger field range by adopting a short-focal-length spliced camera lens.

The short focal length is 7-9 mm, and preferably, an 8mm focal length lens is adopted.

S502: judging the head position of the participant.

S503: and carrying out local shearing on the spliced co-optical heart images so that the position of the head of the participant is positioned in the middle of the co-optical heart images after local shearing.

The middle part of the partially sheared co-optical heart image is in the range of 1/3-2/3 of the vertical direction.

S504: the cut co-photopic image was displayed in a 1:1 ratio to the human.

As shown in fig. 13, in order to obtain a better video presentation effect, no matter whether a person standing in the transmitting end conference place a exists or not, in the video picture received by the receiving end conference place B, the head of the participant always keeps the 1:1 ratio with the real person within the range of 1/3-2/3 of the vertical direction of the picture.

In a word, the invention utilizes the flexibility of convenient replacement of the FA industrial lens, and adopts the FA industrial lens with the same series but different focal lengths for the customers, namely, the 12mm focal length lens of the spliced camera is replaced by the 8mm focal length lens, thereby obtaining a larger visual field range. According to the head position of a participant, particularly the head position of a standing person, the spliced co-illuminated heart image is partially sheared, so that the head position of the participant is kept within the range of 1/3-2/3 of the longitudinal upper part of a picture with the ratio of 1:1 to a real person, and the head position of the participant is smoothly transitionally displayed on a receiving end screen, compared with the traditional method for reducing the image by adopting a zoom camera to a certain proportion, the cost is lower, and if the visual difference is larger in the visual sense of the scaled-down picture, the transition is not smooth, therefore, the scheme of the invention has low cost and better meets the requirements of clients.

Besides the short-focal-length spliced camera lens used in the embodiment, the problem that standing personnel cannot go out of a frame is solved. Because an included angle is formed between the at least 3 spliced reflectors, the at least 3 spliced reflectors cannot be integrally rotated, and each spliced reflector needs to be rotated respectively to obtain an accurate common-optical center image.

Specifically, the position of the head of the participant can be firstly determined, and when the participant is determined to stand, the participant is rotated, for example, after the standing participant is identified by software, electromagnets in a physical splicing adjusting mechanism are automatically and rapidly started to rotate the at least 3 splicing reflectors respectively; the rotation of the physical splicing adjusting mechanism can be automatically started in a self-defined mode, for example, the rotation option of the physical splicing adjusting mechanism is clicked at any time or a key on the remote controller is clicked to trigger the rotation operation.

Because the positions of the spliced reflecting mirrors are different in the standing state and the sitting state, the co-illuminated heart images shot by the spliced cameras are different, and correspondingly, the spliced lines used for cutting and splicing in the two states are also different, so that the corresponding second spliced lines are required to be searched and stored by using the initialization steps in the two states respectively. And selecting the corresponding second spliced lines for cutting and splicing based on the state of the spliced reflecting mirror, so that a high-quality spliced co-optical heart image can be obtained.

Compared with the embodiment of adopting the short-focal-length spliced camera lens, the embodiment of rotating the spliced reflecting mirror has the advantages that the cost is lower and the implementation is easier because the image sensor with higher cost is not required to be replaced and the image ratio is 4:3.

Exemplary apparatus

Correspondingly, the embodiment of the invention also provides a device for splicing the co-optical centers. Fig. 14 is a block diagram of a co-optical center splicing apparatus 100 according to an embodiment of the present invention, as shown in fig. 14, a system 100 provided in this embodiment includes:

the system comprises a co-light heart image acquisition array 110 and a physical stitching adjustment mechanism 120, wherein the co-light heart image acquisition array 110 comprises at least 3 stitching cameras 111, at least 1 panoramic camera 112 and at least 3 stitching reflectors 113 which are in one-to-one correspondence with the at least 3 stitching cameras 111, and the stitching reflectors 113 are provided with mark points marked by colors;

the physical stitching adjustment mechanism 120 is configured to adjust a relative position between the stitching cameras 111 and the corresponding stitching mirror 113 based on the azimuth and the rotation angle of the acquisition area, so as to obtain an adjusted co-optical image acquisition array 110;

the apparatus 100 further comprises an image codec processor 130, the image codec processor 130 comprising a coarse determination unit 131, a coincidence contrast unit 132, a fine determination unit 133 and a cropping unit 134; wherein the method comprises the steps of

The adjusted co-optical heart image acquisition array 110 is used for acquiring images to obtain co-optical heart images with mark points;

the rough determining unit 131 is configured to roughly determine a first stitching line according to the marker points on the co-optical center image;

the panoramic camera 112 is used for collecting images to obtain panoramic images;

the coincidence ratio comparing unit 132 is configured to compare the co-optical heart image with the panoramic image to obtain an image coincidence ratio;

the accurate determination unit 133 is configured to accurately determine a second stitching line in the vicinity of the first stitching line based on the image overlap ratio;

the cutting and stitching unit 134 is configured to cut the stitched edge of the co-optical center image according to the second stitching line, obtain a cut co-optical center image, and stitch the cut co-optical center image according to the cut edge, so as to obtain a spliced co-optical center image.

Selecting one of the spliced reflectors 113, and respectively attaching dark films to other spliced reflectors 113 adjacent to the selected spliced reflector 113;

the adjusted co-optical heart image acquisition array 110 is further used for performing image acquisition to obtain a co-optical heart image with a sign line;

the rough determining unit 131 is further configured to roughly determine the first stitching line according to the sign line on the co-optical center image.

Setting a marker at a corner of the acquisition region;

the adjusted co-optical heart image acquisition array 110 is further used for performing image acquisition to obtain a co-optical heart image with the marker;

the rough determining unit 131 is further configured to rough determine the first stitching line according to the marker on the co-optical center image.

The stitched camera 111 further comprises a short focal length lens 141 for obtaining the stitched co-optical center image with a larger field of view;

the image codec processor 130 further includes a judging unit 135, a local clipping unit 136, and a display unit 137; wherein the method comprises the steps of

The judging unit 135 is used for judging the head position of the participant;

the local shearing unit 136 is configured to locally shear the spliced co-optical center image, so that the head position of the participant is located in the middle of the locally sheared co-optical center image;

the display unit 137 is configured to display the cut co-illuminated heart image in a ratio of 1:1 with the real person.

The middle part of the partially sheared co-optical heart image is in the range of 1/3-2/3 of the vertical direction;

the short focal length is 7-9 mm.

The stitching cameras 111 are respectively responsible for acquiring local images with the horizontal included angles of 25-35 degrees and the aspect ratio of 16:9, and physically stitch the local images into global images with the aspect ratio of 48:9.

The panoramic camera 112 collects panoramic images with a horizontal included angle of 85-95 degrees and an aspect ratio of 16:9.

The circumferential adjustment precision of the XYZ-axis rotating precise thread pair of the physical splicing adjustment mechanism 120 is 0.24-0.26 mm;

the screw pair rotates once, so that the inclination angle of the rotary holder of the physical splicing adjusting mechanism 120 deflects by 0.47-0.49 degrees.

Each XYZ axial rotation cloud platform is an independent module, and X, Y, Z axes are controlled by a hand-screw thread pair and a reverse anti-loosening structure.

Inside the mirror enclosure of the co-illuminated heart image acquisition array 110, all device surfaces are coated with a black low-reflection coating with a reflectivity of less than 2%.

The high-perspective window glass of the co-light heart image acquisition array 110 and the infrared cut-off filter IRCF are made of high-transmittance materials and are plated with antireflection films.

The physical stitching adjustment mechanism 120 is further configured to rotate the stitching mirror 113, so that the position of the participant's head is located in the middle of the co-optical center image.

The image codec processor 130 further includes a judging unit 135 for judging the head position of the participant;

when it is determined that the participant stands, the physical stitching adjustment mechanism 120 is further configured to rotate the stitching mirror 113, so that the position of the participant's head is located in the middle of the co-optical center image.

The physical stitching adjustment mechanism 120 is further configured to rotate the at least 3 stitching mirrors 113 by using electromagnets, respectively.

It should be noted that although the operations of the co-optical center stitching method of the present invention are depicted in the drawings in a particular order, this is not required or implied that these operations must be performed in that particular order or that all of the illustrated operations must be performed to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform.

Furthermore, although several units, or modules, of a common optical center splicing device are mentioned in the above detailed description, such a division is merely exemplary and not mandatory. Indeed, the features and functions of two or more modules described above may be embodied in one module in accordance with embodiments of the present invention. Conversely, the features and functions of one module described above may be further divided into a plurality of modules to be embodied.

While the spirit and principles of the present invention have been described with reference to several particular embodiments, it is to be understood that the invention is not limited to the disclosed embodiments nor does it imply that features of the various aspects are not useful in combination, nor are they useful in any combination, such as for convenience of description. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

The invention provides:

1. a method for splicing a co-optical center is characterized in that a device for splicing the co-optical center is provided, and the device comprises a co-optical center image acquisition array and a physical splicing adjustment mechanism for adjusting the co-optical center image acquisition array, wherein the co-optical center image acquisition array comprises at least 3 splicing cameras, at least 1 panoramic camera and at least 3 splicing reflectors which are in one-to-one correspondence with the at least 3 splicing cameras, and mark points marked by colors are arranged on the splicing reflectors; the method comprises the following steps:

acquiring an image by using the panoramic camera to obtain a panoramic image;

2. The stitching method of claim 1 wherein after the step of obtaining an adjusted co-optical cardiac image acquisition array, the method further comprises:

selecting one of the spliced reflectors, and respectively sticking dark films to other spliced reflectors adjacent to the selected spliced reflector;

acquiring an image by using the adjusted co-optical heart image acquisition array to obtain a co-optical heart image with a mark line;

and roughly determining the first splicing line according to the mark line on the common optical center image.

3. The stitching method of claim 1 or 2, wherein after the step of obtaining an adjusted co-optical cardiac image acquisition array, the method further comprises:

setting a marker at a corner of the acquisition region;

acquiring by using the adjusted co-optical heart image acquisition array to obtain a co-optical heart image with the marker;

And roughly determining the first splicing line according to the marker on the co-optical center image.

4. The splicing method according to any one of claims 1 to 3, characterized in that the method further comprises:

a spliced camera lens with a short focal length is adopted to obtain the spliced co-optical center image with a larger view field range;

judging the head position of the participant;

locally shearing the spliced co-optical heart image to enable the head position of the participant to be positioned in the middle of the locally sheared co-optical heart image;

the cut co-photopic image was displayed in a 1:1 ratio to the human.

5. The splicing method according to claim 4, wherein the middle of the partially sheared co-optical heart image is in a range of 1/3 to 2/3 in the vertical direction;

the short focal length is 7-9 mm.

6. The stitching method according to any one of claims 1-5, wherein the stitching cameras are respectively responsible for acquiring local images with a horizontal included angle of 25 ° to 35 °, an aspect ratio of 16:9, and physically stitching the local images into global images with an aspect ratio of 48:9.

7. The stitching method according to any one of claims 1-6, wherein the panoramic camera captures panoramic images having a horizontal included angle of 85 ° to 95 ° and an aspect ratio of 16:9.

8. The splicing method according to any one of claims 1 to 7, wherein the precision thread pair of XYZ-axis rotation of the physical splice adjustment mechanism has a circumferential adjustment precision of 0.24mm to 0.26mm;

and each time the screw pair rotates, the inclination angle of the rotary holder of the physical splicing adjusting mechanism deflects by 0.47-0.49 degrees.

9. The splicing method according to claim 8, wherein each XYZ axial rotation holder is an independent module, and X, Y, Z axes are controlled by a pair of manual threads and a reverse anti-loosening structure.

10. The method of any one of claims 1-9, wherein a black low-reflection coating with a reflectivity of less than 2% is used on all device surfaces inside a mirror enclosure of the co-optical heart image acquisition array.

11. The splicing method according to any one of claims 1 to 10, wherein the high-transmittance windowpane of the co-optical heart image acquisition array and the infrared cut filter IRCF are both made of a high-transmittance material and are coated with an antireflection film.

12. The splicing method according to any one of claims 1 to 3, characterized in that the method further comprises:

and rotating the spliced reflecting mirror to enable the position of the head of the participant to be positioned in the middle of the co-optical heart image.

13. The method of stitching according to claim 12, wherein the method further comprises:

judging the head position of the participant;

when the participant is judged to stand, the spliced reflecting mirror is rotated, so that the position of the participant head is positioned in the middle of the co-optical heart image.

14. The method of claim 12 or 13, wherein the step of rotating the split mirror specifically includes:

and respectively rotating the at least 3 spliced reflecting mirrors by utilizing an electromagnet.

15. The device is characterized by comprising a co-optical center image acquisition array and a physical splicing adjustment mechanism, wherein the co-optical center image acquisition array comprises at least 3 splicing cameras, at least 1 panoramic camera and at least 3 splicing reflectors which are in one-to-one correspondence with the at least 3 splicing cameras, and mark points marked by colors are arranged on the splicing reflectors;

the panoramic camera is used for collecting images to obtain panoramic images;

16. The splicing device according to claim 15, wherein one of the spliced reflectors is selected, and other spliced reflectors adjacent to the selected spliced reflector are respectively attached to the dark film;

The adjusted co-optical heart image acquisition array is also used for carrying out image acquisition to obtain a co-optical heart image with a mark line;

the rough determination unit is further used for roughly determining the first stitching line according to the mark line on the common optical center image.

17. The splicing device of claim 15 or 16, wherein a marker is provided at a corner of the acquisition region;

the adjusted co-optical heart image acquisition array is also used for carrying out image acquisition to obtain a co-optical heart image with the marker;

the rough determination unit is further used for roughly determining the first stitching line according to the marker on the co-optical center image.

18. The stitching device as recited in any one of claims 15-17 wherein the stitched camera further comprises a short focal length lens for obtaining the stitched co-centered image over a greater field of view;

the device also comprises a judging unit, a local shearing unit and a display unit; wherein the method comprises the steps of

The judging unit is used for judging the head position of the participant;

the local shearing unit is used for locally shearing the spliced co-optical heart images so that the head position of the participant is positioned in the middle of the locally sheared co-optical heart images;

The display unit is used for displaying the sheared co-light heart images according to the ratio of 1:1 with the real person.

19. The splicing device according to claim 18, wherein the middle of the partially sheared co-optical heart image is in a range of 1/3 to 2/3 in a vertical direction;

the short focal length is 7-9 mm.

20. The stitching device according to any one of claims 15-19, wherein the stitching cameras are each responsible for acquiring local images with a horizontal included angle of 25 ° to 35 °, an aspect ratio of 16:9, and physically stitching global images with an aspect ratio of 48:9.

21. The stitching device of any one of claims 15-20 wherein the panoramic camera captures panoramic images having a horizontal included angle of 85 ° to 95 ° and an aspect ratio of 16:9.

22. The splicing device according to any one of claims 15 to 21, wherein the precision thread pair of XYZ-axis rotation of the physical splicing adjustment mechanism has a circumferential adjustment accuracy of 0.24mm to 0.26mm;

23. The splicing device of claim 22, wherein each XYZ axial rotation head is an independent module, and each X, Y, Z axis is controlled by a pair of manual threads and a reverse anti-loosening structure.

24. The stitching device of any one of claims 15-23 wherein a black low reflection coating having a reflectivity of less than 2% is employed on all device surfaces within a mirror enclosure of the co-optical image acquisition array.

25. The splicing device according to any one of claims 15 to 24, wherein the high-transmission window glass and the infrared cut-off filter IRCF of the co-optical heart image acquisition array are both made of a high-transmission material and are coated with an antireflection film.

26. The stitching device of any one of claims 15-17 wherein the physical stitching adjustment mechanism is further configured to rotate the stitching mirror such that a participant's head position is centered on the co-illuminated heart image.

27. The splicing device of claim 26, further comprising a determining unit configured to determine a position of a head of the participant;

when the participant is judged to stand, the physical splicing adjusting mechanism is further used for rotating the splicing reflecting mirror, so that the position of the head of the participant is located in the middle of the co-light heart image.

28. The splice device of either claim 26 or claim 27, wherein the physical splice adjustment mechanism is further configured to rotate the at least 3 splice mirrors with electromagnets, respectively.

Claims

acquiring an image by using the panoramic camera to obtain a panoramic image;

3. The stitching method according to claim 1 or 2, wherein after the step of obtaining an adjusted co-optical heart image acquisition array, the method further comprises:

setting a marker at a corner of the acquisition region;

4. A splicing method according to any of claims 1-3, characterized in that the method further comprises:

judging the head position of the participant;

the cut co-photopic image was displayed in a 1:1 ratio to the human.

5. The stitching method of claim 4 wherein the center of the partially sheared image is in the range of 1/3 to 2/3 in the vertical direction;

the short focal length is 7-9 mm.

6. The stitching method according to any one of claims 1-5, wherein the stitching cameras are respectively responsible for acquiring local images with a horizontal included angle of 25 ° to 35 °, an aspect ratio of 16:9, and physically stitching global images with an aspect ratio of 48:9.

8. The splicing method according to any one of claims 1 to 7, wherein the pitch precision of the XYZ-axis rotating precision screw pair of the physical splice adjustment mechanism is 0.24mm to 0.26mm;

9. The method according to claim 8, wherein each XYZ axial rotation holder is an independent module, and X, Y, Z axes are controlled by a pair of manual threads and a reverse anti-loosening structure.

10. The device is characterized by comprising a co-optical center image acquisition array and a physical splicing adjustment mechanism, wherein the co-optical center image acquisition array comprises at least 3 splicing cameras, at least 1 panoramic camera and at least 3 splicing reflectors which are in one-to-one correspondence with the at least 3 splicing cameras, and mark points marked by colors are arranged on the splicing reflectors;

the panoramic camera is used for collecting images to obtain panoramic images;