CN213522183U

CN213522183U - Image stitching system

Info

Publication number: CN213522183U
Application number: CN202022796432.1U
Authority: CN
Inventors: 张华宾; 张焱; 林铭
Original assignee: Beijing Dushi Technology Co ltd
Current assignee: Beijing Dushi Technology Co ltd
Priority date: 2020-11-27
Filing date: 2020-11-27
Publication date: 2021-06-22
Anticipated expiration: 2030-11-27

Abstract

The application discloses image mosaic system includes: the system comprises a plurality of video acquisition cards (10 a-10 f) and a first video splicing subsystem (20) in communication connection with the video acquisition cards (10 a-10 f), wherein the video acquisition cards (10 a-10 f) are respectively connected with a plurality of corresponding cameras and are used for receiving multiple paths of video images from the cameras and splicing the multiple paths of video images to generate a first spliced video image; and the first video splicing subsystem (20) is used for splicing the multi-path first spliced video images received from the plurality of video acquisition cards (10 a-10 f) to generate a second spliced video image.

Description

Image stitching system

Technical Field

The application relates to the technical field of video image processing, in particular to an image splicing system.

Background

With the development of society, in order to meet the requirements of people for shooting panoramic pictures, under the condition that a single camera cannot shoot panoramic pictures, a plurality of cameras are required to shoot shot images at different angles. And then splicing the shot images at different angles to obtain a final panoramic image to be displayed to a user. In order to ensure the picture quality and the definition of the spliced panoramic image in the process of splicing the high-definition images, the high-definition images shot by a plurality of cameras need to be spliced by strong calculation power. But the higher the computing power, the greater the power consumption, and the greater the pressure on the server. In this case, in order to splice up to tens of video streams, the server needs to configure an accelerator card such as a GPU, which results in excessive power consumption, and the device is bulky and is not favorable for installation and use.

Aiming at the technical problems that the power consumption is large and the pressure caused by a server is large due to the fact that the quality of a spliced panoramic image is guaranteed by splicing a plurality of high-definition images by using a strong calculation force in the existing panoramic video splicing system in the prior art, an effective solution is not provided at present.

SUMMERY OF THE UTILITY MODEL

The utility model provides an image mosaic system to at least, solve the current panoramic video mosaic system who exists among the prior art and use powerful calculation power to splice the quality of assurance concatenation back panoramic picture to a plurality of high definition images, caused the big and big technical problem of pressure that the server caused of consumption.

According to an aspect of the present application, there is provided an image stitching system, including: the video acquisition cards are respectively connected with the corresponding cameras and used for receiving multiple paths of video images from the cameras and splicing the multiple paths of video images to generate a first spliced video image; and the first video splicing subsystem is used for splicing the multiple paths of first spliced video images received from the multiple video acquisition cards to generate a second spliced video image.

Optionally, the first video stitching subsystem comprises: a plurality of first level video splicing cards; the first-stage video splicing cards are respectively in communication connection with the corresponding video acquisition cards and are used for splicing multiple paths of first spliced video images received from the corresponding video acquisition cards to generate third spliced video images; and the second-level video splicing cards are in communication connection with the first-level video splicing cards and are used for splicing multiple paths of third spliced video images received from the first-level video splicing cards to generate second spliced video images.

Optionally, the video capture card is provided with: the image input interface is connected with the corresponding cameras; the image processor is connected with the image input interface; the first image stitching module is connected with the image processor; and the video output interface is connected with the first image splicing module.

Optionally, the first-level video stitching card is provided with: the first video input interface is in communication connection with the video output interface of the corresponding video acquisition card; the second image splicing module is connected with the first video input interface; the first video coding and decoding module is connected with the second image splicing module; and the first network port is connected with the first video coding and decoding module.

Optionally, the second level video stitching card is provided with: a second network port; the second video coding and decoding module is connected with the second network port; and the third image splicing module is connected with the second video coding and decoding module.

Optionally, the method further comprises: and the network switch is in communication connection with the first network port and the second network port.

Optionally, the method further comprises: a terminal device communicatively connected to the network switch.

Optionally, the video capture card is further provided with: the video coding module is connected with the first image splicing module; and the third network port is connected with the video coding module and is in communication connection with the network switch.

Optionally, the video capture card is further provided with a first data transmission interface, and the first data transmission interface is connected with the video coding module; the first-level video splicing card is also provided with a second data transmission interface, and the second data transmission interface is connected with the first video coding and decoding module; and/or the second-level video splicing card is also provided with a third data transmission interface which is connected with the second video coding and decoding module.

Optionally, the method further comprises: and the second video splicing subsystem is in communication connection with the network switch, is configured to communicate with the plurality of video acquisition cards through the network switch, and is configured to splice the plurality of paths of video images received from the plurality of video acquisition cards to generate a fourth spliced video image.

Optionally, the second video stitching subsystem comprises a video stitching card, and the video stitching card of the second video stitching subsystem comprises: the fourth network port is in communication connection with the network switch; the third video coding and decoding module is connected with the fourth network port; and the fourth image splicing module is connected with the third video coding and decoding module.

Optionally, the method further comprises: and the video acquisition card is provided with a line/field synchronizing signal transceiver which is used for receiving the line synchronizing signal and/or the field synchronizing signal sent by the line/field synchronizing signal generator and sending the line synchronizing signal and/or the field synchronizing signal to the plurality of video cameras connected with the video acquisition card.

Therefore, according to the image splicing system provided by the embodiment of the application, the video capture card is adopted to realize the first-stage splicing of the multiple paths of video images. And then, realizing second-stage splicing of the plurality of first spliced video images through the first video splicing subsystem to obtain a final second spliced video image, namely a wide area image. And the video images are further spliced in a grading manner in the first video splicing subsystem, compared with a mode of directly splicing all the video images acquired by the plurality of cameras, the mode of directly transmitting all the video images acquired by the plurality of cameras to the server for splicing achieves the technical effects of reducing the power consumption of the server and avoiding overlarge pressure of the server due to the fact that the video acquisition card and the first video splicing subsystem are used for splicing the acquired video images and compared with a mode of directly transmitting all the video images acquired by the plurality of cameras to the server for splicing. Meanwhile, the problem that the size of equipment is large and the equipment is not favorable for use and installation due to overhigh cost of the server is avoided. And then the technical problems that the power consumption is large and the pressure caused by a server is large due to the fact that the quality of the spliced panoramic image is guaranteed by splicing a plurality of high-definition images by using a strong calculation force of an existing panoramic video splicing system in the prior art are solved. In addition, the present application provides a second video stitching subsystem for local video image stitching, thereby facilitating a user to view captured local video images in real time.

The above and other objects, advantages and features of the present application will become more apparent to those skilled in the art from the following detailed description of specific embodiments thereof, taken in conjunction with the accompanying drawings.

Drawings

Some specific embodiments of the present application will be described in detail hereinafter by way of illustration and not limitation with reference to the accompanying drawings. The same reference numbers in the drawings identify the same or similar elements or components. Those skilled in the art will appreciate that the drawings are not necessarily drawn to scale. In the drawings:

FIG. 1 is a schematic diagram of an image stitching system according to an embodiment of the present application;

fig. 2A is a schematic view of a camera arrangement of a wide area shooting device according to an embodiment of the present application;

FIG. 2B is a schematic diagram of a second stitched video image after stitching according to an embodiment of the present application;

FIG. 3 is a schematic diagram of an internal structure of a video capture card according to an embodiment of the present application;

fig. 4 is a schematic internal structure diagram of a first-level video splicing card according to an embodiment of the present application;

fig. 5 is a schematic internal structure diagram of a second-level video splicing card according to an embodiment of the present application;

FIG. 6 is a schematic diagram of another configuration of an image stitching system according to an embodiment of the present application;

fig. 7 is a schematic diagram of a splicing process of a video splicing card according to an embodiment of the present application;

fig. 8 is a schematic diagram of an internal structure of a video splicing card according to an embodiment of the present application; and

fig. 9 is a schematic diagram of a mechanism for synchronous acquisition by multiple cameras according to an embodiment of the present application.

Detailed Description

It should be noted that, in the present disclosure, the embodiments and features of the embodiments may be combined with each other without conflict. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.

In order to make the technical solutions of the present disclosure better understood by those skilled in the art, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only some embodiments of the present disclosure, not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.

It should be noted that the terms "first," "second," and the like in the description and claims of the present disclosure and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances for describing the embodiments of the disclosure herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.

Fig. 1 is a schematic diagram of an image stitching system according to an embodiment of the present application, and referring to fig. 1, the image stitching system includes: the video splicing system comprises a plurality of video acquisition cards 10 a-10 f and a first video splicing subsystem 20 in communication connection with the video acquisition cards 10 a-10 f, wherein the video acquisition cards 10 a-10 f are respectively connected with a plurality of corresponding cameras and are used for receiving multi-channel video images from the plurality of cameras and splicing the multi-channel video images to generate a first spliced video image; and the first video stitching subsystem 20 is used for stitching the multiple paths of first stitched video images received from the multiple video capture cards 10 a-10 f to generate a second stitched video image.

As described in the background art, in order to ensure the picture quality and the definition of the stitched panoramic image in the process of high-definition image stitching, a strong calculation force is required to stitch the high-definition images shot by the plurality of cameras. In this case, in order to splice up to tens of video streams, the server needs to configure an accelerator card such as a GPU, which results in excessive power consumption, and the device is bulky and is not favorable for installation and use.

In view of the above, referring to fig. 1 and 2, the image stitching system provided by the present application includes a plurality of video capture cards 10 a-10 f. The video capture cards 10a to 10f may be, for example, multi-view capture cards, so that the video capture cards 10a to 10f may be respectively connected to a plurality of corresponding cameras (for example, cameras in a wide area shooting device), and are configured to receive multiple paths of video images from the plurality of cameras and to stitch the multiple paths of video images to generate a first stitched video image. Wherein each video capture card 10 a-10 f can be connected to 2-4 cameras.

Specifically, fig. 2A shows a camera arrangement diagram of a wide area shooting device including a plurality of cameras, and referring to fig. 1 and 2A, for example, the wide area shooting device is provided with 3 rows and 8 columns of cameras 1 to 24, and then each video capture card 10a to 10f can be connected with 4 cameras, for example. For example, referring to FIG. 1, a video capture card 10a is connected to cameras 1-4, a video capture card 10b is connected to cameras 9-12, and so on. And the video capture card 10a can receive 4 paths of video images from the cameras 1-4, and then the video capture card 10a splices the received 4 paths of video images to generate a first spliced video image spliced together by the 4 paths of video images. In addition, the video capture card 10a may be referred to for the connection between the other video capture cards 10 b-10 f and the camera and the operation of splicing the received multiple video images, and thus, the description thereof is omitted. Therefore, 6 first spliced video images corresponding to the cameras 1-24 can be obtained through the video acquisition cards 10 a-10 f.

In addition, the number of the video capture cards 10a to 10f is not limited to 6, and may be set according to the number of cameras of the wide area shooting device.

Further, referring to fig. 1, the image stitching system further includes a first video stitching subsystem 20 communicatively connected to the video capture cards 10a to 10f, where the first video stitching subsystem 20 is configured to stitch multiple paths of first stitched video images received from the video capture cards 10a to 10f to generate a second stitched video image.

Specifically, referring to fig. 1, for example, the first video stitching subsystem 20 may receive 6 first stitched video images from the video capture cards 10a to 10f, and then stitch the 6 first video images to generate a second stitched video image. Fig. 2B shows a schematic diagram of a second stitched video image, which is shown with reference to fig. 2B, wherein the second stitched video image may be, for example, a wide area image (including a panoramic image) that is finally required.

In addition, the video capture cards 10a to 10f (which may be proprietary image processors, for example) and the stitching cards used by the first video stitching subsystem 20 (which may be proprietary image stitching processors, for example) are all based on proprietary SoC systems, and are used for processing multiple video captures and image stitching, which has the technical effects of low power consumption and high integration level.

Therefore, the image stitching system provided by the embodiment of the application adopts the video capture cards 10 a-10 f to realize the first-stage stitching of the multi-channel video images. Then, the second-stage stitching of the multiple first stitched video images is realized through the first video stitching subsystem 20, and a final second stitched video image, that is, a wide area image, is obtained. Through the mode of carrying out multistage splicing on the video images collected by the plurality of cameras, due to the fact that the video capture cards 10 a-10 f and the first video splicing subsystem 20 are adopted to carry out multistage video image splicing on the collected video images, compared with the mode of directly transmitting all the video images collected by the plurality of cameras to the server for splicing, the technical effects of reducing the power consumption of the server and avoiding overlarge pressure on the server are achieved. Meanwhile, the problem that the size of equipment is large and the equipment is not favorable for use and installation due to overhigh cost of the server is avoided. And then the technical problems that the power consumption is large and the pressure caused by a server is large due to the fact that the quality of the spliced panoramic image is guaranteed by splicing a plurality of high-definition images by using a strong calculation force of an existing panoramic video splicing system in the prior art are solved.

Optionally, the first video stitching subsystem 20 includes: a plurality of first-level video splicing cards 20 a-20 b; the first-stage video splicing cards 20a to 20b are respectively in communication connection with the corresponding video acquisition cards 10a to 10f and are used for splicing multiple paths of first spliced video images received from the corresponding video acquisition cards 10a to 10f to generate third spliced video images; and the second-level video splicing card 20c is in communication connection with the plurality of first-level video splicing cards 20a to 20b, and is configured to splice multiple paths of third spliced video images received from the plurality of first-level video splicing cards 20a to 20b to generate a second spliced video image.

Specifically, the first video stitching subsystem 20 includes first-level video stitching cards 20 a-20 b and a second-level video stitching card 20 c. Referring to fig. 1, the first-level video stitching card 20a may receive, for example, 3 first stitched video images of the video capture cards 10a to 10c, so as to perform secondary stitching on the received 3 first stitched video images, and generate a third stitched video image. Similarly, the first-stage video stitching card 20b may receive 3 channels of the first stitched video images from the video capture cards 10d to 10f, so as to perform secondary stitching on the received 3 channels of the first stitched video images to generate a third stitched video image. In addition, the third stitched video images are only images stitched by different first-level video stitching cards 20a to 20b, and the third stitched video images are different.

Further, the second-level video splicing card 20c is in communication connection with the first-level video splicing card 20a and the first-level video splicing card 20b, and is configured to receive 2 paths of third spliced video images from the first-level video splicing cards 20a to 20b, so as to further splice the 2 paths of third spliced video images, and generate a final second spliced video image, that is, a wide area image.

Therefore, the first video splicing subsystem 20 is divided into two stages of video splicing cards, and the multi-path first spliced video images received from the video acquisition cards 10 a-10 f are spliced in 2 stages again, so that the technical problem that the server is stressed too much as the multi-path video images are directly transmitted to the server for splicing is solved.

Optionally, the video capture cards 10a to 10f are provided with: the image input interfaces 110a to 110f, the image input interfaces 110a to 110f are connected with a plurality of corresponding cameras; the image processors 120a to 120f, the image processors 120a to 120f are connected with the image input interfaces 110a to 110 f; the first image stitching modules 130a to 130f, the first image stitching modules 130a to 130f are connected with the image processors 120a to 120 f; and video output interfaces 140a to 140f, the video output interfaces 140a to 140f being connected to the first image stitching modules 130a to 130 f.

Specifically, referring to fig. 2A and 3, the image input interfaces 110a to 110f may be, for example, MIPI and LVDS interfaces, and are used to connect with the cameras 1 to 24 and receive the captured multiple video images from the cameras 1 to 24.

For example, the image input interface 110a can receive 4 captured video images from the cameras 1-4 and then send the video images to the image processor 120 a. The image processor 120a pre-processes the received 4-way video images and pre-processes the video images into video images suitable for the first image stitching module 130a to process. The first image stitching module 130a stitches the 4 channels of preprocessed video images to generate a first stitched video image, and then transmits the first stitched video image through the video output interface 140 a.

The video output interface 140a may be, for example, a MIPI interface, a LVDS interface, or the like. In addition, the configuration of the other video capture cards 10 b-10 f is the same as that of the video capture card 10a, and thus the description thereof is omitted. Therefore, the initial splicing of the multiple paths of video images is realized through the configuration of the video capture cards 10 a-10 f, and then a first spliced video image is generated.

Optionally, the first-level video splicing cards 20a to 20b are provided with: the first video input interfaces 210 a-210 b, the first video input interfaces 210 a-210 b are in communication connection with the video output interfaces 140 a-140 f of the corresponding video capture cards 10 a-10 f; the second image stitching modules 220a to 220b, the second image stitching modules 220a to 220b are connected with the first video input interfaces 210a to 210 b; the first video coding and decoding modules 230a to 230b, the first video coding and decoding modules 230a to 230b are connected with the second image splicing modules 220a to 220 b; and first ports 240a to 240b, the first ports 240a to 240b being connected to the first video encoding and decoding modules 230a to 230 b.

Specifically, referring to fig. 4, the first video input interfaces 210 a-210 b configured on the first-level video splicing cards 20 a-20 b may be MIPI interfaces, LVDS interfaces, etc., and are configured to be used with the video output interfaces 140 a-140 f of the video capture cards 10 a-10 f.

For example, the first video input interface 210a receives 3 channels of the first stitched video images from the video output interfaces 140 a-140 c of the video capture cards 10 a-10 c, and then transmits the received first stitched video images to the second image stitching module 220 a. The second image stitching module 220a stitches the 3 channels of first stitched video images to generate a third stitched video image, and then transmits the third stitched video image to the first video codec module 230 a. The first video codec module 230a encodes the third spliced video image and transmits the encoded third spliced video image to the second-level video splicing card 20c through the first internet access 240 a. In addition, the configuration of the first-level video splicing card 20b refers to the first video splicing card 20a, and is not described in detail here.

Therefore, by the mode, the splicing of the multiple paths of first spliced video images is completed, and the generated third spliced image is compressed and then transmitted, so that the pressure of network transmission is reduced.

Optionally, the second-level video splicing card 20c is provided with: a second port 210 c; the second video coding and decoding module 220c, the second video coding and decoding module 220c is connected with the second port 210 c; and a third image stitching module 230c, wherein the third image stitching module 230c is connected to the second video codec module 220 c.

Specifically, referring to fig. 5, the second-level video splicing card 20c may receive the 2-path compressed third spliced video image from the first ports 240a to 240b of the first-level video splicing cards 20a to 20b through the second port 210c, and send the third spliced video image to the second video codec module 220 c. The second video codec module 220c decompresses the received compressed third spliced video image and transmits the decompressed third spliced video image to the third image splicing module 230 c. The third image stitching module 230c stitches the received decompressed 2 channels of third stitched video images to generate a second stitched video image and transmits the second stitched video image to the second video encoding and decoding module 220 c. Then, the second video codec module 220c compresses the received second stitched video image and transmits the compressed second stitched video image to the terminal device 60 through the second network port 210c via the network. Accordingly, the above configuration by the second-stage video stitching card 20c enables stitching of multiple third stitched video images, thereby generating a second stitched video image (wide area image).

Optionally, the image stitching system further comprises: the network switch 30, the network switch 30 is communicatively connected to the first ports 240a to 240b and the second port 210 c.

Specifically, referring to fig. 1, the network switch 30 is used to implement data transmission between the first-level video splicing cards 20a to 20b and the second-level video splicing card 20 c. For example, the first ports 240a to 240b of the first-stage video splicing cards 20a to 20b may transmit the compressed third spliced video image to the second port 220c of the second-stage video splicing card 20c through the network switch 30, for example. Thereby realizing network data transmission between the first-level video splicing cards 20 a-20 b and the second-level video splicing card 20c through the network switch 30.

In addition, the network switch 30 may provide NTP (network time protocol) server service, and all the video capture cards 10a to 10f, the first-level video splicing cards 20a to 20b, and the second-level video splicing card 20c obtain uniform system time from the NTP server, so that clock sources of all the video capture cards 10a to 10f, the first-level video splicing cards 20a to 20b, and the second-level video splicing card 20c are synchronized (millisecond synchronization). Therefore, when the video capture cards 10a to 10f or the first-stage video splicing cards 20a to 20b or the second-stage video splicing card 20c transmit a frame of video image via the network, the timestamp signal and a frame of video image data are packaged and transmitted together. And when the receiver receives the data packet, the time stamp information in the data packet is passed. Synchronization of the video images between the various channels can be achieved so that images of the same time stamp (or with less deviation of the information of the time stamp) are stitched together.

Optionally, the image stitching system further comprises: a terminal device 60 communicatively connected to the network switch 30.

Specifically, referring to fig. 1, the terminal device 60 may be, for example, a PC terminal with a display, and an operator may perform operations such as viewing and processing on the image stitching system through the terminal device 60. For example, the second portal 220c of the second-stage splice card 20c can transmit the compressed second spliced video image to the terminal device 60 via the network switch 30. So that the operator can view the stitched second stitched video image, i.e. the wide area image, on the terminal device 60.

Optionally, the video capture cards 10a to 10f are further provided with: the video coding modules 150a to 150f, the video coding modules 150a to 150f are connected with the first image splicing modules 130a to 130 f; and third ports 160a to 160f, the third ports 160a to 160f are connected to the video encoding modules 150a to 150f and are connected to the network switch 30 in communication.

Specifically, referring to FIG. 3, the video capture cards 10 a-10 f further include video encoding modules 150 a-150 f and third ports 160 a-160 f communicatively connected to the network switch 30.

For example, the video encoding module 150a of the video capture card 10a receives the first stitched video image from the first image stitching module 130a, compresses the first stitched video image, and transmits the compressed first stitched video image to the first-stage video stitching cards 20 a-20 b through the third port 160 a. In addition, the video encoding modules 150 b-150 f and the third ports 160 b-160 f of the other video capture cards 10 b-10 f are described with reference to the video capture card 10a, and are not repeated here. Therefore, the video coding modules 150a to 150f and the third network ports 160a to 160f are arranged in the video capture cards 10a to 10f to compress and transmit the spliced first spliced video images, and the pressure of network data transmission is further reduced.

Optionally, the video capture cards 10a to 10f are further provided with first data transmission interfaces 170a to 170f, and the first data transmission interfaces 170a to 170f are connected with the video encoding modules 150a to 150 f; the first-level video splicing cards 20a to 20b are further provided with second data transmission interfaces 250a to 250b, and the second data transmission interfaces 250a to 250b are connected with the first video coding and decoding modules 230a to 230 b; and/or the second-level video splicing card 20c is further provided with a third data transmission interface 240c, and the third data transmission interface 240c is connected with the second video coding and decoding module 220 c.

Specifically, referring to fig. 3 to 5, the first data transmission interfaces 170a to 170f, the second data transmission interfaces 250a to 250b, and the third data transmission interface 240c may be data transmission interfaces such as USB, for example.

The first data transmission interfaces 170a to 170f are connected to the second data transmission interfaces 250a to 250f, for example, and can transmit the compressed first spliced video image to the first-stage video splicing cards 20a to 20b, and the second data transmission interfaces 250a to 250b can transmit the compressed third spliced video image to the third data transmission interface 240c of the second-stage video splicing card 20c, for example.

Therefore, the wired data transmission among the video acquisition cards 10a to 10f, the first-level video splicing cards 20a to 20b and the second-level video splicing cards 20c is realized by arranging the first data transmission interfaces 170a to 170f, the second data transmission interfaces 250a to 250b on the first-level video splicing cards 20a to 20b and the third data transmission interface 240c on the second-level video splicing cards 20c, so that the normal transmission of data can be further ensured under the condition that the network switch 30 fails.

In addition, referring to fig. 4, the video inputs of the first-level video splicing cards 20a to 20b may be MIPI interfaces (first video input interfaces 210a to 210b), USB (second data transmission interfaces 250a to 250b), and network video streams (first network ports 240a to 240 b). After the multiple first spliced video image streams are spliced by the second image splicing modules 220a to 220b, the video is compressed, and then the video is output through the first network ports 240a to 240 b. Therefore, real-time wide-area image splicing of the image splicing system (multi-view splicing system) is realized through the plurality of video acquisition cards 10 a-10 f, the first-level video splicing cards 20 a-20 b and the second-level video splicing card 20 c.

Optionally, the image stitching system further comprises: and the second video stitching subsystem 40 is in communication connection with the network switch 30, and the second video stitching subsystem 40 is configured to communicate with the plurality of video capture cards 10 a-10 f through the network switch 30, and is configured to stitch the plurality of paths of video images received from the plurality of video capture cards 10 a-10 f to generate a fourth stitched video image.

Specifically, referring to fig. 6, the second video stitching subsystem 40 may be, for example, a local video stitching subsystem that partially stitches video images captured by multiple cameras. And the second video stitching subsystem 40 is in communication connection with the video capture cards 10a to 10f, and is configured to receive, from the video capture cards 10a to 10f, video images corresponding to the video images captured by the cameras selected by the user, so as to perform local image stitching on the video images captured by the local cameras selected by the user.

In addition, referring to fig. 2A, when the video images of the plurality of cameras selected by the user correspond to one video capture card 10a to 10f, the video capture cards 10a to 10f can directly transmit the video images to the video splicing cards 40a to 40b without performing processing such as splicing on the video images. For example, when the user selects to view the local images of the cameras 4 to 5, 12 to 13, the corresponding video capture cards 10a to 10b, 10d to 10e do not need to process the received video images and directly transmit the video images to the video capture cards 40a to 40 b. In addition, for example, referring to fig. 7, for example, if the user selects to view the local stitched video images of the cameras 1 to 2 and 9 to 10, the video capture card 10a corresponding to the cameras 1 to 2 performs the first-stage stitching on the 2 channels of video images to generate the stitched video images and transmit the stitched video images to the second video stitching subsystem 40, and then the video capture card 10b corresponding to the cameras 9 to 10 stitches the 2 channels of video images to generate the stitched video images and transmit the stitched video images to the second video stitching subsystem 40. Finally, the received stitched video image is stitched by the second video stitching subsystem 40 to generate a fourth stitched video image (shown on the right side of fig. 7). Wherein the fourth stitched video image is a partial video image that the user wants to view.

Therefore, the local images of the wide area shooting device can be spliced through the second video splicing subsystem 40, and a user can view the corresponding local video images at any time according to requirements.

In addition, the local image content selected by the user to be viewed should be the video images shot by the adjacent cameras, so that the complete local video images can be spliced.

Optionally, the second video stitching subsystem 40 includes video stitching cards 40 a-40 b, and the video stitching cards 40 a-40 b of the second video stitching subsystem 40 include: the fourth network ports 410a to 410b, the fourth network ports 410a to 410b are in communication connection with the network switch 30; the third video coding and decoding modules 420a to 420b, the third video coding and decoding modules 420a to 420b are connected with the fourth network ports 410a to 410 b; and fourth image mosaic modules 430 a-430 b, the fourth image mosaic modules 430 a-430 b are connected with the third video coding and decoding modules 420 a-420 b.

Specifically, referring to FIG. 6, the second video stitching subsystem 40 may include, for example, at least one video stitching card. For example, a plurality of video tiles 40 a-40 b (which may be but is not limited to 2 video tiles) may be provided to satisfy the user's requirement of viewing multiple non-adjacent partial video images simultaneously.

Further, referring to fig. 8, for example, the fourth port 410a of the video stitching card 40a may receive 2 channels of the compressed first stitched video image (the first stitched video image is only an image stitched by the video capture card) from the third ports 160 a-160 b of the video capture cards 10 a-10 b (the video capture cards 10 a-10 b are only for illustration and are not limited to the video capture cards 10 a-10 b) through the network switch 30, and then send the received 2 channels of the compressed first stitched video image to the third video codec module 420a for decompressing the compressed first stitched video image. The third video codec module 420 transmits the decompressed first stitched video image to the fourth image stitching module 430 a. The fourth image stitching module 430a stitches the decompressed multiple first stitched video images, so as to generate a fourth stitched video image, and transmit the fourth stitched video image to the third video codec module 420 a. The third video codec module 420a compresses the fourth stitched video image and transmits the fourth stitched image to the fourth port 410a, so that the fourth port 410a can transmit the compressed fourth stitched image to the terminal device 60 through the network switch 30. In addition, the operation process of the video splicing card 40b is shown with reference to the video splicing card 40a, and is not described in detail here. Therefore, the local video images are spliced through the arrangement of the video splicing cards 40 a-40 b, the technical effect that a user can conveniently check local images is achieved, and the technical effect that the user can simultaneously check a plurality of non-adjacent local video images is achieved through the arrangement of the video acquisition cards 40 a-40 b.

In addition, the terminal device 60 is provided with a program for decompressing the compressed video image, so that the user can decompress and display the received spliced video image on the terminal device 60.

In addition, the video stitching cards 40 a-40 b may be, for example, local video stitching cards, which may dynamically access video streams of adjacent image capturing areas (i.e., adjacent video capturing cards 10 a-10 f) to complete local image stitching and output. The video interface is connected to the outputs of the different video capture cards 10 a-10 f via a network interface (i.e., the fourth network ports 410 a-410 b) via a network.

Optionally, the image stitching system further comprises: the line/field synchronizing signal generator 50 is connected with the video acquisition cards 10a to 10f, the video acquisition cards 10a to 10f are provided with line/field synchronizing signal transceivers 180a to 180f, and the line/field synchronizing signal transceivers 180a to 180f are used for receiving line synchronizing signals and/or field synchronizing signals sent by the line/field synchronizing signal generator 50 and sending the line synchronizing signals and/or the field synchronizing signals to the cameras connected with the video acquisition cards 10a to 10 f.

Specifically, referring to FIG. 9, the camera sensors of cameras 1-24 are operated in a slave mode, i.e., each camera sensor itself does not actively acquire and transmit video image data, but rather performs image acquisition and transmission based on the received synchronization signal. Thus, in this manner, the image stitching system of the present application generates line/field synchronization signals via the line/field signal generator 50 and sends the line/field synchronization signals to each of the video capture cards 10 a-10 f, and each capture card 100 then sends the line/field synchronization signals to the connected camera sensor. Therefore, each camera sensor can simultaneously acquire one frame or one line of video images and transmit the video images under the driving of the line/field synchronizing signals. Synchronization between the individual camera sensors can thereby be achieved.

Therefore, according to the image stitching system provided by the embodiment of the application, the video capture cards 10a to 10f are adopted to realize the first-stage stitching of the multi-channel video images. Then, the first video stitching subsystem 20 performs second-level stitching on the plurality of first stitched video images to obtain a final second stitched video image, i.e., a wide-area image. And the video images are further spliced in stages in the first video splicing subsystem 20, compared with a mode of directly splicing all the video images acquired by a plurality of cameras, the mode that the video acquisition cards 10a to 10f and the first video splicing subsystem 20 are adopted to splice multi-stage video images of the acquired video images is compared with a mode of directly transmitting all the video images acquired by a plurality of cameras to a server for splicing, the power consumption of the server is reduced, and the technical effect that the server is prevented from being stressed too much is achieved. Meanwhile, the problem that the size of equipment is large and the equipment is not favorable for use and installation due to overhigh cost of the server is avoided. And then the technical problems that the power consumption is large and the pressure caused by a server is large due to the fact that the quality of the spliced panoramic image is guaranteed by splicing a plurality of high-definition images by using a strong calculation force of an existing panoramic video splicing system in the prior art are solved. In addition, the present application provides a second video stitching subsystem 40 for partial video image stitching, thereby facilitating a user to view captured partial video images in real-time.

Furthermore, referring to FIG. 3, the video capture cards 10 a-10 f may be, for example, multi-view capture cards, wherein the video capture cards 10 a-10 f have the following characteristics:

1. the method supports real-time acquisition of 2-4 paths of video images;

2. each video capture card 10 a-10 f supports the maximum access of 4 paths of image sensors for collection;

3. the image sensor data is processed by an image processor, and can be directly output through video output interfaces 140 a-140 f (MIPI or LVDS) after multi-view splicing is realized; or after being compressed by a video encoder, the compressed video is output through a first data transmission interface 170-170 f (USB) and a third interface 160-160 f;

4. video output modes of the video capture cards 10a to 10 f: raw video (MIPI interface), compressed video stream (USB, network);

5. in order to realize the synchronization of all the acquisition devices (cameras), each video acquisition card 10 a-10 f is internally provided with a line/field synchronizing signal transceiver 180 a-180 f, and the line/field synchronizing signal transceiver 180 a-180 f receives the synchronizing signal from the line/field synchronizing signal generator 50 and transmits the synchronizing signal to each camera sensor; the camera sensor operates in a slave mode and performs exposure according to line/field signals from the video capture cards 10 a-10 f.

The relative arrangement of the components and steps, the numerical expressions, and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise. Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description. Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate. In all examples shown and discussed herein, any particular value should be construed as merely illustrative, and not limiting. Thus, other examples of the exemplary embodiments may have different values. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.

Spatially relative terms, such as "above … …," "above … …," "above … …," "above," and the like, may be used herein for ease of description to describe one device or feature's spatial relationship to another device or feature as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if a device in the figures is turned over, devices described as "above" or "on" other devices or configurations would then be oriented "below" or "under" the other devices or configurations. Thus, the exemplary term "above … …" can include both an orientation of "above … …" and "below … …". The device may be otherwise variously oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly.

In the description of the present disclosure, it is to be understood that the orientation or positional relationship indicated by the directional terms such as "front, rear, upper, lower, left, right", "lateral, vertical, horizontal" and "top, bottom", etc., are generally based on the orientation or positional relationship shown in the drawings, and are presented only for the convenience of describing and simplifying the disclosure, and in the absence of a contrary indication, these directional terms are not intended to indicate and imply that the device or element being referred to must have a particular orientation or be constructed and operated in a particular orientation, and therefore, should not be taken as limiting the scope of the disclosure; the terms "inner and outer" refer to the inner and outer relative to the profile of the respective component itself.

The above description is only for the preferred embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An image stitching system, comprising: a plurality of video capture cards (10 a-10 f) and a first video stitching subsystem (20) communicatively coupled to the video capture cards (10 a-10 f), wherein

The video capture cards (10 a-10 f) are respectively connected with the corresponding cameras and are used for receiving the multiple paths of video images from the cameras and splicing the multiple paths of video images to generate a first spliced video image; and

the first video stitching subsystem (20) is used for stitching the multiple paths of first stitched video images received from the plurality of video capture cards (10 a-10 f) to generate a second stitched video image.

2. The image stitching system of claim 1, wherein the first video stitching subsystem (20) comprises: a plurality of first-level video splicing cards (20 a-20 b); and a second level video splicing card (20c), wherein

The first-level video splicing cards (20 a-20 b) are respectively in communication connection with the corresponding video acquisition cards (10 a-10 f) and are used for splicing multiple paths of first spliced video images received from the corresponding video acquisition cards (10 a-10 f) to generate third spliced video images; and

the second-level video splicing cards (20c) are in communication connection with the plurality of first-level video splicing cards (20 a-20 b) and are used for splicing a plurality of paths of third spliced video images received from the plurality of first-level video splicing cards (20 a-20 b) to generate second spliced video images.

3. Image stitching system according to claim 2, characterized in that the video acquisition cards (10 a-10 f) are provided with:

image input interfaces (110 a-110 f), the image input interfaces (110 a-110 f) being connected to a plurality of corresponding cameras;

the image processors (120 a-120 f), the image processors (120 a-120 f) are connected with the image input interfaces (110 a-110 f);

the first image stitching modules (130 a-130 f), the first image stitching modules (130 a-130 f) are connected with the image processors (120 a-120 f); and

and the video output interfaces (140 a-140 f), wherein the video output interfaces (140 a-140 f) are connected with the first image splicing modules (130 a-130 f).

4. The image stitching system according to claim 3, wherein the first level video stitching card (20 a-20 b) is provided with:

the first video input interfaces (210 a-210 b) are in communication connection with the video output interfaces (140 a-140 f) of the corresponding video capture cards (10 a-10 f);

second image stitching modules (220 a-220 b), wherein the second image stitching modules (220 a-220 b) are connected with the first video input interfaces (210 a-210 b); and

first video encoding and decoding modules (230 a-230 b), wherein the first video encoding and decoding modules (230 a-230 b) are connected with the second image splicing modules (220 a-220 b); and

and the first network ports (240 a-240 b), wherein the first network ports (240 a-240 b) are connected with the first video coding and decoding modules (230 a-230 b).

5. Image stitching system according to claim 4, characterized in that the second level video stitching card (20c) is provided with:

a second port (210 c);

a second video codec module (220c), the second video codec module (220c) being connected to the second port (210 c); and

a third image stitching module (230c), the third image stitching module (230c) being connected with the second video codec module (220 c).

6. The image stitching system of claim 5, further comprising:

a network switch (30), the network switch (30) communicatively connected to the first portal (240 a-240 b) and the second portal (210 c); and/or

A terminal device (60), the terminal device (60) communicatively connected with the network switch (30).

7. The image stitching system according to claim 6, wherein the video capture cards (10 a-10 f) are further provided with:

the video coding modules (150 a-150 f), the video coding modules (150 a-150 f) are connected with the first image splicing modules (130 a-130 f); and

and third network ports (160 a-160 f), wherein the third network ports (160 a-160 f) are connected with the video coding modules (150 a-150 f) and are connected with the network switch (30) in a communication way.

8. The image stitching system of claim 7,

the video capture cards (10 a-10 f) are also provided with first data transmission interfaces (170 a-170 f), and the first data transmission interfaces (170 a-170 f) are connected with the video coding modules (150 a-150 f);

the first-level video splicing cards (20 a-20 b) are also provided with second data transmission interfaces (250 a-250 b), and the second data transmission interfaces (250 a-250 b) are connected with the first video coding and decoding modules (230 a-230 b); and/or

The second-level video splicing card (20c) is further provided with a third data transmission interface (240c), and the third data transmission interface (240c) is connected with the second video coding and decoding module (220 c).

9. The image stitching system of claim 6, further comprising: a second video stitching subsystem (40) communicatively coupled to the network switch (30), and the second video stitching subsystem (40) is configured to communicate with the plurality of video capture cards (10 a-10 f) via the network switch (30) and configured to stitch multiple video images received from the plurality of video capture cards (10 a-10 f) to generate a fourth stitched video image, wherein

The second video stitching subsystem (40) comprises video stitching cards (40 a-40 b), and the video stitching cards (40 a-40 b) of the second video stitching subsystem (40) comprise:

a fourth network port (410 a-410 b), the fourth network port (410 a-410 b) being communicatively connected to the network switch (30);

third video coding and decoding modules (420 a-420 b), wherein the third video coding and decoding modules (420 a-420 b) are connected with the fourth network ports (410 a-410 b); and

and the fourth image splicing modules (430 a-430 b), and the fourth image splicing modules (430 a-430 b) are connected with the third video coding and decoding modules (420 a-420 b).

10. The image stitching system of claim 1, further comprising: a line/field synchronizing signal generator (50) connected to said plurality of video capture cards (10 a-10 f), and

the video acquisition cards (10 a-10 f) are provided with line/field synchronizing signal transceivers (180 a-180 f), and the line/field synchronizing signal transceivers (180 a-180 f) are used for receiving line synchronizing signals and/or field synchronizing signals sent by the line/field synchronizing signal generator (50) and sending the line synchronizing signals and/or the field synchronizing signals to a plurality of cameras connected with the video acquisition cards (10 a-10 f).