CN109286742B - Image acquisition equipment and method - Google Patents
Image acquisition equipment and method Download PDFInfo
- Publication number
- CN109286742B CN109286742B CN201811223339.2A CN201811223339A CN109286742B CN 109286742 B CN109286742 B CN 109286742B CN 201811223339 A CN201811223339 A CN 201811223339A CN 109286742 B CN109286742 B CN 109286742B
- Authority
- CN
- China
- Prior art keywords
- image
- unit
- processing
- sending
- chip
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/50—Constructional details
- H04N23/54—Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/81—Camera processing pipelines; Components thereof for suppressing or minimising disturbance in the image signal generation
Abstract
The invention discloses an image acquisition device and a method, wherein the image acquisition device comprises: the system comprises a master chip, a slave chip and an image sensor; the main chip comprises an image coding unit and an image processing unit; the image processing unit carries out ISP processing on the received image and sends the image processed by the ISP to the slave chip; the slave chip performs fusion processing on the images, and the image coding unit performs coding processing on the images. In the embodiment of the invention, an image sensor acquires an image, an image processing unit in a main chip performs ISP processing on the image, the image processed by the ISP is sent to a slave chip, the slave chip performs fusion processing on the image, and an image coding unit in the main chip performs image coding processing. The working pressure of the slave chip is reduced, and the computing capability of the master chip is utilized.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to image acquisition equipment and an image acquisition method.
Background
At present, with the rapid development of the field of video monitoring, the multi-view camera is more and more popular, and the multi-view camera has wide application and powerful functions. The multi-view camera comprises a lens and a plurality of image sensor sensors, the plurality of image sensors can respectively collect infrared images, color images, black and white images and the like of the same scene, and then the images are fused, so that the quality of the finally obtained images is better.
In the prior art, after a plurality of image sensors respectively acquire images, a slave chip performs image signal ISP processing on each image so as to remove noise of each image, then performs fusion processing on each image after noise removal, sends the image after the fusion processing to a master chip, and the master chip encodes and transmits the image after the fusion processing to an external processing device. The processes of performing ISP processing and fusion processing on each image require strong computing power, so that the working pressure of the slave chip in the prior art is large, and the master chip only encodes the image, so that the computing power of the master chip is not utilized.
Disclosure of Invention
The embodiment of the invention provides image acquisition equipment and method, which are used for solving the problems that the working pressure of a slave chip is higher and the computing capacity of a master chip is not utilized in the prior art.
An embodiment of the present invention provides an image capturing apparatus, including: a master chip, a slave chip and at least one image sensor; the main chip comprises an image coding unit and an image processing unit;
each image sensor is connected with the image processing unit and used for acquiring images and sending the images to the image processing unit;
the image processing unit is also connected with the slave chip and is used for carrying out ISP processing on the received image and sending the image processed by the ISP to the slave chip;
the slave chip is also connected with the image coding unit and used for carrying out fusion processing on the received image processed by the ISP and sending the image processed by the fusion processing to the image coding unit;
the image coding unit is connected with the processing device, and is used for coding the received image after fusion processing and sending the image after coding processing to the processing device.
Further, the image processing unit comprises the same number of image processing subunits as the image sensors, and each image sensor is connected with the corresponding image processing subunit.
Further, the slave chip comprises an image fusion unit and a fused image sending unit;
the image processing unit is connected with the image fusion unit, the image fusion unit is also connected with the fused image sending unit, and the fused image sending unit is also connected with the image coding unit;
the image processing unit is specifically configured to send the image processed by the ISP to the image fusion unit, and the image fusion unit is configured to perform fusion processing on the received image processed by the ISP and send the image processed by the fusion processing to the fused image sending unit; and the fused image sending unit is used for sending the received fused image to the image coding unit.
Further, the master chip further comprises an image splicing unit, and the slave chip further comprises an image splitting unit;
the image splicing unit is respectively connected with the image splitting unit and the image processing unit;
the image processing unit is specifically configured to send the image processed by the ISP to the image stitching unit;
the image splicing unit is used for splicing the images processed by the ISP and sending the spliced images and the splicing styles to the image splitting unit;
the image splitting unit is connected with the image fusion unit and used for splitting the spliced image according to the splicing style and sending the split image to the image fusion unit.
Further, the image stitching unit is specifically configured to stitch the images processed by the ISP in the up-down direction or the left-right direction.
Further, the main chip further comprises an image selection unit and a control unit;
the image selection unit is respectively connected with the fused image sending unit, the control unit, the image coding unit and each image processing unit;
the image processing unit is also used for sending the image processed by the ISP to the image selection unit; the fused image sending unit is specifically configured to send the received image after fusion processing to the image selecting unit;
the control unit is used for sending the identification information of the image transmission channel to the image selection unit; the image selection unit is used for determining a target image corresponding to the identification information of the image transmission channel and sending the target image to the image coding unit;
the image encoding unit is specifically configured to perform encoding processing on the received target image.
Furthermore, the control unit is also connected with the image splicing unit, the image splitting unit, the image fusion unit and the fused image sending unit and is used for controlling the image splicing unit, the image splitting unit, the image fusion unit and the fused image sending unit to stop working or work.
Furthermore, the image acquisition equipment also comprises a master chip and a slave chip sharing memory;
the image processing unit is also connected with the master chip and the slave chip shared memory so as to send the image processed by the ISP to the master chip and the slave chip shared memory; the slave chip is also connected with the master-slave chip shared memory so as to acquire the image processed by the ISP from the master-slave chip shared memory.
Further, the slave chip includes: the system comprises a field programmable gate array FPGA chip, a digital signal processing technology DSP chip or a graphic processing GPU chip.
In another aspect, an embodiment of the present invention provides an image acquisition method, where the method includes:
the main chip receives the image sent by each image sensor, performs ISP (Internet service provider) processing on the received image, and sends the image processed by the ISP to the slave chip, so that the slave chip performs fusion processing on the received image processed by the ISP;
and receiving the image after fusion processing sent by the slave chip, coding the received image after fusion processing, and sending the image after coding processing to a processing device.
Further, the sending the ISP-processed image to the slave chip, so that the slave chip performs fusion processing on the received ISP-processed image includes:
and splicing the image processed by the ISP, sending the spliced image and the splicing pattern to the slave chip, splitting the spliced image by the slave chip according to the splicing pattern, and fusing the split image.
Further, after receiving the image sent by each image sensor and receiving the fusion processed image sent by the slave chip, the method further comprises:
receiving identification information of an image transmission channel, determining a target image corresponding to the identification information of the image transmission channel, encoding the target image, and sending the encoded target image to a processing device.
The embodiment of the invention provides an image acquisition device and a method, wherein the image acquisition device comprises: a master chip, a slave chip and at least one image sensor; the main chip comprises an image coding unit and an image processing unit; each image sensor is connected with the image processing unit and used for acquiring images and sending the images to the image processing unit; the image processing unit is also connected with the slave chip and is used for carrying out ISP processing on the received image and sending the image processed by the ISP to the slave chip; the slave chip is also connected with the image coding unit and used for carrying out fusion processing on the received image processed by the ISP and sending the image processed by the fusion processing to the image coding unit; the image coding unit is connected with the processing device, and is used for coding the received image after fusion processing and sending the image after coding processing to the processing device.
In the embodiment of the invention, after the image sensor acquires the image, the image processing unit in the main chip performs ISP processing on the image, then the image processed by the ISP is sent to the slave chip, the slave chip performs fusion processing on the image, and then the image coding unit in the main chip performs image coding processing. The ISP processing and the coding processing of the image are finished by the main chip, and the fusion processing of the image is finished by the slave chip, so that the working pressure of the slave chip is reduced, and the computing capability of the main chip is utilized.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 is a schematic structural diagram of an image capturing apparatus provided in embodiment 1 of the present invention;
fig. 2 is a schematic structural diagram of an image capturing apparatus provided in embodiment 2 of the present invention;
fig. 3 is a schematic structural diagram of an image capturing apparatus provided in embodiment 3 of the present invention;
fig. 4 is a schematic structural diagram of another image capturing apparatus provided in embodiment 3 of the present invention;
fig. 5 is a schematic structural diagram of an image capturing apparatus provided in embodiment 4 of the present invention;
fig. 6 is a schematic diagram of image stitching provided in embodiment 4 of the present invention;
fig. 7 is a schematic structural diagram of another image capturing apparatus provided in embodiment 4 of the present invention;
fig. 8 is a schematic structural diagram of an image capturing apparatus provided in embodiment 5 of the present invention;
fig. 9 is a schematic structural diagram of another image capturing apparatus provided in embodiment 5 of the present invention;
fig. 10 is a schematic diagram of image transmission provided in embodiment 5 of the present invention;
fig. 11 is a schematic structural diagram of an image capturing apparatus according to embodiment 6 of the present invention;
fig. 12 is a schematic structural diagram of an image capturing apparatus according to embodiment 7 of the present invention;
fig. 13 is a schematic diagram of an image acquisition process provided in embodiment 8 of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the attached drawings, and it should be understood that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Example 1:
fig. 1 is a schematic structural diagram of an image capturing device according to an embodiment of the present invention, where the image capturing device includes: a master chip 11, a slave chip 12 and at least one image sensor 13; the main chip 11 comprises an image coding unit 111 and an image processing unit 112;
each image sensor 13 is connected with the image processing unit 112, and is used for acquiring an image and sending the image to the image processing unit 112;
the image processing unit 112 is further connected to the slave chip 12, and is configured to perform ISP processing on the received image and send the ISP-processed image to the slave chip 12;
the slave chip 12 is further connected to the image encoding unit 111, and is configured to perform fusion processing on the received image processed by the ISP, and send the image after fusion processing to the image encoding unit 111;
the image encoding unit 111 is configured to be connected to a processing device, and configured to perform encoding processing on the received image subjected to fusion processing, and send the image subjected to encoding processing to the processing device.
As shown in fig. 1, in the embodiment of the present invention, the image pickup device includes at least one image sensor 13 and an image processing unit 112. Each image sensor 13 is connected to an image processing unit 112. Each image sensor 13 may perform image acquisition and then transmit the acquired image to the image processing unit 112 connected thereto.
The image processing unit 112 may perform ISP processing on the received image, thereby removing image noise. The image processing unit 112 is also connected to the slave chip 12, and the slave chip 12 may perform fusion processing on the received ISP-processed image after receiving the ISP-processed image transmitted by the image processing unit 112. The process of performing ISP processing on the image by the image processing unit 112 and the process of performing fusion processing on the image by the slave chip 12 belong to the prior art, and are not described herein again.
The slave chip 12 is also connected to an image encoding unit 111. The slave chip 12 can transmit the image after the fusion processing to the image encoding unit 111. The image encoding unit 111 performs encoding processing on the received fusion-processed image, and then transmits the image to a processing apparatus connected to itself.
In the embodiment of the invention, after the image sensor acquires the image, the image processing unit in the main chip performs ISP processing on the image, then the image processed by the ISP is sent to the slave chip, the slave chip performs fusion processing on the image, and then the image coding unit in the main chip performs image coding processing. The ISP processing and the coding processing of the image are finished by the main chip, and the fusion processing of the image is finished by the slave chip, so that the working pressure of the slave chip is reduced, and the computing capability of the main chip is utilized.
Example 2:
in order to improve the efficiency of performing ISP processing on an image, on the basis of the foregoing embodiment, fig. 2 is a schematic structural diagram of an image capturing apparatus according to an embodiment of the present invention, where the image processing unit 112 includes image processing sub-units 1121, the number of which is the same as that of the image sensors 13, and each image sensor 13 is connected to a corresponding image processing sub-unit 1121.
As shown in fig. 2, in the embodiment of the present invention, the image processing units 112 include the same number of image processing sub-units 1121 as the number of image sensors 13. The image sensors 13 are in one-to-one correspondence with the image processing sub-units 1121, wherein the image sensors 13 are connected with the corresponding image processing sub-units 1121. Each image sensor 13 may perform image acquisition and then send the acquired image to the image processing subunit 1121 connected thereto.
The image processing sub-unit 1121 may perform ISP processing on the received image, thereby removing image noise. Each image processing subunit 1121 is also connected to the slave chip 12, and each image processing subunit 1121 transmits an ISP-processed image to the slave chip 12 for subsequent processing by the slave chip 12.
Since in the embodiment of the present invention, the image processing sub-units 1121 are included in the image processing unit 112 in the same number as the image sensors 13. The image processing subunit 1121 corresponding to each image sensor 13 performs ISP processing on the image, that is, parallel processing on the image acquired by each image sensor 13 can be realized, so that efficiency of performing ISP processing on the image can be improved.
Example 3:
fig. 3 is a schematic structural diagram of an image capturing device according to an embodiment of the present invention, where the slave chip 12 includes an image fusion unit 121 and a fused image sending unit 122;
the image processing unit 112 is connected to the image fusion unit 121, the image fusion unit 121 is further connected to the fused image transmission unit 122, and the fused image transmission unit 122 is further connected to the image encoding unit 111;
the image processing unit 112 is specifically configured to send the image processed by the ISP to the image fusion unit 121, where the image fusion unit 121 is configured to perform fusion processing on the received image processed by the ISP and send the image processed by the fusion processing to the fused image sending unit 122; the fused image sending unit 122 is configured to send the received image after the fusion processing to the image encoding unit 111.
The slave chip 12 specifically includes an image fusion unit 121 and a fused image transmission unit 122, the image fusion unit 121 may perform fusion processing on the image processed by the ISP, and then the fused image transmission unit 122 transmits the image processed by the fusion processing to the image encoding unit 111 in the master chip 11.
Fig. 4 is a schematic structural diagram of another image capturing apparatus according to an embodiment of the present invention, and as shown in fig. 4, the image processing unit 112 includes image processing sub-units 1121, the number of which is the same as that of the image sensors 13, each image sensor 13 is connected to a corresponding image processing sub-unit 1121, and each image processing sub-unit 1121 is connected to the image fusion unit 121. Each image processing sub-unit 1121 may transmit the ISP-processed image to the image fusion unit 121.
Example 4:
to avoid the frame loss and frame error phenomenon when transmitting images between the master chip 11 and the slave chip 12 and the problem of poor image transmission real-time performance, on the basis of the above embodiment, fig. 5 is a schematic structural diagram of an image acquisition device according to an embodiment of the present invention, where the master chip 11 further includes an image stitching unit 113, and the slave chip 12 further includes an image splitting unit 123;
the image stitching unit 113 is respectively connected with the image splitting unit 123 and the image processing unit 112;
the image processing unit 112 is specifically configured to send the image processed by the ISP to the image stitching unit 113;
the image splicing unit 113 is configured to splice the images processed by the ISP, and send the spliced images and the splicing style to the image splitting unit 123;
the image splitting unit 123 is connected to the image fusion unit 121, and is configured to split the spliced image according to the splicing style, and send the split image to the image fusion unit 121.
As shown in fig. 5, the master chip 11 further includes an image stitching unit 113, the slave chip 12 further includes an image splitting unit 123, the image stitching unit 113 is connected to the image processing unit 112, and the image stitching unit 113 is further connected to the image splitting unit 123. After performing ISP processing on the received image, the image processing unit 112 sends the image after ISP processing to the image stitching unit 113. The image stitching unit 113 may stitch the received images into one large image, and then send the stitched large image to the image splitting unit 123.
The image stitching unit 113 is specifically configured to stitch the ISP-processed images in the up-down direction or the left-right direction.
As shown in fig. 6, the image output by the image processing unit 112 is, for example, two 1080P YUV420 images, that is, an image with a resolution of 1920 × 1080. If the two images are stitched in the up-and-down direction, a large image with a resolution of 1920 × 2160 is obtained, and if the two images are stitched in the left-and-right direction, a large image with a resolution of 3840 × 1080 is obtained.
When the image stitching unit 113 stitches the ISP-processed images in the up-down direction, it is not specifically limited which image is on top and which image is on bottom; when the image processed by the ISP is processed by stitching in the left-right direction, it is not particularly limited which image is on the left and which image is on the right.
After the image splicing unit 113 performs splicing processing on the image processed by the ISP, the image after the splicing processing is sent to the image splitting unit 123, and meanwhile, the splicing style also needs to be sent to the image splitting unit 123. The image splitting unit 123 receives the spliced image and the splicing style, and splits the spliced image according to the splicing style.
If the image stitching style is the left stitching style in fig. 6, the image splitting unit 123 equally splits the stitched image into two images according to the left-right order, and if the image stitching style is the right stitching style in fig. 6, the image splitting unit 123 equally splits the stitched image into two images according to the up-down order.
The image splitting unit 123 is further connected to the image fusion unit 121, and after splitting the spliced image, the image splitting unit 123 sends the split image to the image fusion unit 121, and then the image fusion unit 121 performs fusion processing on the split image.
Fig. 7 is a schematic structural diagram of another image capturing apparatus according to an embodiment of the present invention, and as shown in fig. 7, the image processing unit 112 includes image processing sub-units 1121, the number of which is the same as that of the image sensors 13, each image sensor 13 is connected to a corresponding image processing sub-unit 1121, and each image processing sub-unit 1121 is connected to the image stitching unit 113. Each image processing sub-unit 1121 may transmit the ISP-processed image to the image stitching unit 113.
In the embodiment of the present invention, the image stitching unit 113 performs stitching processing on the image sent by the image processing unit 112, then sends the stitched large image to the image splitting unit 123, and sends the split large image to the image fusion unit 121 for fusion processing after the split large image is split by the image splitting unit 123. Therefore, the phenomenon of frame loss and frame error which are possibly caused by that each image processing unit 112 independently sends the image is avoided, the real-time performance of image transmission is better, and the synchronization of the image transmission time sequence can be ensured.
Example 5:
in order to improve user experience, on the basis of the foregoing embodiments, fig. 8 is a schematic structural diagram of an image capturing device according to an embodiment of the present invention, where the main chip 11 further includes an image selecting unit 114 and a control unit 115;
the image selection unit 114 is connected to the fused image transmission unit 122, the control unit 115, the image encoding unit 111, and each image processing unit 112, respectively;
the image processing unit 112 is further configured to send the image processed by the ISP to the image selecting unit 114; the fused image sending unit 122 is specifically configured to send the received fused image to the image selecting unit 114;
the control unit 115 is configured to send identification information of an image transmission channel to the image selection unit 114; the image selecting unit 114 is configured to determine a target image corresponding to the identification information of the image transmission channel, and send the target image to the image encoding unit 111;
the image encoding unit 111 is specifically configured to perform encoding processing on the received target image.
In the embodiment of the present invention, the main chip 11 further includes an image selecting unit 114, and the image processing unit 112 and the fused image sending unit 122 are further connected to the image selecting unit 114 respectively. Each image processing unit 112 may transmit the image processed by the ISP to the image selecting unit 114, and the fused image transmitting unit 122 may also transmit the image processed by the fusion to the image selecting unit 114.
The main chip 11 further includes a control unit 115, and the control unit 115 is connected to the image selection unit 114 and is used for controlling which image is selected by the image selection unit 114. Specifically, the control unit 115 may store in advance identification information of each image transmission channel, where the identification information of each image transmission channel may be identification information of each image sensor 13 and identification information of the fused image transmitting unit 122. The control unit 115 may send the identification information of the image transmission channel to the image selection unit 114, then the image selection unit 114 determines a target image corresponding to the identification information of the image transmission channel, and sends the target image to the image encoding unit 111, and the image encoding unit 111 may perform encoding processing on the received target image and send the encoded target image to a processing device connected to the image encoding unit 111.
For example, the image pickup apparatus includes three image sensors 13 whose identification information is A, B, C respectively, and the identification information of the fused image transmitting unit 122 in the slave chip 12 is D. When the identification information of the image transmission channel sent by the control unit 115 to the image selection unit 114 is a, the image selection unit 114 selects an ISP-processed image corresponding to the image sent by the image sensor 13 with the identification information of a as a target image, and sends the target image to the image encoding unit 111. When the identification information of the image transmission channel sent by the control unit 115 to the image selection unit 114 is D, the image selection unit 114 selects the image sent by the fused image sending unit 122 with the identification information of D as the target image, and sends the target image to the image encoding unit 111.
Fig. 9 is a schematic structural diagram of another image capturing apparatus according to an embodiment of the present invention, and as shown in fig. 9, the image processing unit 112 includes image processing sub-units 1121, the number of which is the same as that of the image sensors 13, each image sensor 13 is connected to a corresponding image processing sub-unit 1121, and each image processing sub-unit 1121 is connected to the image selecting unit 114. Each image processing sub-unit 1121 may transmit the ISP-processed image to the image selection unit 114.
In the embodiment of the present invention, the control unit 115 may send the identification information of the image transmission channel to the image selection unit 114 according to the user requirement, and then control the image selection unit 114 to transmit the image corresponding to the identification information, so that the user experience may be improved.
Fig. 10 is a schematic diagram of image transmission, and as shown in fig. 10, after the image capturing apparatus is started, a plurality of video streams are output by the image processing unit 112, that is, the image processing unit 112 outputs an image whose current processing is completed. And then the image acquisition equipment judges whether the images need to be fused or not, and if not, one path of images is selected for coding and is sent to the processing equipment. If the image is the image of the target object, the image is spliced and sent to the slave chip 12, the image is split and fused in the slave chip, the fused image is sent to the master chip 11, and the master chip 11 is used for coding and sending the coded image to the processing equipment.
Example 6:
sometimes, a user only needs to check an image transmitted by one or some image processing units 112, but does not need to check an image after fusion processing, at this time, on the basis of the above embodiments, in order to reduce power consumption of the slave chip 12, fig. 11 is a schematic structural diagram of an image capturing device provided in the embodiments of the present invention, and the control unit 115 is further connected to the image splicing unit 113, the image splitting unit 123, the image fusion unit 121, and the fused image sending unit 122, and is configured to control the image splicing unit 113, the image splitting unit 123, the image fusion unit 121, and the fused image sending unit 122 to stop working or perform working.
When the user does not need to view the fused image, the control unit 115 may send a control instruction to stop working to the image stitching unit 113, the image splitting unit 123, the image fusing unit 121, and the fused image sending unit 122, and the image stitching unit 113, the image splitting unit 123, the image fusing unit 121, and the fused image sending unit 122 stop working after receiving the control instruction to stop working. This may reduce the operating power consumption of the slave chip 12.
Specifically, the image capturing device may pre-agree to stop the operation, for example, the stop control instruction is the digital code 1234, and then the image splicing unit 113, the image splitting unit 123, the image fusion unit 121, and the fused image sending unit 122 stop the operation when receiving the digital code 1234.
Certainly, a control instruction for performing work may also be agreed in advance in the image capturing device, for example, the control instruction for performing work is a digital code 5678, when a user needs to view an image after fusion processing, the control unit 115 may send the control instruction for performing work to the image splicing unit 113, the image splitting unit 123, the image fusion unit 121, and the fused image sending unit 122, that is, send the digital code 5678, and after receiving the digital code 5678, the image splicing unit 113, the image splitting unit 123, the image fusion unit 121, and the fused image sending unit 122, the image is restarted and performs work.
Example 7:
on the basis of the foregoing embodiments, fig. 12 is a schematic structural diagram of an image acquisition device according to an embodiment of the present invention, where the image acquisition device further includes a master-slave chip shared memory 14;
the image processing unit 112 is further connected to the master-slave chip shared memory 14, so as to send the image processed by the ISP to the master-slave chip shared memory 14; the slave chip 12 is further connected to the master-slave chip shared memory 14, so as to obtain an image processed by the ISP from the master-slave chip shared memory 14.
In order to avoid the problem that the images cannot be normally transmitted due to mutual exclusion of the connection interface between the image processing unit 112 and the image fusion unit 121, as shown in fig. 12, the image capturing device further includes a master-slave chip shared memory 14. The image processing unit 112 is further connected to the master-slave chip shared memory 14, and after performing ISP processing on the image, the image processing unit 112 sends the ISP-processed image to the master-slave chip shared memory 14. The image fusion unit 121 is further connected to the master-slave chip shared memory 14, and the image fusion unit 121 obtains the image processed by the ISP from the master-slave chip shared memory 14 and performs fusion processing. Since the master-slave chip shared memory 14 has the characteristic of mutual exclusion, the problem that the images cannot be normally transmitted due to mutual exclusion of the connection interface between the image processing unit 112 and the image fusion unit 121 can be avoided.
The slave chip 12 in the embodiment of the present invention may be, but is not limited to, a field programmable gate array FPGA chip, a digital signal processing technology DSP chip, or a graphics processing GPU chip.
Example 8:
on the basis of the foregoing embodiments, fig. 13 is a schematic diagram of an image acquisition process provided in an embodiment of the present invention, where the process includes the following steps:
s101: the main chip receives the images sent by each image sensor, carries out ISP processing on the received images, sends the images processed by the ISP to the slave chip, and enables the slave chip to carry out fusion processing on the received images processed by the ISP.
S102: and receiving the image after fusion processing sent by the slave chip, coding the received image after fusion processing, and sending the image after coding processing to a processing device.
The image acquisition method provided by the embodiment of the invention is applied to the main chip, and the main chip receives the image sent by each image sensor and performs ISP processing on the received image so as to remove image noise. The ISP processed image is then transmitted to the slave chip. After receiving the ISP-processed image sent by the master chip, the slave chip 12 may perform fusion processing on the received ISP-processed image. And then the fused image is sent to the main chip. And the main chip carries out coding processing on the received image subjected to fusion processing and then sends the image to processing equipment connected with the main chip.
In the embodiment of the invention, after the image sensor acquires the image, the image processing unit in the main chip performs ISP processing on the image, then the image processed by the ISP is sent to the slave chip, the slave chip performs fusion processing on the image, and then the image coding unit in the main chip performs image coding processing. The ISP processing and the coding processing of the image are finished by the main chip, and the fusion processing of the image is finished by the slave chip, so that the working pressure of the slave chip is reduced, and the computing capability of the main chip is utilized.
Example 9:
in order to avoid the frame loss and frame error phenomenon when transmitting an image between the master chip 11 and the slave chip 12 and the problem of poor real-time performance of image transmission, on the basis of the foregoing embodiments, in an embodiment of the present invention, the sending an image processed by an ISP to a slave chip, so that the slave chip performs fusion processing on the received image processed by the ISP includes:
and splicing the image processed by the ISP, sending the spliced image and the splicing pattern to the slave chip, splitting the spliced image by the slave chip according to the splicing pattern, and fusing the split image.
The main chip splices the received images into a large image after ISP processing is carried out on the received images, then sends the spliced large image to the slave chip, simultaneously sends the splicing pattern to the slave chip, after the slave chip receives the spliced images, the spliced images can be split according to the splicing pattern, and then the split images are fused.
The images processed by the ISP may be stitched in the up-down direction or the left-right direction.
For example, the image received by the main chip is two 1080P YUV420 images, that is, an image with a resolution of 1920 × 1080. If the two images are stitched in the up-and-down direction, a large image with a resolution of 1920 × 2160 is obtained, and if the two images are stitched in the left-and-right direction, a large image with a resolution of 3840 × 1080 is obtained.
When the main chip splices and processes the images processed by the ISP in the up-down direction, it is not specifically limited which image is on the top and which image is on the bottom; when the image processed by the ISP is processed by stitching in the left-right direction, it is not particularly limited which image is on the left and which image is on the right.
And after splitting the spliced image, the slave chip sends the split image to the master chip, and then the master chip performs fusion processing on the split image.
Example 10:
in order to improve the user experience, on the basis of the foregoing embodiments, in an embodiment of the present invention, after receiving the image sent by each image sensor and receiving the fusion-processed image sent by the slave chip, the method further includes:
receiving identification information of an image transmission channel, determining a target image corresponding to the identification information of the image transmission channel, encoding the target image, and sending the encoded target image to a processing device.
In the embodiment of the present invention, the master chip may further receive identification information of the image transmission channels, where the identification information of each image transmission channel may be identification information of each image sensor and identification information of the slave chip. And after receiving the identification information of the image transmission channel, the main chip determines a target image corresponding to the identification information of the image transmission channel, codes the target image and then sends the target image to the processing equipment connected with the main chip.
For example, the image capturing apparatus includes three image sensors, whose identification information is A, B, C respectively, and the identification information of the slave chip is D. And when the identification information of the image transmission channel received by the main chip is A, selecting an ISP processed image corresponding to the image sent by the image sensor with the identification information of A as a target image. And when the identification information of the image transmission channel received by the master chip is D, selecting the image which is sent by the slave chip and has the identification information of D and subjected to fusion processing as a target image.
In the embodiment of the invention, the main chip can determine the target image corresponding to the identification information of the image transmission channel according to the user requirement, encode the target image and transmit the target image to the processing equipment, so that the user experience can be improved.
The embodiment of the invention provides an image acquisition device and a method, wherein the image acquisition device comprises: a master chip, a slave chip and at least one image sensor; the main chip comprises an image coding unit and an image processing unit; each image sensor is connected with the image processing unit and used for acquiring images and sending the images to the image processing unit; the image processing unit is also connected with the slave chip and is used for carrying out ISP processing on the received image and sending the image processed by the ISP to the slave chip; the slave chip is also connected with the image coding unit and used for carrying out fusion processing on the received image processed by the ISP and sending the image processed by the fusion processing to the image coding unit; the image coding unit is connected with the processing device, and is used for coding the received image after fusion processing and sending the image after coding processing to the processing device.
In the embodiment of the invention, after the image sensor acquires the image, the image processing unit in the main chip performs ISP processing on the image, then the image processed by the ISP is sent to the slave chip, the slave chip performs fusion processing on the image, and then the image coding unit in the main chip performs image coding processing. The ISP processing and the coding processing of the image are finished by the main chip, and the fusion processing of the image is finished by the slave chip, so that the working pressure of the slave chip is reduced, and the computing capability of the main chip is utilized.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.
Claims (10)
1. An image capturing apparatus characterized in that the image capturing apparatus comprises: a master chip, a slave chip and at least one image sensor; the main chip comprises an image coding unit and an image processing unit;
each image sensor is connected with the image processing unit and used for acquiring images and sending the images to the image processing unit;
the slave chip comprises an image fusion unit, the master chip also comprises an image splicing unit, and the slave chip also comprises an image splitting unit; the image splicing unit is respectively connected with the image splitting unit and the image processing unit;
the image processing unit is used for carrying out ISP processing on the received image and sending the image processed by the ISP to the image splicing unit;
the image splicing unit is used for splicing the images processed by the ISP and sending the spliced images and the splicing styles to the image splitting unit;
the image splitting unit is connected with the image fusion unit and is used for splitting the spliced image according to the splicing style and sending the split image to the image fusion unit;
the image fusion unit is used for carrying out fusion processing on the received split image;
the slave chip is also connected with the image coding unit and used for sending the image subjected to fusion processing to the image coding unit;
the image coding unit is connected with the processing device, and is used for coding the received image after fusion processing and sending the image after coding processing to the processing device.
2. The image capturing device according to claim 1, wherein the image processing unit includes the same number of image processing sub-units as the image sensors, each image sensor being connected to a corresponding image processing sub-unit.
3. The image pickup device according to claim 1, wherein the slave chip includes a fused image transmitting unit;
the image fusion unit is also connected with the fused image sending unit, and the fused image sending unit is also connected with the image coding unit;
the image fusion unit is used for sending the fused image to the fused image sending unit; and the fused image sending unit is used for sending the received fused image to the image coding unit.
4. The image capturing device according to claim 1, wherein the image stitching unit is specifically configured to stitch the ISP-processed images in an up-down direction or a left-right direction.
5. The image capturing device according to claim 1, wherein the main chip further includes an image selecting unit and a control unit; the slave chip comprises a fused image sending unit;
the image selection unit is respectively connected with the fused image sending unit, the control unit, the image coding unit and each image processing unit;
the image processing unit is also used for sending the image processed by the ISP to the image selection unit; the fused image sending unit is specifically configured to send the received image after fusion processing to the image selecting unit;
the control unit is used for sending the identification information of the image transmission channel to the image selection unit; the image selection unit is used for determining a target image corresponding to the identification information of the image transmission channel and sending the target image to the image coding unit;
the image encoding unit is specifically configured to perform encoding processing on the received target image.
6. The image capturing device according to claim 5, wherein the control unit is further connected to the image stitching unit, the image splitting unit, the image fusing unit, and the fused image sending unit, and is configured to control the image stitching unit, the image splitting unit, the image fusing unit, and the fused image sending unit to stop working or to work.
7. The image capturing device of claim 1, wherein the image capturing device further comprises a master-slave chip shared memory;
the image processing unit is also connected with the master chip and the slave chip shared memory so as to send the image processed by the ISP to the master chip and the slave chip shared memory; the slave chip is also connected with the master-slave chip shared memory so as to acquire the image processed by the ISP from the master-slave chip shared memory.
8. The image capturing device of any one of claims 1 to 7, wherein the slave chip comprises: the system comprises a field programmable gate array FPGA chip, a digital signal processing technology DSP chip or a graphic processing GPU chip.
9. An image acquisition method, characterized in that the method comprises:
the main chip receives the image sent by each image sensor, performs ISP (Internet service provider) processing on the received image, and sends the image processed by the ISP to the slave chip, so that the slave chip performs fusion processing on the received image processed by the ISP;
receiving the fusion processed image sent by the slave chip, coding the received fusion processed image, and sending the coding processed image to a processing device;
the sending of the image processed by the ISP to the slave chip, so that the fusion processing of the received image processed by the ISP by the slave chip includes:
and splicing the image processed by the ISP, sending the spliced image and the splicing pattern to the slave chip, splitting the spliced image by the slave chip according to the splicing pattern, and fusing the split image.
10. The method of claim 9, wherein after receiving the image transmitted by each image sensor and receiving the fusion processed image transmitted by the slave chip, the method further comprises:
receiving identification information of an image transmission channel, determining a target image corresponding to the identification information of the image transmission channel, encoding the target image, and sending the encoded target image to a processing device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811223339.2A CN109286742B (en) | 2018-10-19 | 2018-10-19 | Image acquisition equipment and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811223339.2A CN109286742B (en) | 2018-10-19 | 2018-10-19 | Image acquisition equipment and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109286742A CN109286742A (en) | 2019-01-29 |
| CN109286742B true CN109286742B (en) | 2021-02-02 |
Family
ID=65178189
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811223339.2A Active CN109286742B (en) | 2018-10-19 | 2018-10-19 | Image acquisition equipment and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109286742B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111031227A (en) * | 2019-12-31 | 2020-04-17 | 华勤通讯技术有限公司 | Auxiliary shooting module and camera device comprising same |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102007058943A1 (en) * | 2007-12-07 | 2009-06-10 | Emt Ingenieurgesellschaft Dipl.-Ing. Hartmut Euer Mbh | Multi-spectral video device for air-based surveillance and real time aerial photograph monitoring of uncrewed aircraft, has electronic mosaic/fusion device designed such that fusion device receives image signals of video scanner cameras |
| CN104035913A (en) * | 2014-06-06 | 2014-09-10 | 中国电子科技集团公司第三十八研究所 | High-performance BW100 chip based SAR (Synthetic Aperture Radar) parallel processing method and device |
| CN105430285A (en) * | 2015-12-31 | 2016-03-23 | 深圳市华途数字技术有限公司 | Multi-image-sensor video camera and exposure method thereof |
| CN107203965A (en) * | 2016-03-18 | 2017-09-26 | 中国科学院宁波材料技术与工程研究所 | A kind of Panorama Mosaic method merged based on multichannel image |
| CN107220208A (en) * | 2017-07-07 | 2017-09-29 | 深圳市图芯智能科技有限公司 | A kind of image processing system and method |
| CN107959769A (en) * | 2016-10-17 | 2018-04-24 | 杭州海康威视数字技术股份有限公司 | A kind of video camera |
| CN107959787A (en) * | 2016-10-17 | 2018-04-24 | 杭州海康威视数字技术股份有限公司 | A kind of image processing method and device |
-
2018
- 2018-10-19 CN CN201811223339.2A patent/CN109286742B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102007058943A1 (en) * | 2007-12-07 | 2009-06-10 | Emt Ingenieurgesellschaft Dipl.-Ing. Hartmut Euer Mbh | Multi-spectral video device for air-based surveillance and real time aerial photograph monitoring of uncrewed aircraft, has electronic mosaic/fusion device designed such that fusion device receives image signals of video scanner cameras |
| CN104035913A (en) * | 2014-06-06 | 2014-09-10 | 中国电子科技集团公司第三十八研究所 | High-performance BW100 chip based SAR (Synthetic Aperture Radar) parallel processing method and device |
| CN105430285A (en) * | 2015-12-31 | 2016-03-23 | 深圳市华途数字技术有限公司 | Multi-image-sensor video camera and exposure method thereof |
| CN107203965A (en) * | 2016-03-18 | 2017-09-26 | 中国科学院宁波材料技术与工程研究所 | A kind of Panorama Mosaic method merged based on multichannel image |
| CN107959769A (en) * | 2016-10-17 | 2018-04-24 | 杭州海康威视数字技术股份有限公司 | A kind of video camera |
| CN107959787A (en) * | 2016-10-17 | 2018-04-24 | 杭州海康威视数字技术股份有限公司 | A kind of image processing method and device |
| CN107220208A (en) * | 2017-07-07 | 2017-09-29 | 深圳市图芯智能科技有限公司 | A kind of image processing system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109286742A (en) | 2019-01-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Schneider et al. | Augmented reality based on edge computing using the example of remote live support | |
| CN105915937B (en) | Panoramic video playing method and device | |
| US20140071131A1 (en) | Image processing apparatus, image processing method and program | |
| KR100776805B1 (en) | Efficient image transmission method and apparatus using stereo vision processing for intelligent service robot system | |
| CN103118247B (en) | Video data handling procedure and device, video cascade system and chip | |
| US9948921B2 (en) | Imaging processing apparatus, image processing method, and program | |
| JP2007097129A5 (en) | ||
| CN113329174B (en) | Control method, device and system of multi-view camera and electronic device | |
| CN111612878B (en) | Method and device for making static photo into three-dimensional effect video | |
| CN109286742B (en) | Image acquisition equipment and method | |
| US20230281916A1 (en) | Three dimensional scene inpainting using stereo extraction | |
| US20130120530A1 (en) | Image processing apparatus and method and program | |
| CN109146916B (en) | Moving object tracking method and device | |
| CN113038137B (en) | Video data output method, system, device and computer readable storage medium | |
| EP3429186B1 (en) | Image registration method and device for terminal | |
| EP2890124A1 (en) | Coding method and device applied to hevc-based 3dvc | |
| CN112650596B (en) | Cross-process sharing method, device and equipment for target data and storage medium | |
| CN108810395B (en) | Method and device for quickly displaying initial frame image and quickly shooting image | |
| CN105208396A (en) | Video frame coding method, video frame decoding method, video frame coding device, and video frame decoding device | |
| CN109819139B (en) | Intelligent electronic equipment and image processing unit, device and method | |
| JP2020102124A (en) | Image generator, method for generating image, and program | |
| KR20180046129A (en) | Device and method of image split transmission | |
| CN111405262A (en) | Viewpoint information generation method, apparatus, system, device and medium | |
| CN109756664B (en) | Intelligent electronic equipment and image processing unit, device and method | |
| CN112788325B (en) | Image processing method, encoding device, decoding device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |