CN214507229U - Image acquisition system and vehicle - Google Patents

Abstract

The utility model provides an image acquisition system and vehicle, this image acquisition system include control module group, transmission module and a plurality of camera module, the control module group includes first treater and pulse trigger unit, the transmission module group includes deserializer and serializer, first treater with the pulse triggers the unit connection, first treater the pulse trigger unit with the serializer respectively with the deserializer is connected, the serializer with the camera module is connected. The deserializer and the serializer are used for remotely transmitting the pulse signals, so that a plurality of camera modules can be triggered simultaneously, the synchronism of image acquisition is guaranteed, the deserializer and the serializer are used for remotely transmitting image data, the quality of the image data transmitted to the first processor can be guaranteed, and the effect of splicing subsequent images is better.

Description

Image acquisition system and vehicle

Technical Field

The utility model belongs to the technical field of image acquisition, in particular to image acquisition system and vehicle.

Background

As is well known, in some vehicles, there is a need for around view stitching or Advanced Driving Assistance System (ADAS) detection, for example, in commercial vehicles, because of the length of the vehicle body, 6 to 8 cameras are required to fully cover the vehicle, and at this time, images of multiple cameras need to be synchronously acquired, and then the acquired images need to be stitched. When images of multiple cameras are synchronously acquired, on one hand, the length required by a camera transmission cable is too long and image data needs to be transmitted in a long distance due to the length of a vehicle body; on the other hand, the pulse signals triggering the multi-channel cameras to shoot also need to be transmitted remotely, the existing image acquisition system is easy to generate signal interference and data loss in the remote transmission process of the pulse signals and the image data, and is often difficult to acquire the image data of the multi-channel cameras synchronously, so that the image splicing effect is poor.

SUMMERY OF THE UTILITY MODEL

An object of the utility model is to provide an image acquisition system and vehicle aims at solving the relatively poor technical problem of image concatenation effect among the prior art.

The utility model provides an image acquisition system, including control module group, transmission module and a plurality of camera module, the control module group includes first treater and pulse trigger unit, the transmission module group includes deserializer and serializer, first treater with the pulse triggers the unit connection, first treater the pulse trigger unit with the serializer respectively with the deserializer is connected, the serializer with the camera module is connected.

Further, the first processor includes a MIPI interface for connecting with the deserializer.

Further, the pulse trigger unit comprises a microcontroller, and the microcontroller is connected with the deserializer.

Further, the deserializer comprises an FPD-Link interface and/or a GMSL interface for connecting with the serializer.

Further, the deserializer and the serializer are in communication connection through an FPD-Link III protocol.

Furthermore, the transmission module comprises a plurality of deserializers, each deserializer is connected with at least one serializer, and the serializers are connected with the camera modules in a one-to-one correspondence manner.

Further, the first processor comprises an image acquisition unit and an image splicing unit, and the image splicing unit, the pulse triggering unit and the deserializer are respectively connected with the image acquisition unit.

Further, the camera module comprises a second processor and an image device, the second processor is connected with the image device, and the second processor and the image device are respectively connected with the serializer.

Further, the image device is a high-definition digital camera, and the high-definition digital camera is connected with the second processor through a CSI-2 interface.

The utility model also provides a vehicle, including the automobile body and as above-mentioned image acquisition system.

The utility model provides an image acquisition system and vehicle's beneficial effect lies in:

the image acquisition system comprises a control module, a transmission module and a plurality of camera modules, wherein the control module can comprise a first processor and a pulse trigger unit, and the transmission module can comprise a deserializer and a serializer. The pulse trigger unit sends out a pulse signal, the pulse signal is remotely transmitted to the camera module through the deserializer and the serializer, the camera module is triggered to acquire image data, and the image data is converted into a serial differential signal through the serializer and the deserializer to be remotely transmitted to the first processor.

The deserializer and the serializer are used for remotely transmitting the pulse signals, so that a plurality of camera modules can be triggered simultaneously, the synchronism of image acquisition is guaranteed, the deserializer and the serializer are used for remotely transmitting image data, the quality of the image data transmitted to the first processor can be guaranteed, and the effect of splicing subsequent images is better.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic structural diagram of an image acquisition system provided in this embodiment;

FIG. 2 is a schematic diagram of the operation of the pulse trigger unit in the present embodiment;

fig. 3 is a schematic diagram of the operation of the image capturing system in this embodiment.

The designations in the figures mean:

1. a control module; 11. a first processor; 12. a pulse trigger unit; 2. a transmission module; 21. a deserializer; 22. a serializer; 3. a camera module; 31. a second processor; 32. an imaging device.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the invention.

It is to be understood that the terms "upper", "lower", "left", "right", and the like, as used herein, refer to an orientation or positional relationship based on that shown in the drawings, which is for convenience of description only, and do not indicate or imply that the referenced device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore should not be considered limiting of this patent. The terms "first", "second" and "first" are used merely for descriptive purposes and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features. The meaning of "plurality" is two or more unless specifically limited otherwise. Furthermore, the terms "horizontal", "vertical", "overhang" and the like do not imply that the components are required to be absolutely horizontal or overhang, but may be slightly inclined. For example, "horizontal" merely means that the direction is more horizontal than "vertical" and does not mean that the structure must be perfectly horizontal, but may be slightly inclined.

It should also be noted that, unless expressly stated or limited otherwise, the terms "disposed," "mounted," "connected," and "connected" are to be construed broadly and may for example be fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meaning of the above terms in the present invention can be understood in specific cases to those skilled in the art.

In order to explain the technical solution of the present invention, the following detailed description is made with reference to the specific drawings and examples.

Referring to fig. 1, the embodiment provides an image capturing system, which includes a control module 1, a transmission module 2 and a plurality of camera modules 3, wherein the control module 1 includes a first processor 11 and a pulse trigger unit 12, the transmission module 2 includes a deserializer 21 and a serializer 22, the first processor 11 is connected to the pulse trigger unit 12, the first processor 11, the pulse trigger unit 12 and the serializer 22 are respectively connected to the deserializer 21, and the serializer 22 is connected to the camera modules 3.

In this embodiment, the control module 1, the transmission module 2 and the camera module 3 may be connected in sequence. The connection mode can include wired connection or wireless connection, and only the control module 1, the transmission module 2 and the camera module 3 need to be ensured to realize data communication.

As shown in fig. 1, in the present embodiment, the control module 1 may include a first processor 11 and a pulse trigger unit 12, and the transmission module 2 may include a deserializer 21 and a serializer 22. The pulse trigger unit 12 is configured to send a pulse signal to the camera module 3 to trigger the camera module 3 to shoot. The pulse triggering unit 12 may be connected to the first processor 11 so that the first processor 11 receives information of the transmission pulse signal.

The pulse trigger unit 12, the deserializer 21, the serializer 22 and the camera module 3 may be connected in sequence. Specifically, the pulse signal output end of the pulse trigger unit 12 may be connected to the pulse signal input end of the deserializer 21, the pulse signal output end of the deserializer 21 may be connected to the pulse signal input end of the serializer 22, and the pulse signal output end of the serializer 22 may be connected to the image device 32.

In a possible embodiment, the pulse signal sent by the pulse trigger unit 12 is a General-purpose input/output (GPIO) signal, and the deserializer 21 and the serializer 22 convert the GPIO signal into an LVDS signal for high-speed long-distance transmission, so that the problem of interference resistance in far-end transmission can be solved. The pulse signal carries out remote transmission through deserializer 21 and serializer 22 and reachs camera module 3, plays the effect that triggers a plurality of camera modules 3 in step and carry out the shooting, guarantees that a plurality of camera modules 3 gather image data in step, and then makes the image concatenation effect better.

The first processor 11 may also be connected to a deserializer 21. Specifically, the image data output end of the camera module 3 may be connected to the image data input end of the serializer 22, the image data output end of the serializer 22 may be connected to the image data input end of the deserializer 21, and the image data output end of the deserializer 21 may be connected to the first processor 11.

In one possible embodiment, the camera module 3 acquires image data, and the image data may be converted into a Differential Signaling (LVDS) signal by a serializer 22 and then converted into a serial signal by a deserializer 21, so as to remotely transmit multiple paths of image data to the first processor 11. The anti-interference performance of the image data in the long-distance transmission is ensured, and the data loss is avoided, so that the quality of the image data transmitted to the first processor 11 is ensured, and the image splicing effect is better.

The image capturing system provided by the embodiment includes a control module 1, a transmission module 2 and a plurality of camera modules 3, the control module 1 may include a first processor 11 and a pulse trigger unit 12, and the transmission module 2 may include a deserializer 21 and a serializer 22. The pulse trigger unit 12 sends out a pulse signal, the pulse signal is remotely transmitted to the camera module 3 through the deserializer 21 and the serializer 22, the camera module 3 is triggered to acquire image data, and the image data is converted into a serial differential signal through the serializer 22 and the deserializer 21 and is remotely transmitted to the first processor 11.

The remote transmission of the pulse signals is carried out through the deserializer 21 and the serializer 22, the multiple camera modules 3 can be triggered simultaneously, the image acquisition synchronism is guaranteed, meanwhile, the deserializer 21 and the serializer 22 are used for carrying out remote transmission of image data, the quality of the image data transmitted to the first processor 11 can be guaranteed, and therefore the subsequent image splicing effect is better.

In one example, the serializer 22 may be a DS90UB913 serializer, the deserializer 21 may be a DS90UB954 deserializer, and in this example, each DS90UB954 deserializer may receive two paths of image data. The DS90UB913 serializer converts the DVP signal of the image data into an LVDS signal, and the DS90UB954 deserializer uses two virtual channels to combine two corresponding LVDS signals into one serial signal, thereby realizing image acquisition and remote transmission of the plurality of camera modules 3.

In one embodiment, the first processor 11 includes a MIPI interface for connecting with the deserializer 21.

In this embodiment, the first Processor 11 and the deserializer 21 may be connected through a Mobile Industry Processor Interface (MIPI). The MIPI interface is differential serial transmission, and the speed is high; interface signal lines are few, and due to the fact that the low-voltage differential signals are adopted, generated interference is small, and anti-interference capacity is high. The transmission quality and efficiency of the image data are further ensured, so that the subsequent image splicing effect is better.

In a possible embodiment, after the LVDS signal transmitted by the serializer 22 is transmitted to the deserializer 21, the LVDS signal is further converted into a serial MIPI CSI-2 signal according to the image transmission characteristics of the camera module 3 and the first processor 11, and then the MIPI CSI-2 signal is transmitted to the first processor 11 through the MIPI interface. The image transmission characteristics comprise an image transmission format, a parallel transmission bit number, a transmission rate and the like.

The first processor 11 may use the haisi chip Hi3559AV100 platform. The Haisi chip Hi3559AV100 platform supports 16 × Lane MIPI interfaces at most, wherein the MIPI interfaces comprise various working modes such as 1 × 16Lane, 2 × 8Lane, 4 × 4Lane, 2 × 4Lane +4 × 2Lane and 8 × 2 Lane.

In this embodiment, the camera module 3 may include the image device 32, wherein the image device 32 may be a high-definition digital camera, and each camera interface needs to use 4Lane differential signals in order to meet the bandwidth speed requirement of the high-definition digital camera, i.e. the haisi chip Hi3559AV100 platform may use a 4 × 4Lane operating mode in this embodiment. The Haisi chip Hi3559AV100 platform can be connected with the deserializer 21 through a four-way 4Lane MIPI interface.

In one embodiment, the pulse triggering unit 12 comprises a microcontroller, which is connected to the deserializer 21.

In this embodiment, the pulse trigger Unit 12 may adopt a Microcontroller (MCU), specifically, the MCU may adopt an SPC5744 vehicle MCU, and the MCU simulates a plurality of square wave pulse signals with corresponding frequencies through an internal self-contained timer and a plurality of GPIOs. And the multi-path pulse signals are respectively connected into the multi-path GPIO input ports of the deserializer 21, so that the transmission of the pulse signals is realized.

In one embodiment, the deserializer 21 includes an FPD-Link interface and/or a GMSL interface for connecting with the serializer 22.

In this embodiment, the deserializer 21 may be connected to the serializer 22 through an FPD-Link interface and/or a GMSL interface, where FPD-Link is a high-speed differential transmission bus designed by TI and is mainly used to transmit image data, and GMSL is a gigabit multimedia serial Link.

In one embodiment, the deserializer 21 and the serializer 22 are communicatively coupled via the FPD-Link III protocol.

In the present embodiment, the deserializer 21 and the serializer 22 may be communicatively connected by the FPD-Link iii protocol. Specifically, the FPD-Link iii protocol communication connection is connected in a hardware manner, for example, the connection may be between an FPD-Link interface of the deserializer 21 and an FPD-Link interface of the serializer 22. In the long-distance transmission of the synchronous pulse signals, a mode of combined application of a microcontroller MCU and an FPD-Link III can be adopted, so that the problem of long-distance transmission of the pulse signals is solved, the anti-interference requirement of the long-distance transmission of the pulse signals is met, and the requirement of synchronously triggering low-delay control of the multi-path camera module 3 is met.

As shown in fig. 2, the deserializer 21 is a DS90UB954 deserializer, the serializer 22 is a DS90UB913 serializer, the transmission module 2 uses the FPD-Link iii protocol communication connection scheme of texas instruments, and the number of the camera modules 3 is eight, wherein the camera modules 3 may include the image device 32.

The MCU sends four paths of pulse signals, the four paths of pulse signals are respectively accessed to four paths of GPIO input ports of the DS90UB954 deserializers, the GPIO signals are converted into LVDS differential signals through BC-GPIO return channels of the DS90UB954 deserializers through FPD-Link III protocols, the LVDS differential signals are transmitted to the DS90UB913 serializer at the far end, and then the LVDS differential signals are sent to a framesync pin of the connected image equipment 32 through GPIO pins of the DS90UB913 serializer, and finally synchronous triggering of the far-end multi-channel image equipment 32 is achieved. The specific process of converting the GPIO signal into the LVDS differential signal by the FPD-Link iii protocol may be implemented by the prior art, which is not described herein again.

The embodiment transmits the GPIO signal through the LVDS differential signal, and can solve the problem of interference resistance in remote transmission. Meanwhile, the frame period of the BC-GPIO return channel of the DS90UB954 deserializer is 600ns (30 bits multiplied by 20ns/bit), and the requirement that the synchronous trigger delay error is required to be below the ms level is completely met.

In one embodiment, the transmission module 2 includes a plurality of deserializers 21, and at least one serializer 22 is connected to each deserializer 21, and the serializers 22 are connected to the camera modules 3 in a one-to-one correspondence.

In this embodiment, the deserializer 21 may be a DS90UB954 deserializer, or may be another deserializer 21 supporting multiple virtual channel transmission. The serializer 22 can be a DS90UB913 serializer, or a DS90UB933 serializer or a DS90UB953 serializer, and can achieve long-distance transmission of up to 15 meters. The transmission module 2 may include a plurality of deserializers 21, each deserializer 21 may be connected to one or more serializers 22, the number of serializers 22 is the same as that of the camera modules 3, and the serializers 22 are connected to the camera modules 3 in a one-to-one correspondence, that is, each image device 32 corresponds to one serializer 22.

As shown in fig. 3, in one possible implementation, the first processor 11 is a haisi chip Hi3559AV100 platform, the deserializer 21 is a DS90UB954 deserializer, the serializer 22 is a DS90UB913 serializer, and the number of the camera modules 3 is eight. The transmission module 2 comprises four DS90UB954 deserializers and eight DS90UB913 serializers, the DS90UB913 serializers convert DVP signals of image data into LVDS differential signals for long-distance transmission, and the LVDS signals are converted into MIPI CSI-2 signals at the end of the DS90UB954 deserializers and then are accessed to an MIPI interface of a Haisi chip Hi3559AV100 platform.

Since the Haisi chip Hi3559AV100 platform has four paths of 4lane MIPI interfaces, the Hi chip Hi3559AV100 platform can only be connected with four paths of image equipment 32 at most. To access more than four lanes of the image device 32, each 4lane MIPI interface may be accessed to multiple virtual receive lanes. Each 4lane MIPI interface can support four virtual receive lanes while the DS90UB954 deserializer supports four virtual lane transmission.

In this embodiment, the number of the camera modules 3 is eight, and in this scheme, each DS90UB954 deserializer receives two paths of image data of the image device 32 and transmits the image data by using two paths of virtual channels. Each DS90UB954 deserializer may support two Remote Exchange (RX) paths, each RX path accessing one DS90UB913 serializer path, i.e., one DS90UB954 deserializer may access two image devices 32.

In one embodiment, the first processor 11 includes an image acquisition unit and an image stitching unit, and the image stitching unit, the pulse trigger unit 12 and the deserializer 21 are respectively connected with the image acquisition unit.

In this embodiment, the first processor 11 may include an image acquisition unit and an image stitching unit. As shown in fig. 3, in a possible embodiment, the camera module 3 may include a second processor 31 and a video device 32, the video device 32 acquires image data, processes the image data by a pre-ISP (i.e., the second processor 31), transmits the image data to an image transmission unit (i.e., the transmission module 2), and transmits the image data to the Hi3559av100 main control image acquisition unit through the serializer 22 and the deserializer 21 (i.e., converts a DVP signal of the image data into an Lvds differential signal for long-distance transmission, and converts the Lvds signal into an MIPI CSI-2 signal for transmission to the image acquisition unit).

The image acquisition unit transmits the acquired images to the image splicing unit so as to splice the images. The image stitching Unit may be a Central Switching Unit (CSU), and the CSU may implement image stitching recognition.

In one embodiment, the camera module 3 includes a second processor 31 and a camera 32, the second processor 31 is connected to the camera 32, and the second processor 31 and the camera 32 are respectively connected to the serializer 22.

In the present embodiment, the image device 32 acquires raw image data and transmits the raw image data to the second processor 31 for processing. The second Processor 31 may be an Image Signal Processor (ISP), and performs post-processing such as linear correction, noise removal, dead pixel removal, interpolation, white balance, automatic exposure control, and the like on the original Image data acquired by the imaging device 32, so as to ensure the imaging quality of the imaging device 32. The specific process of processing the original image data by the ISP can be implemented by the prior art, and is not described herein again.

For ease of understanding, the following description will take as an example that the ISP needs to process the RAW format into the YUV format. Each camera module 3 corresponds to one image device 32 and one ISP, the ISP converts RAW image data in RAW format into YUV format after processing, and finally each camera module 3 directly outputs YUV image data without using the ISP provided by the control module 1 to sequence the RAW image data during subsequent image acquisition or image splicing, so that multiple pieces of RAW image data are synchronously processed, the image processing efficiency is further ensured, and the image splicing effect is better.

The ISP and the image device 32 corresponding to each camera module 3 may be an ann ISP and an ann sensor, or other image devices 32 supporting a multi-input filtering synchronization mode (framesync) to control synchronous trigger exposure.

In one embodiment, the video device 32 is a high definition digital camera, which is connected to the second processor 31 via a CSI-2 interface.

In this embodiment, in order to meet the requirement of image stitching identification, the acquired image data needs to be a high-definition image, and based on this, the image device 32 may be a high-definition digital camera, specifically, the high-definition digital camera may be a conventional 720P camera, a 1080P camera, or may be a 4K ultra-high-definition digital camera.

The high-definition digital camera can be connected with the second processor 31 through the CSI-2 interface, the CSI-2 interface can be very flexibly simplified or expanded, and for an application scene with few interfaces, the CSI-2 interface can complete the data serial transmission process of the camera only by using a group of differential data signal lines and a group of differential clock lines, so that the load is reduced, and a certain transmission rate can be met.

The embodiment also provides a vehicle, which comprises a vehicle body and the image acquisition system.

The embodiment provides a vehicle, and an image acquisition system is installed on a vehicle body. For example, an image capture system is mounted on the body of a commercial vehicle. Because the automobile body of commercial car is longer, when carrying out look around concatenation, Advanced Driving Assistance System (ADAS) to commercial car and detecting, because automobile body length reason, need multichannel camera can the full coverage, and the transmission distance of image data is far away.

It should be noted that the commercial vehicle may include all trucks and passenger cars with more than 9 seats, and may be a passenger car, a truck, a semi-trailer tractor, a non-complete vehicle of a passenger car, a non-complete vehicle of a truck, and the like. The vehicle may be a commercial vehicle, or may be a general vehicle having the arrangement requirement of the number of cameras, such as a car.

By adopting the image acquisition system, pulse signals and image data can be transmitted in a long distance, and synchronous triggering and synchronous image acquisition of the multi-channel image equipment 32 are ensured, so that the image splicing effect is better, and the precision of the panoramic splicing and ADAS detection of the commercial vehicle can be improved.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.

The principles and embodiments of the present invention have been explained herein using specific examples, which are presented only to assist in understanding the methods and their core concepts. It should be noted that there are infinite specific structures due to the limited character expressions, and it will be apparent to those skilled in the art that various improvements, decorations or changes can be made without departing from the principles of the present invention, and the technical features can be combined in a suitable manner; the application of these modifications, variations or combinations, or the application of the concepts and solutions of the present invention in other contexts without modification, is not intended to be considered as a limitation of the present invention.

Claims (10)

1. The utility model provides an image acquisition system, its characterized in that, includes control module group, transmission module group and a plurality of camera module group, the control module group includes first treater and pulse trigger unit, the transmission module group includes deserializer and serializer, first treater with the pulse trigger unit is connected, first treater the pulse trigger unit with the serializer respectively with the deserializer is connected, the serializer with the camera module group is connected.

2. The image acquisition system of claim 1 wherein the first processor comprises a MIPI interface for connecting with the deserializer.

3. The image acquisition system of claim 1 wherein the pulse trigger unit comprises a microcontroller, the microcontroller being coupled to the deserializer.

4. The image acquisition system of claim 3 wherein the deserializer comprises a FPD-Link interface and/or a GMSL interface for connecting with the serializer.

5. The image acquisition system of claim 4 wherein the deserializer and the serializer are communicatively coupled via an FPD-Link III protocol.

6. The image capturing system of claim 1, wherein the transmission module comprises a plurality of deserializers, and at least one serializer is connected to each deserializer, and the serializers are connected to the camera modules in a one-to-one correspondence.

7. The image acquisition system of claim 1, wherein the first processor comprises an image acquisition unit and an image stitching unit, the pulse triggering unit, and the deserializer being connected to the image acquisition unit, respectively.

8. The image capturing system of claim 1, wherein the camera module includes a second processor and a camera device, the second processor is connected to the camera device, and the second processor and the camera device are respectively connected to the serializer.

9. The image acquisition system of claim 8 wherein the video device is a high definition digital camera connected to the second processor through a CSI-2 interface.

10. A vehicle characterized by comprising a vehicle body and an image capturing system as claimed in any one of claims 1 to 9.

Images