CN114071080A - Automatic driving platform based on imaging adjustment intelligent gateway - Google Patents
Automatic driving platform based on imaging adjustment intelligent gateway Download PDFInfo
- Publication number
- CN114071080A CN114071080A CN202111267539.XA CN202111267539A CN114071080A CN 114071080 A CN114071080 A CN 114071080A CN 202111267539 A CN202111267539 A CN 202111267539A CN 114071080 A CN114071080 A CN 114071080A
- Authority
- CN
- China
- Prior art keywords
- intelligent gateway
- chip
- vehicle
- isp
- imaging
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003384 imaging method Methods 0.000 title claims abstract description 49
- 238000000034 method Methods 0.000 claims abstract description 14
- 230000008569 process Effects 0.000 claims abstract description 10
- 230000005540 biological transmission Effects 0.000 claims description 29
- 230000006978 adaptation Effects 0.000 claims description 11
- 239000013307 optical fiber Substances 0.000 claims description 9
- 230000003287 optical effect Effects 0.000 claims description 7
- RVCKCEDKBVEEHL-UHFFFAOYSA-N 2,3,4,5,6-pentachlorobenzyl alcohol Chemical compound OCC1=C(Cl)C(Cl)=C(Cl)C(Cl)=C1Cl RVCKCEDKBVEEHL-UHFFFAOYSA-N 0.000 claims description 6
- 238000004891 communication Methods 0.000 claims description 6
- 230000009286 beneficial effect Effects 0.000 abstract 1
- 230000006870 function Effects 0.000 description 9
- 238000010586 diagram Methods 0.000 description 4
- 102100034112 Alkyldihydroxyacetonephosphate synthase, peroxisomal Human genes 0.000 description 2
- 101000799143 Homo sapiens Alkyldihydroxyacetonephosphate synthase, peroxisomal Proteins 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 2
- 238000000848 angular dependent Auger electron spectroscopy Methods 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000000446 fuel Substances 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 239000003208 petroleum Substances 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/22—Adaptations for optical transmission
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
Abstract
The invention provides an automatic driving platform based on an imaging adjustment intelligent gateway, which comprises: the vehicle-mounted camera is connected with the intelligent gateway through a vehicle-mounted Ethernet bus; original bayer format video data acquired by the vehicle-mounted camera is transmitted to the intelligent gateway through an IEEE1722 AVTP protocol; the intelligent gateway at least comprises: an ISP chip; the ISP chip processes the data in the original bayer format and outputs an image of an RGB space domain to be stored in a cache for being called by external equipment; or the ISP chip controls the lens and the image sensor of the vehicle-mounted camera and the logic circuit unit of the ISP chip according to the data processing result of the bayer format in a feedback mode. The technical scheme provided by the invention can adjust the imaging of the vehicle-mounted camera at the intelligent gateway, and is beneficial to different hosts to directly acquire video data.
Description
Technical Field
The invention relates to the field of image acquisition of an automatic driving platform, in particular to an automatic driving platform based on an imaging adjustment intelligent gateway.
Background
The intelligent automobile is a comprehensive system integrating functions of environmental perception, planning decision, multi-level auxiliary driving and the like, and the intelligent automobile mainly provides safety and comfort of the automobile and provides an excellent man-vehicle exchange interface at present. Such as vehicle-mounted entertainment, automatic driving, active braking and the like, have become standard accessories of intelligent automobiles. The functions are realized on the basis of visual images, images are acquired through videos, the images are analyzed in real time, and automatic driving, vehicle-mounted entertainment, active braking and the like are carried out according to the image analysis results. However, the conventional image capturing system, as shown in fig. 1, firstly captures images by a conventional vehicle-mounted camera using an LVDS interface, then forming an MIPI signal the MIPI image signal of the Sensor is converted into a proprietary serial protocol through the FPD-link Serializer connector, and then transmitted through the FPD _ link coax or stp transmission medium, then converted to MIPI signals at the receiving end using the De-seralizer connector in the FPD-link III hub, then the MIPI signal is transmitted to a host computer of the platform for image processing, such as an automatic driving platform host computer or a look-around platform host computer, the essence of this method is that the MIPI is extended and the control signal is multiplexed, the video data transmission of each camera requires an exclusive channel, for example, in the FPD-link III hub, each camera needs to be configured with a transmission channel, and the transmission channels of data cannot be combined for transmission. And the protocol of the connector of the camera produced by different manufacturers is private and not universal. If the camera is replaced with another camera, the connector needs to be replaced again due to incompatibility of the proprietary protocol, so that the hardware and the software are highly coupled. In fig. 2, a camera in the prior art integrates an ISP chip and an image processor. In addition, there is also a vehicle-mounted camera based on an interface of LVDS transmission, as shown in fig. 3, the vehicle-mounted camera is not provided with an ISP chip, the ISP chip is bound on the SOC processing chip, and the MIPI signal can only be directly transmitted to the ISP chip, so that data acquisition can be performed only through the SOC processing chip of a corresponding platform, and video data cannot be directly acquired in other ways, which limits applications in various occasions. For example, the vehicle-mounted all-round needs adopt images, or a camera of the vehicle-mounted all-round needs is arranged, or the vehicle-mounted all-round needs goes to a host computer through an automatic driving platform, but because an automatic driving platform ISP chip is used for the image service needed by automatic driving, the automatic driving platform ISP chip cannot meet the all-round requirements, debugging needs to be carried out again, but configuration can be changed after debugging, and adaptation cannot be carried out.
Disclosure of Invention
Based on the defects in the prior art, the invention provides an automatic driving platform based on an imaging adjustment intelligent gateway, which comprises: the vehicle-mounted camera is connected with the intelligent gateway through a vehicle-mounted Ethernet bus;
original bayer format video data acquired by the vehicle-mounted camera is transmitted to the intelligent gateway through an IEEE1722 AVTP protocol;
the intelligent gateway at least comprises: an ISP chip; the ISP chip processes the data in the original bayer format and outputs an image of an RGB space domain to be stored in a cache for being called by external equipment; or the ISP chip controls the lens and the image sensor of the vehicle-mounted camera and the logic circuit unit of the ISP chip according to the data processing result of the bayer format in a feedback mode.
The utility model provides an autopilot platform based on formation of image regulation intelligent gateway, it is further optional, when the vehicle-mounted camera connector quantity of reserving in the intelligent gateway is less than vehicle-mounted camera quantity, the data of original bayer form that vehicle-mounted camera gathered first export external ethernet switch, then assemble to intelligent gateway through external ethernet switch.
An automatic driving platform based on imaging adjustment intelligent gateway is further optional, if external equipment needs to call video data of a vehicle-mounted camera, the external equipment and the intelligent gateway establish communication connection based on SOA service, and the intelligent gateway processes an original bayer format image through an ISP chip, converts the original bayer format image into a data format required by corresponding external equipment, stores the data format in a cache and sends the data format to the corresponding external equipment;
the external device at least comprises a host containing an SoC chip and an ARM chip.
An automatic driving platform for adjusting an intelligent gateway based on imaging, further optionally, the ISP chip at least comprises: ISP logic circuit unit and firmware, wherein the firmware is packaged with an image adjusting processing algorithm in a bayer format.
An automatic driving platform based on imaging adjustment intelligent gateway, further optionally, the intelligent gateway further comprises: gateway body, install the PCBA circuit board on the body, the PCBA circuit board is equipped with: the AVB/TSN Ethernet gateway, the ISP chip, the MCU chip and the connector, wherein the MCU chip and the connector are respectively connected with the AVB/TSN Ethernet gateway.
An automatic driving platform based on imaging adjustment intelligent gateway is further optional, wherein a vehicle-mounted Ethernet transmission bus between a vehicle-mounted Ethernet communication camera and the intelligent gateway comprises a shielding STP single-pair twisted-pair cable with a transmission rate of 1G or 10G;
or 10G vehicle ethernet transmission bus comprises optical fiber, using optical fiber as transmission medium.
An automatic driving platform based on imaging adjustment intelligent gateway, further optionally, the connector at least comprises one or more of an ethernet PHY chip, a CAN chip, an RGMII chip, and a PCIE PHY chip;
the Ethernet PHY chip at least comprises an electrical interface PHY chip or an optical interface PHY chip;
when the PHY chip of the electrical interface is adopted, the transmission medium adopts a shielded twisted pair; when the optical interface PHY chip is used, the transmission medium uses an optical fiber as the transmission medium.
An automatic driving platform for adjusting an intelligent gateway based on imaging is further optional, wherein the firmware at least comprises three modules, and the first module comprises: ISP control unit and ISP basic algorithm; the second module includes: AE automatic exposure, AWB automatic white balance and AEB automatic exposure enclosing algorithm; the third module includes: an image sensor algorithm;
and the ISP control unit schedules the ISP basic algorithm and the second module, and calls the third module to respectively register function call-back to the ISP basic algorithm and the second module so as to realize differentiated sensor adaptation.
An automatic driving platform based on imaging adjustment intelligent gateway, further optionally, the intelligent gateway further comprises: and the H.264 decoding chip is used for compressing the video data to be transmitted and converting the compressed video data into a protocol conforming to H.264 for output.
An automatic driving platform based on an imaging adjustment intelligent gateway is further optional, and an AVB/TSN Ethernet protocol stack module or an AVB/TSN Ethernet gateway can analyze a data packet in a network layer; the data packet conforming to the AVB/TSN protocol can be converted into the data packet in other standard protocol formats, or the data packet in other protocols can be converted into the AVB/TSN protocol; or the data packet of the AVB/TSN protocol can be analyzed and packaged according to the format of the AVB/TSN protocol.
An automatic driving platform for adjusting an intelligent gateway based on imaging, further optionally, the second external device invoking video data through the intelligent gateway includes:
when the intelligent gateway receives the video signal acquired from the second external equipment, acquiring the priority of the second external equipment and the priority of the application scene; acquiring the priority of first equipment and the priority of an application scene;
comparing the priorities of the first equipment and the second external equipment, and selecting the imaging adjustment parameters of the ISP chip corresponding to the equipment with the higher priority to be applied to the imaging of the vehicle-mounted camera;
if the priorities of the first equipment and the second external equipment are the same, selecting imaging adjustment of ISP parameters corresponding to equipment with high application scene priority to be applied to imaging of the vehicle-mounted camera;
and converting the data analyzed by the ISP chip into a format required by the corresponding first equipment or second external equipment and outputting the format to the corresponding first equipment or second external equipment.
Has the advantages that:
by the technical scheme provided by the invention, a plurality of hosts can directly call video data from the image processing gateway in time to make decisions and plans such as active braking, target recognition, looking around images and the like, so that execution measures are adopted. Compared with the traditional scheme, after the image is acquired by the panoramic image host, the image needs to be acquired by an automatic driving host. After the technical scheme of the application is adopted, the image can be directly obtained from the image processing gateway. When the automatic driving host is not needed, the technical scheme can still provide video images. And can easily switch while transmitting image data to a plurality of hosts.
According to the technical scheme provided by the invention, the problem of adaptation of imaging adjustment parameters of an ISP chip in a vehicle-mounted camera called by a plurality of different hosts can be solved.
According to the technical scheme provided by the invention, the problem of channel monopolization can be solved.
Drawings
The following drawings are only schematic illustrations and explanations of the present invention, and do not limit the scope of the present invention.
Fig. 1 is a schematic structural diagram of a video signal acquisition scheme in the prior art;
fig. 2 is a schematic diagram of a hardware structure of a vehicle-mounted camera in the prior art.
Fig. 3 is a schematic diagram of a hardware structure of the vehicle-mounted camera according to the present invention.
Fig. 4 is a schematic view of an autopilot platform of the imaging adjustment intelligent gateway of the present invention.
Fig. 5 is a schematic structural diagram of an autopilot platform of the intelligent gateway with an external switch for image adjustment according to the present invention.
Fig. 6 is a schematic view of the video stream processing structure of the ISP chip according to the present invention.
Fig. 7 is a schematic view of a video capture method of an external device according to the present invention.
Detailed Description
For a more clear understanding of the technical features, objects, and effects herein, embodiments of the present invention will now be described with reference to the accompanying drawings, in which like reference numerals refer to like parts throughout. For the sake of simplicity, the drawings are schematic representations of relevant parts of the invention and are not intended to represent actual structures as products. In addition, for simplicity and clarity of understanding, only one of the components having the same structure or function is schematically illustrated or labeled in some of the drawings.
As for the control system, the functional module, application program (APP), is well known to those skilled in the art, and may take any suitable form, either hardware or software, and may be a plurality of functional modules arranged discretely, or a plurality of functional units integrated into one piece of hardware. In its simplest form, the control system may be a controller, such as a combinational logic controller, a micro-programmed controller, or the like, so long as the operations described herein are enabled. Of course, the control system may also be integrated as a different module into one physical device without departing from the basic principle and scope of the invention.
The term "connected" in the present invention may include direct connection, indirect connection, communication connection, and electrical connection, unless otherwise specified.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, values, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, values, steps, operations, elements, components, and/or groups thereof. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items
It should be understood that the term "vehicle" or "vehicular" or other similar terms as used herein generally includes motor vehicles such as passenger automobiles including Sport Utility Vehicles (SUVs), buses, trucks, various commercial vehicles, watercraft including a variety of boats, ships, aircraft, and the like, and includes hybrid vehicles, electric vehicles, plug-in hybrid electric vehicles, hydrogen-powered vehicles, and other alternative fuel vehicles (e.g., fuels derived from non-petroleum sources). As referred to herein, a hybrid vehicle is a vehicle having two or more power sources, such as both gasoline-powered and electric-powered vehicles.
The first embodiment of the present invention provides an automatic driving platform based on an imaging adjustment intelligent gateway, referring to fig. 3 to 7, which specifically includes: the vehicle-mounted camera is connected with the intelligent gateway through a vehicle-mounted Ethernet bus;
original bayer format video data acquired by the vehicle-mounted camera is transmitted to the intelligent gateway through an IEEE1722 AVTP protocol;
the intelligent gateway at least comprises: an ISP chip; the ISP chip processes the data in the original bayer format and outputs an image of an RGB space domain to be stored in a cache for being called by external equipment; or the ISP chip controls the lens and the image sensor of the vehicle-mounted camera and the logic circuit unit of the ISP chip according to the data processing result of the bayer format in a feedback mode.
In the automatic driving platform provided by the implementation, the intelligent gateway comprises an ISP chip and can adjust the imaging quality of the vehicle-mounted camera, and the original imaging quality of the vehicle-mounted camera, namely the imaging quality of the original Raw bayer format, can be controlled through the intelligent gateway instead of the image processing of the output three-channel primary colors after the imaging adjustment. The method and the system can enable the hosts of various platforms to conveniently call the images, and simultaneously can reduce hardware cost.
In particular, in the background art, the image transmission process of the automatic driving platform in the prior art is exclusive in channel and cannot be shared. When the number of cameras increases, corresponding hardware channels need to be added for transmission. To solve this problem, the present implementation provides the following solutions, specifically as follows:
when the number of reserved vehicle-mounted camera connectors in the intelligent gateway is less than that of the vehicle-mounted cameras, the data in the original bayer format collected by the vehicle-mounted cameras are firstly output to the external Ethernet switch and then are gathered to the intelligent gateway through the external Ethernet switch;
the Ethernet switch adopts IEEE1722 AVTP protocol for data transmission.
If the external equipment needs to call the video data of the vehicle-mounted camera, the external equipment and the intelligent gateway establish communication connection based on SOA service, and the intelligent gateway processes the original bayer pattern image through an ISP chip, converts the primitive bayer pattern image into a data pattern required by the corresponding external equipment, stores the data pattern in a cache and sends the data pattern to the corresponding external equipment;
the external device at least comprises a host containing an SoC chip and an ARM chip.
The ISP chip at least comprises: ISP logic circuit unit and firmware, wherein the firmware is packaged with an image adjusting processing algorithm in a bayer format.
Referring to fig. 6, after a lens of the vehicle-mounted camera projects an optical signal to a photosensitive area of an image sensor, the sensor performs photoelectric conversion, sends an original image in a Bayer format to an ISP chip, and the ISP chip performs algorithm processing to output an image in a format of RGB, YUV, YcrCb, and the like matched with a target to an external device;
the ISP chip controls the ISP logic circuit unit through firmware, so that the lens and the sensor are correspondingly controlled, and functions of automatic aperture, automatic exposure, automatic white balance and the like are further completed;
the ISP logic circuit unit can complete a part of algorithm processing, and can also count the real-time information of the current image, and the firmware can perform recalculation and feedback control on the lens, the sensor and the ISP logic circuit unit by acquiring the image statistical information of the ISP logic unit so as to achieve the purpose of automatically adjusting the image quality.
The firmware is comprised of three modules that are,
the first module includes: ISP control unit and ISP basic algorithm;
the second module includes: AE automatic exposure, AWB automatic white balance and AEB automatic exposure enclosing algorithm;
the third module includes: an image sensor algorithm.
And the ISP control unit schedules the basic algorithm and the second module, and calls the third module to respectively register function call-back to the ISP basic algorithm and the second module so as to realize differentiated sensor adaptation.
Different sensors register control functions to an ISP algorithm library in the form of callback functions;
when the ISP control unit schedules the basic algorithm and the second module, the initialization parameters are obtained through the callback function, and the sensor is controlled, such as exposure time adjustment, analog gain and digital gain adjustment, lens stepping focusing or aperture rotation and the like.
If the external equipment needs to call the video data of the vehicle-mounted camera, the external equipment and the intelligent gateway establish communication connection based on SOA service, and the intelligent gateway processes the original bayer pattern image through an ISP chip, converts the primitive bayer pattern image into a data pattern required by the corresponding external equipment, stores the data pattern in a cache and sends the data pattern to the corresponding external equipment;
the external device at least comprises a host containing an SoC chip and an ARM chip.
Specifically, the process of establishing a connection between an external device that needs to acquire video data and an intelligent gateway includes:
the client sends a request for acquiring service to the server, the server judges whether to provide service to the client according to the content of the service request after receiving the service request, and if the service is authorized, the server corresponds to the client and establishes connection;
the client sends request information, and the server processes response after receiving the request;
when the client side generates or updates the events subscribed by the server, the server sends the updated content to the client side.
The intelligent gateway further comprises: gateway body, install the PCBA circuit board on the body, the PCBA circuit board is equipped with: the device comprises an AVB/TSN Ethernet gateway, an ISP chip, an MCU chip, an H.264 decoding chip, an MCU chip and a connector, wherein the H.264 decoding chip, the MCU chip and the connector are respectively connected with the AVB/TSN Ethernet gateway.
The vehicle-mounted Ethernet transmission bus positioned between the vehicle-mounted Ethernet through camera and the intelligent gateway comprises a shielding STP single-pair twisted pair line with the transmission rate of 1G or 10G;
or 10G vehicle ethernet transmission bus comprises optical fiber, using optical fiber as transmission medium.
The connector at least comprises one or more of an Ethernet PHY chip, a CAN chip, an RGMII chip and a PCIE PHY chip;
the PHY chip of the Ethernet at least comprises an electrical interface PHY chip or an optical interface PHY chip;
when the PHY chip of the electrical interface is adopted, the transmission medium adopts a shielded twisted pair; when the optical interface PHY chip is used, the transmission medium uses an optical fiber as the transmission medium.
And the H.264 decoding chip is used for compressing the video data to be transmitted and converting the compressed video data into a protocol conforming to H.264 for output.
The AVB/TSN Ethernet protocol stack module or the AVB/TSN Ethernet gateway can analyze the data packet in the network layer; the data packet conforming to the AVB/TSN protocol can be converted into the data packet in other standard protocol formats, or the data packet in other protocols can be converted into the AVB/TSN protocol; or the data packet of the AVB/TSN protocol can be analyzed and packaged according to the format of the AVB/TSN protocol.
The AVB/TSN protocol includes at least: IEEE1722 AVTP audio video transport protocol.
Or according to the use of the video data, such as ADAS auxiliary driving, automatic driving and other scenes, the method can add the following steps in the AVB/TSN protocol stack: IEEE 802.1AS, precision clock timing and synchronization protocol (gPTP); IEEE 802.1 Qat: stream Reservation Protocol (SRP); IEEE 802.1 Qav: time sensitive flow queuing and forwarding protocol (FQTSS); IEEE 802.1 BA: and the audio and video bridging system is used for defining a network configuration file.
By the technical scheme provided by the invention, a plurality of hosts can directly call video data from the image processing gateway in time to make decisions and plans such as active braking, target recognition, looking around images and the like, so that execution measures are adopted. Compared with the traditional scheme, after the image is acquired by the panoramic image host, the image needs to be acquired by an automatic driving host. After the technical scheme of the application is adopted, the image can be directly obtained from the image processing gateway. When the automatic driving host is not needed, the technical scheme can still provide video images. And can easily switch, send image data to a plurality of host computers simultaneously, reduce the delay.
Specifically, the vehicle-mounted camera is suitable for many occasions, such as ADAS auxiliary driving, 360-degree around view and vehicle-mounted entertainment, and has different requirements on the imaging quality of the vehicle-mounted camera. In the prior art, cameras are usually added on a vehicle aiming at application scenes, and the sharing of the cameras cannot be realized. In the technical scheme provided by this embodiment, since a plurality of different hosts are involved to call the vehicle-mounted camera, but the application scenes of each host may be the same or different, and different application scenes may cause different imaging parameters of the ISP chip to the vehicle-mounted camera, the technical problem of how to adapt the imaging parameters of the vehicle-mounted camera to the external device and the application scenes without affecting the use needs to be solved.
The concrete solution is as follows:
step S1: calling a corresponding ISP chip according to the application scene to call an internal processing method, adjusting the imaging parameter of the corresponding ID camera and storing the imaging parameter;
step S2: setting the priority of external equipment and the priority of an application scene;
specifically, the priority of the external device can be set when called by the user, and if not, the system automatically sets the priority of the external device. If the priority of the automatic driving host is larger than that of the all-around system host by default, but if the user sets the priority when the user obtains videos during backing at the moment, the priority can be improved, and the default setting is replied after the use;
therefore, when backing a car, obviously, the user does not start the automatic driving mode, and therefore, the driving safety is not influenced.
The second external device includes a device that is about to call video data, but does not include a device that is calling video data; the first device defines a device that includes data that is using video;
step S3: when the intelligent gateway receives the video signal acquired from the second external equipment, acquiring the priority of the second external equipment and the priority of the application scene;
acquiring the priority of first equipment and the priority of an application scene;
step S4: firstly, the priority of the first device and the second device is judged
If the priority of the first equipment is higher than that of the second external equipment, judging whether the image formats required by the second external equipment are the same, if so, directly transmitting the image data to the second external equipment;
if the priority of the first equipment is smaller than that of the second external equipment, interrupting the first equipment which is calling the video data currently; the ISP chip calls imaging parameters corresponding to the second external equipment to adjust the imaging of the camera to obtain an adaptive image; the adapted image is simultaneously output to the second external device and the first device. Specifically, it should be noted that, because the priority of the second device is high, the imaging debugging parameters of the ISP chip will be required by the second device, that is, the imaging parameters required by the second device are preferentially met, and the image obtained by the first device at this time is adjusted by the imaging parameters required by the second device, but does not include an image format, for example: RGB, YUV, YcrCb, etc. Thus the external device may require a different format, such as an RGB format image required by the screen, but the look-around effect may require a YUV format image. At this time, the image format needs to be converted into a format required by the corresponding device and then transmitted.
Step S4: if the priority of the first equipment is equal to the priority of the second external equipment, judging the priority of the application scene, if the priorities are the same, keeping the current imaging parameters unchanged, judging whether the image formats required by the second external equipment are the same, if so, directly transmitting the image data to the second external equipment, otherwise, respectively performing format conversion and then transmitting the image data; sharing video data to the first device and the second external device at the same time;
if the application scene priority of the first equipment is smaller than that of the second external equipment, interrupting the first equipment which is calling the video data currently; the ISP chip calls imaging parameters corresponding to the second external equipment to adjust the imaging of the camera to obtain an adaptive image. The adapted image is simultaneously output to the second external device and the first device.
What has been described above is only a preferred embodiment of the present invention, and the present invention is not limited to the above examples. It is clear to those skilled in the art that the form in this embodiment is not limited thereto, and the adjustable manner is not limited thereto. It is to be understood that other modifications and variations, which may be directly derived or suggested to one skilled in the art without departing from the basic concept of the invention, are to be considered as included within the scope of the invention.
Claims (10)
1. An autopilot platform based on imaging adjustment intelligent gateway, comprising: the vehicle-mounted camera is connected with the intelligent gateway through a vehicle-mounted Ethernet bus; original bayer format video data acquired by the vehicle-mounted camera is transmitted to the intelligent gateway through an IEEE1722 AVTP protocol;
the intelligent gateway at least comprises: an ISP chip; the ISP chip processes the data in the original bayer format and outputs an image of an RGB space domain to be stored in a cache for being called by external equipment; or the ISP chip controls the lens and the image sensor of the vehicle-mounted camera and the logic circuit unit of the ISP chip according to the data processing result of the bayer format in a feedback mode.
2. The automatic driving platform for adjusting the intelligent gateway based on the imaging as claimed in claim 1, wherein when the number of the reserved vehicle-mounted camera connectors in the intelligent gateway is less than the number of the vehicle-mounted cameras, the data in the original bayer format collected by the vehicle-mounted cameras are first output to the external ethernet switch, and then are converged to the intelligent gateway through the external ethernet switch.
3. The automatic driving platform based on the imaging adjustment intelligent gateway of claim 1, wherein if an external device needs to call video data of a vehicle-mounted camera, the external device establishes communication connection based on SOA service with the intelligent gateway, and the intelligent gateway processes an original bayer-format image through an ISP chip, converts the original bayer-format image into a data format required by a corresponding external device, stores the data format in a cache, and sends the data format to the corresponding external device;
the external device at least comprises a host containing an SoC chip and an ARM chip.
4. The imaging-based adaptation of an autopilot platform for intelligent gateways of claim 1, wherein the ISP chip comprises at least: ISP logic circuit unit and firmware, wherein the firmware is packaged with an image adjusting processing algorithm in a bayer format.
5. The imaging-based adaptation of an autopilot platform of an intelligent gateway of claim 1 wherein the intelligent gateway further comprises: gateway body, install the PCBA circuit board on the body, the PCBA circuit board is equipped with: the AVB/TSN Ethernet gateway, the ISP chip, the MCU chip and the connector, wherein the MCU chip and the connector are respectively connected with the AVB/TSN Ethernet gateway.
6. The imaging-based adaptation of the autopilot platform of the intelligent gateway of claim 1 wherein the vehicle ethernet transport bus connecting the vehicle camera to the intelligent gateway comprises a shielded STP single pair of twisted pair lines at 1G or 10G transmission rate;
or 10G vehicle ethernet transmission bus comprises optical fiber, using optical fiber as transmission medium.
7. The imaging-based, ad hoc intelligent gateway-based autopilot platform of claim 5 wherein the connectors comprise at least one or more of ethernet PHY chip, CAN chip, RGMII chip, PCIE PHY chip;
the Ethernet PHY chip at least comprises an electrical interface PHY chip or an optical interface PHY chip;
when the PHY chip of the electrical interface is adopted, the transmission medium adopts a shielded twisted pair; when the optical interface PHY chip is used, the transmission medium uses an optical fiber as the transmission medium.
8. The imaging-based adaptation of an automated driving platform for an intelligent gateway as claimed in claim 4, wherein the firmware comprises at least three modules, the first module comprising: ISP control unit and ISP basic algorithm; the second module includes: AE automatic exposure, AWB automatic white balance and AEB automatic exposure enclosing algorithm; the third module includes: an image sensor algorithm;
and the ISP control unit schedules the ISP basic algorithm and the second module, and calls the third module to respectively register function call-back to the ISP basic algorithm and the second module so as to realize differentiated sensor adaptation.
9. The imaging-based adaptation of an autopilot platform of an intelligent gateway of claim 1 wherein the intelligent gateway further comprises: and the H.264 decoding chip is used for compressing the video data to be transmitted, converting the compressed video data into a protocol conforming to H.264 and outputting the converted video data.
10. The imaging-based adaptation of the automated driving platform as claimed in claim 1, wherein the AVB/TSN ethernet protocol stack module or the AVB/TSN ethernet gateway is capable of parsing the data packet at the network layer; the data packet conforming to the AVB/TSN protocol can be converted into the data packet in other standard protocol formats, or the data packet in other protocols can be converted into the AVB/TSN protocol; or the data packet of the AVB/TSN protocol can be analyzed and packaged according to the format of the AVB/TSN protocol.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111267539.XA CN114071080A (en) | 2021-10-29 | 2021-10-29 | Automatic driving platform based on imaging adjustment intelligent gateway |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111267539.XA CN114071080A (en) | 2021-10-29 | 2021-10-29 | Automatic driving platform based on imaging adjustment intelligent gateway |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114071080A true CN114071080A (en) | 2022-02-18 |
Family
ID=80235996
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111267539.XA Pending CN114071080A (en) | 2021-10-29 | 2021-10-29 | Automatic driving platform based on imaging adjustment intelligent gateway |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114071080A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114598822A (en) * | 2022-03-02 | 2022-06-07 | 上海赫千电子科技有限公司 | Vehicle-mounted Ethernet all-around system with ESD interference resistance |
| CN114619962A (en) * | 2022-03-15 | 2022-06-14 | 上海赫千电子科技有限公司 | Automatic driving system with asymmetric transmission |
-
2021
- 2021-10-29 CN CN202111267539.XA patent/CN114071080A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114598822A (en) * | 2022-03-02 | 2022-06-07 | 上海赫千电子科技有限公司 | Vehicle-mounted Ethernet all-around system with ESD interference resistance |
| CN114619962A (en) * | 2022-03-15 | 2022-06-14 | 上海赫千电子科技有限公司 | Automatic driving system with asymmetric transmission |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114071080A (en) | Automatic driving platform based on imaging adjustment intelligent gateway | |
| US5991842A (en) | Communication system for providing digital data transfer, electronic equipment for transferring data using the communication system, and an interface control device | |
| CN111917988B (en) | Remote camera application method, system and medium of cloud mobile phone | |
| WO2019000676A1 (en) | Method and device for achieving three-dimensional image by using dual cameras | |
| JP2001500331A (en) | Video camera with general-purpose serial bus interface | |
| KR101677663B1 (en) | Method and device for implementing analog high-definition image capturing | |
| WO2020005899A1 (en) | Exchanging an hdr-combined stream and associated exposures between a camera sensor module and a vision processing system | |
| KR20210045106A (en) | Vehicle and controlling method thereof | |
| CN104702906A (en) | High-resolution monitoring video system and control method thereof | |
| CN113132552B (en) | Video stream processing method and device | |
| CN114598843A (en) | Image processing system and method applied to multi-path cameras of large automobile | |
| CN111988557A (en) | Vehicle-mounted video transmission method and vehicle-mounted video transmission system | |
| CN104991877B (en) | A kind of vehicle mounted guidance host efficiently unifies extended method | |
| CN208021321U (en) | A kind of vehicle-mounted viewing system of SD input high definition output | |
| CN216086672U (en) | Vehicle-mounted Ethernet intelligent gateway system with imaging adjusting function | |
| CN116567182A (en) | Domain control system, domain control method, vehicle, and storage medium | |
| CN216134481U (en) | Intelligent gateway with imaging adjusting function | |
| CN217546051U (en) | Repeater and image transmission system | |
| CN114501091B (en) | Method and device for generating remote driving picture and electronic equipment | |
| CN101778206A (en) | Double-lens video conference camera and image processing method thereof | |
| CN216057289U (en) | Vehicle cabin image data acquisition system and vehicle | |
| CN114913624B (en) | Data sharing method among domain controllers, vehicle-mounted system and vehicle | |
| KR102544747B1 (en) | Camera supporting multiple independent ethernet interfaces | |
| CN115297188B (en) | Communication connection method and system applied to intelligent automobile | |
| US20230171373A1 (en) | Multi-data transmission channel fusion apparatus and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |