CN109318799A - Automobile, automobile ADAS system and its control method - Google Patents
Automobile, automobile ADAS system and its control method Download PDFInfo
- Publication number
- CN109318799A CN109318799A CN201710641838.2A CN201710641838A CN109318799A CN 109318799 A CN109318799 A CN 109318799A CN 201710641838 A CN201710641838 A CN 201710641838A CN 109318799 A CN109318799 A CN 109318799A
- Authority
- CN
- China
- Prior art keywords
- image
- control data
- processor
- data
- data stream
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
- B60Q9/00—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C5/00—Registering or indicating the working of vehicles
- G07C5/08—Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
- G07C5/0808—Diagnosing performance data
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C5/00—Registering or indicating the working of vehicles
- G07C5/08—Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
- G07C5/0841—Registering performance data
- G07C5/085—Registering performance data using electronic data carriers
- G07C5/0866—Registering performance data using electronic data carriers the electronic data carrier being a digital video recorder in combination with video camera
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/08—Detecting or categorising vehicles
Landscapes
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Signal Processing (AREA)
- Mechanical Engineering (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a kind of automobile, automobile ADAS system and its control methods.Wherein, ADAS system includes camera, SOC chip, wherein camera is used to shoot the environment of motor vehicle environment to obtain video data stream, and SOC chip integrates multiple processors, and method includes: acquisition video data stream and CAN control data flow；Fusion treatment is carried out to collected video data stream and CAN control data flow, to generate the fused data stream comprising video data stream and CAN control data flow, wherein fused data stream includes multiframe blending image；Image procossing is carried out to every frame blending image to obtain parsing information and CAN control data accordingly；Corresponding image data is exported according to parsing information and CAN control data and/or generates corresponding control signal.The control method can reduce data transmitting of the data flow between each processor of SOC chip, reduce communication load, be conducive to the real-time for promoting ADAS system.
Description
Technical field
The present invention relates to automotive field more particularly to a kind of control methods of automobile ADAS system, a kind of automobile ADAS system
System and a kind of automobile.
Background technique
With being significantly increased year by year for China's car ownership, traffic accident is consequently increased.For more efficient protection
The security of the lives and property of driver and occupant, each automobile factory commercial city are being dedicated to developing relatively reliable vehicle-mounted passive peace
Entirely and active safety auxiliary system, wherein active safety auxiliary system is even more more and more of interest by people.
In the related technology, active safety auxiliary system using monocular cam and SOC chip (System-on-a-Chip,
System level chip) realize automobile ADAS (Advanced Driver Assistant Systems, advanced driving assistance system)
Function, wherein ADAS function includes deviation detection, pedestrian's identification, vehicle identification, speed(-)limit sign identification and traffic lights
Identification etc..In the technology, the control data flow of video data stream and CAN be it is concurrent, i.e., what video data flowed away is video flowing
Channel, and what the control data of CAN flowed away is another data flow path.
Since reason device obtains image data frame from video data stream and handles everywhere in SOC chip, while also from CAN
Control data flow obtain control data, and image data frame and control data throughout manage carried out between device data transmitting all
" internuclear communication " mode dependent on SOC chip can make the reality of whole system when " internuclear communication " is constantly in busy state
When property is deteriorated, and then influences the discrimination of image and the reliability of system.
Summary of the invention
The present invention is directed to solve one of the technical problem in above-mentioned technology at least to a certain extent.For this purpose, of the invention
One purpose is to propose a kind of control method of automobile ADAS system, this method can reduce managed everywhere in SOC chip device it
Between data transmitting, reduce the load of " internuclear communication ", be conducive to promoted ADAS system real-time.
Second object of the present invention is to propose a kind of automobile ADAS system.
Third object of the present invention is to propose a kind of automobile.
In order to achieve the above objectives, first aspect present invention embodiment proposes a kind of control method of automobile ADAS system,
The ADAS system includes camera, SOC chip, wherein the camera is used to shoot the environment of the motor vehicle environment to obtain
Video data stream is taken, multiple processors are integrated in the SOC chip, and at least one processor executes image processing algorithm,
At least one processor executes decision making algorithm, the described method comprises the following steps: an acquisition institute is selected from multiple processors
State video data stream and CAN control data flow；Select one to the collected video data stream from multiple processors and
The CAN control data flow carries out fusion treatment, controls melting for data flow comprising the video data stream and the CAN to generate
Close data flow, wherein the fused data stream includes multiframe blending image；When processor executes image processing algorithm, to every frame
Blending image carries out image procossing to obtain parsing information and CAN control data accordingly；When processor executes decision making algorithm, root
Corresponding image data is exported according to the parsing information and CAN control data and/or generates corresponding control signal.
CAN can be controlled data flow with picture number by the control method of automobile ADAS system according to an embodiment of the present invention
According to form be dissolved into video data stream, to obtain the fused data stream in the form of image data, and then existed by fused data stream
It manages everywhere in SOC chip and carries out data transmission between device.Thereby, it is possible to reduce the data in SOC chip between each processor
Transmission, reduces communication load, is conducive to the real-time for promoting ADAS system.
In order to achieve the above objectives, second aspect of the present invention embodiment proposes a kind of automobile ADAS system, comprising: camera shooting
Head, the camera are used to shoot the environment of the motor vehicle environment to obtain video data stream；SOC chip, in the SOC chip
It is integrated with multiple processors, wherein processor is used for: acquiring the video data stream and CAN control data flow, and to collecting
The video data stream and the CAN control data flow carry out fusion treatment, with generate include the video data stream and institute
State the fused data stream of CAN control data flow, wherein the fused data stream includes multiframe blending image, and is executing figure
When as Processing Algorithm, dissection process is carried out to every frame blending image to obtain parsing information and CAN control data accordingly, and
When executing decision making algorithm, corresponding image data is exported according to the parsing information and CAN control data and/or generates phase
The control signal answered.
Further, the invention proposes a kind of automobiles comprising the automobile ADAS system of the above embodiment of the present invention.
The automobile of the embodiment of the present invention can be reduced between each processor of SOC chip using above-mentioned automobile ADAS system
Data transmission, reduce communication load, be conducive to promoted ADAS system real-time.
Detailed description of the invention
Fig. 1 is the flow chart of the control method of automobile ADAS system according to an embodiment of the present invention；
Fig. 2 is the flow chart of the control method of automobile ADAS system accord to a specific embodiment of that present invention；
Fig. 3 is the flow chart of the control method step S102 of automobile ADAS system according to an embodiment of the invention；
Fig. 4 is the schematic diagram of an exemplary video image frame format according to the present invention；
Fig. 5 is the flow chart of the image processing algorithm of automobile ADAS system according to an embodiment of the present invention；
Fig. 6 is the flow chart of the decision making algorithm of automobile ADAS system according to an embodiment of the invention；
Fig. 7 is the block diagram of automobile ADAS system according to an embodiment of the present invention；
Fig. 8 is the block diagram of automobile according to an embodiment of the present invention.
Specific embodiment
The embodiment of the present invention is described below in detail, examples of the embodiments are shown in the accompanying drawings, wherein from beginning to end
Same or similar label indicates same or similar element or element with the same or similar functions.Below with reference to attached
The embodiment of figure description is exemplary, it is intended to is used to explain the present invention, and is not considered as limiting the invention.
Below with reference to the accompanying drawings the automobile proposed according to embodiments of the present invention, automobile ADAS system and its control method are described.
Fig. 1 is the flow chart of the control method of automobile ADAS system according to an embodiment of the invention.
In an embodiment of the present invention, ADAS system includes camera and SOC chip, wherein camera is for shooting vapour
Environment around vehicle is integrated with multiple processors to obtain video data stream in SOC chip, at least one processor executes image
Processing Algorithm, at least one processor execute decision making algorithm.
Wherein, camera can be monocular cam.
In this embodiment, framework between device is managed everywhere in integrating in SOC chip and dominant frequency is distinct, and user can be with
The concrete model of SOC chip is selected according to the specific demand of ADAS system, and it is specific to specify specific processor to implement
Function.
As shown in Figure 1, the control method of automobile ADAS system the following steps are included:
S101 passes through the acquisition video data stream and CAN control data flow in multiple processors.
Specifically, selected from multiple processors according to the dominant frequency of each processor one for acquire video data stream and
CAN controls data flow.Wherein, it if the difference of the maxima and minima in the dominant frequency of multiple processors is greater than preset value, selects
Select the smallest processor acquisition video data stream of dominant frequency and CAN control data flow.If the maximum in the dominant frequency of multiple processors
The difference of value and minimum value is less than or equal to preset value, then one processor acquisition video data stream of any selection and CAN control data
Stream.
S102 controls data flow to collected video data stream and CAN by one in multiple processors and melts
Conjunction processing, to generate the fused data stream comprising video data stream and CAN control data flow, wherein fused data stream includes more
Frame blending image.
Specifically, according to for acquire the video data stream and the CAN control data flow processor dominant frequency from
One is selected to carry out fusion treatment for controlling data flow to collected video data stream and CAN in multiple processors.Wherein,
If the dominant frequency of the processor for acquiring video data stream and CAN control data flow is more than or equal to predeterminated frequency, it is being used for
Collected video data stream and CAN control data are flowed on the processor of acquisition video data stream and CAN control data flow
Row fusion treatment；If the dominant frequency of the processor for acquiring video data stream and CAN control data flow is less than predeterminated frequency,
One is selected to carry out fusion treatment to collected video data stream and CAN control data flow from other processors.
S103, it is corresponding to obtain that processor carries out image procossing when executing image processing algorithm, to every frame blending image
Parsing information and CAN control data.
In an embodiment of the present invention, the image processing algorithm that processor executes in ADAS system can be, but not limited to be vehicle
Road deviation detection algorithm, traffic lights and pedestrian detection algorithm, speed(-)limit sign recognizer and vehicle identification algorithm etc..
S104, processor is when executing decision making algorithm, according to parsing information and the corresponding figure of CAN control data output
As data and/or generate corresponding control signal.
In this embodiment, if generating corresponding control signal, automobile can be carried out according to control signal corresponding
Control.
Specifically, as shown in Fig. 2, acquiring video data stream and CAN control data flow simultaneously by a processor, and lead to
It crosses a processor to be dissolved into CAN control data flow in video data stream in the form of image data, to obtain image data
The fused data stream of form, by fused data stream substitute CAN control data flow and video data stream in SOC chip everywhere in
Carry out data transmission between reason device, and then can according to need and image processing algorithm, such as traffic are executed by corresponding processor
Lamp and pedestrian's recognizer, deviation detection algorithm, speed(-)limit sign recognizer, vehicle identification algorithm, to fused data stream
Carrying out image procossing can be obtained corresponding control signal and/or the corresponding image data of output.Thereby, it is possible to reduce CAN control
Data transmitting of the data flow processed in SOC chip between each processor, reduces the communication load between processor, is conducive to be promoted
The real-time of ADAS system.
In one embodiment of the invention, as shown in figure 3, above-mentioned steps S102 may include:
S1021 carries out frame decoding processing to video data stream to obtain multi-frame video image, wherein the format of video image
For YUV420SP.
In this embodiment, if the format of video image is not YUV420SP, the format first by video image is needed
It is converted into YUV420SP.
S1022 is split processing to video image, to obtain the ROI region of every frame video image, and according to the area ROI
The Pixel Information in domain obtains current travelling data, wherein ROI region includes the sky areas in every frame video image, driving
Data include driving and driving at night in the daytime.
Specifically, processing is split to the video image currently obtained, to include sky portion in selecting video image
Region as a ROI region, then to the Pixel Information of the ROI region (such as luminance information) carry out statistics and analysis, just
Step judges that current driving conditions are in daytime, the night of being in get to current travelling data, and by number of driving a vehicle
It is recorded according to (i.e. driving/driving at night in the daytime).
S1023 carries out histogram equalization processing to the Y-component of every frame video image to generate corresponding equalization data.
Specifically, the equalization of a histogram is carried out to the Y-component of the video image currently obtained, and equalization is tied
Fruit is saved, thus, it is possible to avoid repeating the equalization processing that many places carry out histogram when then executing image processing algorithm.
S1024 obtains CAN from CAN control data flow and controls data.
Wherein, CAN, which controls data, can include but is not limited to current dipped headlight and high beam state, speed, steering wheel turn
Angle, braking state, the safe information of driver and/or occupant and system debugging, test and diagnosis request in one
Or it is multiple.
S1025 replaces the last line data of the Y value of every frame video image with travelling data and CAN control data, with
To blending image.
Specifically, the video image frame format currently obtained can as shown in figure 4, if the resolution ratio of camera be 720P,
The blending image frame format of the YUV420SP format then exported after fusion treatment is made of two parts data: being located at front position
What is set is 1280 × 720 Y values, and what it is positioned at back location is 1280 × 350 UV data being alternately present.
The Y value last line data (i.e. the last line of equalization data) of the video image of 720P are directly substituted for and include
The data of following information: (including current dipped headlight and high beam state, speed, steering wheel turn for travelling data, CAN control data
Angle, braking state and driver and member security's information；The information such as debugging, test and the diagnosis request of system).
Video data stream and CAN control data flow are after fusion treatment, and video data stream and CAN control data flow are to melt
The form transmission for closing data flow then continues through processor in ADAS system and executes each specific image processing algorithm to melting
Data flow is closed to be handled.Wherein, these image processing algorithm some operate in above the same processor of SOC chip, have
Operate in above different processors, be specifically dependent upon these image processing algorithms be need and meanwhile run or timesharing transport
Row.If multiple images Processing Algorithm is needed while being run, run on the different processors, if multiple images are handled
Algorithm can be with time-division operation, then it may operate in same processor.
In an embodiment of the present invention, each image processing algorithm can specific region in alignment processing blending image (i.e.
The specific region used itself carries out image procossing), for example, including in deviation detection algorithm alignment processing blending image
The region in lane includes the region of speed(-)limit sign in speed(-)limit sign recognizer alignment processing blending image.
When processor executes an image processing algorithm, after obtaining a frame blending image, it is initially directed to this and melts
The last line of the Y value of image is closed, then CAN therein control data are parsed.As shown in Figure 5, the specific steps are as follows:
S501 judges whether the CAN control data in the last line data of the Y value of blending image include debugging, test
It is requested with diagnosis, if it is not, executing step S504.
S502 judges whether ADAS system is external if including debugging, test and diagnosis request in CAN control data
HDMI shows equipment, if it is not, executing step S504.
S503 answers the specific region image data in blending image if the external HDMI of ADAS system shows equipment
Corresponding output state is made, and selects predeterminable area relevant to image processing algorithm from blending image, to selection
Predeterminable area carries out image procossing, to obtain parsing information accordingly.
Wherein, the specific region image data copied in output state can be used for corresponding when executing decision making algorithm
HMMI is shown in show in equipment.
S504, if ADAS system does not have external HDMI to show equipment, selection is calculated with image procossing from blending image
The relevant predeterminable area of method, and image procossing is carried out to the predeterminable area of selection, to obtain parsing information accordingly.
In an example of the invention, traffic lights and pedestrian's recognition detection algorithm and deviation can be detected
Algorithm executes in same processor, and their time-division operations are arranged, and e.g., when speed is greater than threshold value Vh, does not execute friendship
Logical lamp and pedestrian's recognition detection algorithm do not execute deviation detection algorithm when speed is less than threshold value Vl.
Furthermore it is also possible to according in blending image travelling data judgement be currently in the daytime driving or driving at night, into
And can be selected according to travelling data when carrying out deviation detection or vehicle identification etc. different specific Processing Algorithms come
The discrimination of image is improved, to reduce the probability of error detection and leak detection.For example, can scheme accordingly in driving at night
Front vehicles are identified as increasing the detection of automobile tail light in Processing Algorithm.
S505, by parsing information preservation to output state.
Specifically, if the external HDMI of ADAS system shows equipment, first specific region image data is saved to current
Output state, then will parse information and replace two row of foremost of current output state, most by the Y value of blending image
A line CAN controls data and copies to the position that the last two rows of current output state start afterwards, this is because specific region
Less than corresponding blending image.
If ADAS does not have external HDMI to show equipment, current output state does not include specific region picture number
According to will directly parse information at this time and replace two row of foremost of current output state, by the Y value last line of blending image
CAN control data copy to the position that the last two rows of current output state start, this is because specific region be less than pair
The blending image answered.
S506 spreads defeated output state to fused data.
Wherein, input buffer when which may be performed as decision making algorithm.
Further, as shown in fig. 6, specifically executing following steps when processor executes decision making algorithm:
S601 obtains an input buffer from fused data stream.
Wherein, in the input buffer can include: specific region image data (when there is external HDMI to show equipment), solution
It analyses information and CAN controls data.
It should be noted that the size of the input buffer is equal to the space size that specific region image data occupies.
S602 reads CAN from input buffer and controls data, and judges that the whether external HDMI of ADAS system is shown and set
It is standby, if so, S603 is thened follow the steps, it is no to then follow the steps S605.
Specifically, the position since input buffer the last two rows of storing data reads CAN and controls data,
S603 reads parsing information from input buffer, and generates corresponding image data according to parsing information.
Wherein, two row of foremost of specific region image data is the parsing letter of the image processing algorithm output of previous stage
Breath is deviated to the left or is deviated to the right for example, executing deviation detection algorithm to judge whether to detect deviation,
Or whether detect left and right lane line；It executes traffic lights and pedestrian detection algorithm and judgement has recognized how many a targets, this
Height, width and coordinate information, the horizontal and vertical range information also apart from optical axis of the corresponding identification frame of target a bit
Deng.
Further, corresponding image data can be generated according to above-mentioned parsing information, such as OSD (On-Screen-
Display, screen menu type display technology) figure, for example, by lane line, pedestrian, vehicle, traffic sign and traffic lights etc. with
The form of lines, wire frame or text is depicted.
Image data is superimposed in specific region by S604, and is shown in external HDMI and is shown in equipment.
It should be noted that in the development phase of ADAS system product, use HDMI show equipment output associated picture with
And data result, can be in order to search SOC chip software the problem of, it is also convenient for each processor in monitoring SOC chip
On each algorithm operating status.It is detected and is tested also by function of this mode to product in the later period, thus constantly
The function and performance for improving product.After product volume production, default can be not external HDMI and show equipment, but
Relevant control signal externally directly can be issued by vehicle body CAN network.
S605, according to the parsing information and/or CAN control data generation control signal in current input buffer.
For example, need to detect the safe information of driver and occupant when CAN control data are booting, and with sound or
Person graphically carries out necessary prompting on instrument board interface；It is not open left and right turn signal when CAN controls data,
When parsing information is that traveling lane is deviated considerably from, warning reminding can be carried out；When parsing information be detect red light and
There are pedestrian, the data for not having driver to slow down in CAN control data in front, or it has been more than detected that parsing information, which is speed,
Speed(-)limit sign limit speed when, warning reminding can be carried out to driver in such a way that acoustic information or steering wheel are shaken etc.；
When parsing information is to detect to be likely to occur more serious risk of collision, the form of CAN message can be used to ESP
(Electronic Stability Program, body electronics systems stabilisation) module issues automatic braking request.
To sum up, CAN can be controlled data flow to scheme by the control method of automobile ADAS system according to an embodiment of the present invention
As the form of data is dissolved into video data stream, while can also be by each image processing algorithm for the sound of CAN control data
Result is answered also to be dissolved into video data stream in the form of image data, to obtain the fused data stream in the form of image data, into
And by fused data stream in SOC chip everywhere in manage device between carry out data transmission.It is each in SOC chip thereby, it is possible to reduce
Data transmission between processor, reduces the communication load between processor, is conducive to the real-time for promoting ADAS system.
Fig. 7 is the block diagram of automobile ADAS system according to an embodiment of the present invention.As shown in fig. 7, automobile ADAS system
100 include: camera 10 and SOC chip 20.
Wherein, camera 10 is used to shoot the environment of motor vehicle environment to obtain video data stream.It is integrated in SOC chip 20
Multiple processors 21, wherein processor 21 controls data flow for acquiring video data stream and CAN, and to collected video
Data flow and CAN control data flow carry out fusion treatment, to generate the fusion number comprising video data stream and CAN control data flow
According to stream, wherein fused data stream includes multiframe blending image, and carries out dissection process to every frame blending image to obtain accordingly
Parsing information and CAN control data, and corresponding image data and/or life are exported according to parsing information and CAN control data
At corresponding control signal.
In an embodiment of the present invention, processor 21 is carried out to collected video data stream and CAN control data flow
When fusion treatment, specifically for carrying out frame decoding processing to video data stream to obtain multi-frame video image, wherein video image
Format is YUV420SP；Processing is split to video image, to obtain the ROI region of every frame video image, and according to the area ROI
The Pixel Information in domain obtains current travelling data, wherein ROI region includes the sky areas in every frame video image, driving
Data include driving and driving at night in the daytime；It is corresponding to generate that histogram equalization processing is carried out to the Y-component of video image
Equalization data；CAN is obtained from CAN control data flow and controls data, wherein CAN control data bag includes current dipped headlight and remote
Light lamp state, speed, steering wheel angle, braking state, driver and/or the safe information of occupant and system debugging,
One or more of test and diagnosis request；The Y value of every frame video image is replaced most with travelling data and CAN control data
Data line afterwards, to obtain blending image.
In an embodiment of the present invention, one is selected to control from multiple processors for acquiring video data stream and CAN
Data flow, wherein if the difference of the maxima and minima in the dominant frequency of multiple processors is greater than preset value, select dominant frequency most
Small processor；If the difference of the maxima and minima in the dominant frequency of multiple processors is less than or equal to preset value, any to select
Select a processor.
It is possible to further select one to control data to collected video data stream and CAN from multiple processors
Stream carry out fusion treatment, wherein if for acquire video data stream and CAN control data flow processor dominant frequency be greater than etc.
In predeterminated frequency, then to collected video data on the processor for acquiring video data stream and CAN control data flow
Stream and CAN control data flow carry out fusion treatment；If being used to acquire the processor of video data stream and CAN control data flow
Dominant frequency is less than predeterminated frequency, then selects one to control data flow to collected video data stream and CAN from other processors
Carry out fusion treatment.
In this embodiment, processor 21 is specifically used for when executing image processing algorithm: a frame blending image is obtained,
And judge blending image Y value last line data in CAN control data in whether include debugging, test and diagnosis ask
It asks；If it includes debugging, test and diagnosis request in data that CAN, which is controlled, judges that the whether external HDMI of ADAS system is shown and set
It is standby；If the external HDMI of ADAS system shows equipment, first the image data of the specific region of fused data is copied directly to
Then corresponding output state selects predeterminable area relevant to image processing algorithm from blending image, and to selection
Predeterminable area is parsed, to obtain parsing information accordingly；If not including debugging, test and diagnosis in CAN control data
Request, and/or, ADAS does not have external HDMI to show equipment, then selection is related to image processing algorithm directly from blending image
Predeterminable area, and the predeterminable area of selection is parsed, to obtain parsing information accordingly；And then will parsing information and
CAN control data are saved to corresponding output state, and spread defeated output state to fused data.
Further, processor 21 is specifically used for when executing decision making algorithm: an input is obtained from fused data stream
Buffer；CAN is read from input buffer and controls data, and judges that the whether external HDMI of ADAS system shows equipment；Such as
The external HDMI of fruit ADAS system shows equipment, then parsing information is read from input buffer, to generate phase according to parsing information
The image data answered；Image data is superimposed in the specific region of blending image, and is shown in external HDMI and shows in equipment；
Corresponding control signal is generated according to parsing information and CAN control data.
It should be noted that the specific embodiment of the automobile ADAS system of the embodiment of the present invention can be found in implementation of the present invention
The specific embodiment of the control method of the automobile ADAS system of example.
CAN can be controlled data flow and melted in the form of image data by automobile ADAS system according to an embodiment of the present invention
Enter into video data stream, while each image processing algorithm can also be controlled into the response results of data for CAN with picture number
According to form be dissolved into video data stream, obtain the fused data stream of image data form, and then by fused data stream in SOC
It manages everywhere on chip and carries out data transmission between device.Thereby, it is possible to reduce the data biography in SOC chip between each processor
It is defeated, the communication load between processor is reduced, the real-time for promoting ADAS system is conducive to.
Fig. 8 is the block diagram of automobile according to an embodiment of the present invention.As shown in figure 8, the automobile 1000 includes aforesaid right
Automobile ADAS system 100 in it is required that.
The automobile of the embodiment of the present invention can reduce CAN control data flow in SOC using above-mentioned automobile ADAS system
Data transmitting on chip between each processor, reduces the communication load between processor, is conducive to the reality for promoting ADAS system
Shi Xing.
In the description of the present invention, it is to be understood that, term " first ", " second " are used for description purposes only, and cannot
It is interpreted as indication or suggestion relative importance or implicitly indicates the quantity of indicated technical characteristic.Define as a result, " the
One ", the feature of " second " can explicitly or implicitly include at least one of the features.In the description of the present invention, " multiple "
It is meant that at least two, such as two, three etc., unless otherwise specifically defined.
In the present invention unless specifically defined or limited otherwise, term " installation ", " connected ", " connection ", " fixation " etc.
Term shall be understood in a broad sense, for example, it may be being fixedly connected, may be a detachable connection, or integral；It can be mechanical connect
It connects, is also possible to be electrically connected；It can be directly connected, can also can be in two elements indirectly connected through an intermediary
The interaction relationship of the connection in portion or two elements, unless otherwise restricted clearly.For those of ordinary skill in the art
For, the specific meanings of the above terms in the present invention can be understood according to specific conditions.
In the present invention unless specifically defined or limited otherwise, fisrt feature in the second feature " on " or " down " can be with
It is that the first and second features directly contact or the first and second features pass through intermediary mediate contact.Moreover, fisrt feature exists
Second feature " on ", " top " and " above " but fisrt feature be directly above or diagonally above the second feature, or be merely representative of
First feature horizontal height is higher than second feature.Fisrt feature can be under the second feature " below ", " below " and " below "
One feature is directly under or diagonally below the second feature, or is merely representative of first feature horizontal height less than second feature.
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show
The description of example " or " some examples " etc. means specific features, structure, material or spy described in conjunction with this embodiment or example
Point is included at least one embodiment or example of the invention.In the present specification, schematic expression of the above terms are not
It must be directed to identical embodiment or example.Moreover, particular features, structures, materials, or characteristics described can be in office
It can be combined in any suitable manner in one or more embodiment or examples.In addition, without conflicting with each other, the skill of this field
Art personnel can tie the feature of different embodiments or examples described in this specification and different embodiments or examples
It closes and combines.
Those of ordinary skill in the art may be aware that list described in conjunction with the examples disclosed in the embodiments of the present disclosure
Member and algorithm steps can be realized with the combination of electronic hardware or computer software and electronic hardware.These functions are actually
It is implemented in hardware or software, the specific application and design constraint depending on technical solution.Professional technician
Each specific application can be used different methods to achieve the described function, but this realization is it is not considered that exceed
The scope of the present invention.
It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description,
The specific work process of device and unit, can refer to corresponding processes in the foregoing method embodiment, and details are not described herein.
In several embodiments provided herein, it should be understood that disclosed systems, devices and methods, it can be with
It realizes by another way.For example, the apparatus embodiments described above are merely exemplary, for example, the unit
It divides, only a kind of logical function partition, there may be another division manner in actual implementation, such as multiple units or components
It can be combined or can be integrated into another system, or some features can be ignored or not executed.Another point, it is shown or
The mutual coupling, direct-coupling or communication connection discussed can be through some interfaces, the indirect coupling of device or unit
It closes or communicates to connect, can be electrical property, mechanical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme
's.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit
It is that each unit physically exists alone, can also be integrated in one unit with two or more units.
It, can be with if the function is realized in the form of SFU software functional unit and when sold or used as an independent product
It is stored in a computer readable storage medium.Based on this understanding, technical solution of the present invention is substantially in other words
The part of the part that contributes to existing technology or the technical solution can be embodied in the form of software products, the meter
Calculation machine software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be a
People's computer, server or network equipment etc.) it performs all or part of the steps of the method described in the various embodiments of the present invention.
And storage medium above-mentioned includes: that USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), arbitrary access are deposited
The various media that can store program code such as reservoir (RAM, Random Access Memory), magnetic or disk.
Although the embodiments of the present invention has been shown and described above, it is to be understood that above-described embodiment is example
Property, it is not considered as limiting the invention, those skilled in the art within the scope of the invention can be to above-mentioned
Embodiment is changed, modifies, replacement and variant.
Claims (13)
1. a kind of control method of automobile ADAS system, which is characterized in that the ADAS system includes camera, SOC chip,
In, the camera is used to shoot the environment of the motor vehicle environment to obtain video data stream, is integrated in the SOC chip more
A processor, and at least one processor executes image processing algorithm, at least one processor executes decision making algorithm, the method
The following steps are included:
Pass through the acquisition video data stream and CAN control data flow in the multiple processor；
Data flow is controlled to the collected video data stream and the CAN by one in the multiple processor to carry out
Fusion treatment, to generate the fused data stream comprising the video data stream and CAN control data flow, wherein described to melt
Closing data flow includes multiframe blending image；
Processor carries out image procossing when executing image processing algorithm, to every frame blending image to obtain parsing information accordingly
Data are controlled with CAN；
Processor exports corresponding picture number when executing decision making algorithm, according to the parsing information and CAN control data
According to and/or generate corresponding control signal.
2. the control method of automobile ADAS system as described in claim 1, which is characterized in that described to the collected view
Frequency data stream and CAN control data flow carry out fusion treatment and include:
Frame decoding processing is carried out to obtain multi-frame video image to the video data stream, wherein the format of the video image is
YUV420SP；
Processing is split to the video image, to obtain the ROI region of every frame video image, and according to the ROI region
Pixel Information obtain current travelling data, wherein the ROI region includes the sky areas in every frame video image, institute
It states travelling data and includes driving and driving at night in the daytime；
Histogram equalization processing is carried out to generate corresponding equalization data to the Y-component of the video image；
From the CAN control data flow in obtain CAN control data, wherein the CAN control data bag include current dipped headlight and
High beam state, speed, steering wheel angle, braking state, driver and/or the safe information of occupant and system tune
One or more of examination, test and diagnosis request；
The last line data of the Y value of every frame video image are replaced, with the travelling data and CAN control data to obtain
The blending image.
3. the control method of automobile ADAS system as described in claim 1, which is characterized in that according to the dominant frequency of each processor
One is selected from the multiple processor, for acquiring video data stream and CAN control data flow, wherein
If the difference of the maxima and minima in the dominant frequency of the multiple processor is greater than preset value, select dominant frequency the smallest
Processor；
If the difference of the maxima and minima in the dominant frequency of the multiple processor is less than or equal to the preset value, any to select
Select a processor.
4. the control method of automobile ADAS system as claimed in claim 1 or 2, which is characterized in that according to described for acquiring
The dominant frequency of the processor of video data stream and CAN control data flow selects one from the multiple processor, to be used for
When carrying out fusion treatment to the collected video data stream and CAN control data flow, wherein
If the dominant frequency of the processor for acquiring the video data stream and CAN control data flow is more than or equal to default frequency
Rate then selects the processor for acquiring the video data stream and CAN control data flow；
If the dominant frequency of the processor for acquiring the video data stream and CAN control data flow is less than the default frequency
Rate then selects one from other processors.
5. the control method of automobile ADAS system as claimed in claim 2, which is characterized in that processor is executing image procossing
When algorithm, following steps are specifically executed:
Obtain a frame blending image, and judge the blending image Y value last line data in CAN control data in be
No includes the debugging, test and diagnosis request；
If in the CAN control data including the debugging, test and diagnosis request, judge whether the ADAS system is outer
It meets HDMI and shows equipment；
If the external HDMI of ADAS system shows equipment, the specific region image data in the blending image is replicated
To corresponding output state, and predeterminable area relevant to described image Processing Algorithm is selected from the blending image, it is right
The predeterminable area of selection is parsed, to obtain parsing information accordingly；
If in the CAN control data do not include the debugging, test and diagnosis request, and/or, the ADAS is not external
HDMI shows equipment, then predeterminable area relevant to described image Processing Algorithm is selected from the blending image, and to selection
Predeterminable area parsed, to obtain parsing information accordingly；
Parsing information and CAN control data are saved to the output state；
The defeated output state is spread to the fused data.
6. the control method of automobile ADAS system as claimed in claim 5, which is characterized in that processor is executing decision making algorithm
When, specifically execute following steps:
An input buffer is obtained from the fused data stream；
CAN is read from the input buffer and controls data, and is judged that the whether external HDMI of the ADAS system is shown and set
It is standby；
If the external HDMI of ADAS system shows equipment, parsing information is read, from the input buffer with basis
The parsing information generates corresponding image data；
By described image data investigation into blending image specific region, and it is shown in the external HDMI and shows in equipment；
Control signal is generated according to the parsing information and CAN control data.
7. a kind of automobile ADAS system characterized by comprising
Camera, the camera are used to shoot the environment of the motor vehicle environment to obtain video data stream；
SOC chip is integrated with multiple processors in the SOC chip, wherein processor is used for:
The video data stream and CAN control data flow are acquired, and the collected video data stream and the CAN are controlled
Data flow carries out fusion treatment, to generate the fused data stream comprising the video data stream and CAN control data flow,
In, the fused data stream includes multiframe blending image, and
When executing image processing algorithm, dissection process is carried out to obtain parsing information and CAN accordingly to every frame blending image
Data are controlled, and when executing decision making algorithm, corresponding image is exported according to the parsing information and CAN control data
Data and/or the corresponding control signal of generation.
8. automobile ADAS system as claimed in claim 7, which is characterized in that the processor is to the collected video
When data flow and CAN control data flow carry out fusion treatment, it is specifically used for:
Frame decoding processing is carried out to obtain a frame video image from the video data stream to the video data stream, wherein institute
The format for stating video image is YUV420SP；
Processing is split to the video image, to obtain the ROI region of every frame video image, and according to the ROI region
Pixel Information obtain current travelling data, wherein the ROI region includes the sky areas in every frame video image, institute
It states travelling data and includes driving and driving at night in the daytime；
Histogram equalization processing is carried out to generate corresponding equalization data to the Y-component of the video image；
From the CAN control data flow in obtain CAN control data, wherein the CAN control data bag include current dipped headlight and
High beam state, speed, steering wheel angle, braking state, driver and/or the safe information of occupant and system tune
One or more of examination, test and diagnosis request；
The last line data of the Y value of every frame video image are replaced, with the travelling data and CAN control data to obtain
The blending image.
9. automobile ADAS system as claimed in claim 7 or 8, which is characterized in that select one from the multiple processor
For acquiring video data stream and CAN control data flow, wherein
If the difference of the maxima and minima in the dominant frequency of the multiple processor is greater than preset value, select dominant frequency the smallest
Processor；
If the difference of the maxima and minima in the dominant frequency of the multiple processor is less than or equal to the preset value, any to select
Select a processor.
10. automobile ADAS system as claimed in claim 7 or 8, which is characterized in that select one from the multiple processor
Fusion treatment is carried out to the collected video data stream and CAN control data flow, wherein
If the dominant frequency of the processor for acquiring the video data stream and CAN control data flow is more than or equal to default frequency
Rate, then to the collected video on the processor for acquiring the video data stream and CAN control data flow
Data flow and CAN control data flow carry out fusion treatment；
If the dominant frequency of the processor for acquiring the video data stream and CAN control data flow is less than the default frequency
Rate then selects one to melt the collected video data stream and CAN control data flow from other processors
Conjunction processing.
11. automobile ADAS system as claimed in claim 8, which is characterized in that processor is when executing image processing algorithm, tool
Body is used for:
Obtain a frame blending image, and judge the blending image Y value last line data in CAN control data in be
No includes the debugging, test and diagnosis request；
If in the CAN control data including the debugging, test and diagnosis request, judge whether the ADAS system is outer
It meets HDMI and shows equipment；
If the external HDMI of ADAS system shows equipment, the specific region image data in the blending image is replicated
To corresponding output state, and predeterminable area relevant to described image Processing Algorithm is selected from the blending image, it is right
The predeterminable area of selection is parsed, to obtain parsing information accordingly；
If in the CAN control data do not include the debugging, test and diagnosis request, and/or, the ADAS is not external
HDMI shows equipment, then predeterminable area relevant to described image Processing Algorithm is selected from the blending image, and to selection
Predeterminable area parsed, to obtain parsing information accordingly；
Parsing information and CAN control data are saved to the output state；
The defeated output state is spread to the fused data.
12. automobile ADAS system as claimed in claim 11, which is characterized in that processor is when executing decision making algorithm, specifically
For:
An input buffer is obtained from the fused data stream；
CAN is read from the input buffer and controls data, and is judged that the whether external HDMI of the ADAS system is shown and set
It is standby；
If the external HDMI of ADAS system shows equipment, parsing information is read, from the input buffer with basis
The parsing information generates corresponding image data；
By in the specific region of described image data investigation to blending image, and it is shown in the external HDMI and shows in equipment；
Control signal is generated according to the parsing information and CAN control data.
13. a kind of automobile, which is characterized in that including the automobile ADAS system as described in any one of claim 7-12.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710641838.2A CN109318799B (en) | 2017-07-31 | 2017-07-31 | Automobile, automobile ADAS system and control method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710641838.2A CN109318799B (en) | 2017-07-31 | 2017-07-31 | Automobile, automobile ADAS system and control method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109318799A true CN109318799A (en) | 2019-02-12 |
| CN109318799B CN109318799B (en) | 2020-11-20 |
Family
ID=65245797
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710641838.2A Active CN109318799B (en) | 2017-07-31 | 2017-07-31 | Automobile, automobile ADAS system and control method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109318799B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110765928A (en) * | 2019-10-21 | 2020-02-07 | 上海汽车集团股份有限公司 | Visual recognition system, recognition method, industrial personal computer and storage medium |
| CN112040173A (en) * | 2020-07-20 | 2020-12-04 | 南斗六星系统集成有限公司 | Method for rapidly displaying reversing video through Android car machine system |
| CN112449083A (en) * | 2019-08-27 | 2021-03-05 | 深圳市麦道微电子技术有限公司 | Night vision camera for automobile |
| CN114684178A (en) * | 2020-12-31 | 2022-07-01 | 观致汽车有限公司 | Vehicle control system, vehicle control method and vehicle |
| CN114782916A (en) * | 2022-04-11 | 2022-07-22 | 广州优创电子有限公司 | ADAS rear vehicle identification system carried by rearview mirror and based on multi-sensor fusion |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101739843A (en) * | 2009-12-04 | 2010-06-16 | 河海大学常州校区 | Device and method for real-time three-dimensionally sensing safe driving of vehicle having mixed multi-visual information |
| US20100328055A1 (en) * | 2009-06-30 | 2010-12-30 | The Hong Kong Polytechnic University | Detection System for Assisting A Driver When Driving A Vehicle |
| CN102063095A (en) * | 2009-11-18 | 2011-05-18 | 吉林大元电子科技有限公司 | CAN bus electronic control device for automobile |
| CN202549060U (en) * | 2011-11-17 | 2012-11-21 | 无锡物联网产业研究院 | Multi-sensor system |
| CN205498764U (en) * | 2016-03-07 | 2016-08-24 | 郑州精益达汽车零部件有限公司 | Integrated rear portion initiative safety precaution's ADAS system based on vision |
-
2017
- 2017-07-31 CN CN201710641838.2A patent/CN109318799B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100328055A1 (en) * | 2009-06-30 | 2010-12-30 | The Hong Kong Polytechnic University | Detection System for Assisting A Driver When Driving A Vehicle |
| CN102063095A (en) * | 2009-11-18 | 2011-05-18 | 吉林大元电子科技有限公司 | CAN bus electronic control device for automobile |
| CN101739843A (en) * | 2009-12-04 | 2010-06-16 | 河海大学常州校区 | Device and method for real-time three-dimensionally sensing safe driving of vehicle having mixed multi-visual information |
| CN202549060U (en) * | 2011-11-17 | 2012-11-21 | 无锡物联网产业研究院 | Multi-sensor system |
| CN205498764U (en) * | 2016-03-07 | 2016-08-24 | 郑州精益达汽车零部件有限公司 | Integrated rear portion initiative safety precaution's ADAS system based on vision |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112449083A (en) * | 2019-08-27 | 2021-03-05 | 深圳市麦道微电子技术有限公司 | Night vision camera for automobile |
| CN110765928A (en) * | 2019-10-21 | 2020-02-07 | 上海汽车集团股份有限公司 | Visual recognition system, recognition method, industrial personal computer and storage medium |
| CN112040173A (en) * | 2020-07-20 | 2020-12-04 | 南斗六星系统集成有限公司 | Method for rapidly displaying reversing video through Android car machine system |
| CN114684178A (en) * | 2020-12-31 | 2022-07-01 | 观致汽车有限公司 | Vehicle control system, vehicle control method and vehicle |
| CN114782916A (en) * | 2022-04-11 | 2022-07-22 | 广州优创电子有限公司 | ADAS rear vehicle identification system carried by rearview mirror and based on multi-sensor fusion |
| CN114782916B (en) * | 2022-04-11 | 2024-03-29 | 广州优创电子有限公司 | ADAS rear-car recognition system based on multi-sensor fusion and carried on rearview mirror |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109318799B (en) | 2020-11-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109318799A (en) | Automobile, automobile ADAS system and its control method | |
| KR102618662B1 (en) | 3D feature prediction for autonomous driving | |
| KR102605807B1 (en) | Generating ground truth for machine learning from time series elements | |
| KR102126824B1 (en) | Parking support information display method and parking assist device | |
| CN108431881B (en) | Parking assistance information display method and parking assistance device | |
| EP2990291A2 (en) | Control system, relay device and control method | |
| DE102014009065B4 (en) | Device and method for controlling vehicle safety with the aid of cameras | |
| CN107589745A (en) | Drive manner, vehicle carried driving end, remotely drive end, equipment and storage medium | |
| US10259383B1 (en) | Rear collision alert system | |
| CN106537904A (en) | Vehicle with surroundings-monitoring device and method for operating such a monitoring device | |
| CN103448653A (en) | Vehicle collision warning system and method | |
| CN106314424B (en) | Householder method of overtaking other vehicles, device and automobile based on automobile | |
| DE102009016580A1 (en) | Data processing system and method for providing at least one driver assistance function | |
| CN105865467A (en) | Map-based navigation method and device | |
| US11202030B2 (en) | System and method for providing complete event data from cross-referenced data memories | |
| US11250279B2 (en) | Generative adversarial network models for small roadway object detection | |
| CN103786644A (en) | Apparatus and method for tracking the position of a peripheral vehicle | |
| CN111010530A (en) | Emergency vehicle detection | |
| CN107826092A (en) | Advanced drive assist system and method, equipment, program and medium | |
| CN110263623A (en) | Train climbs monitoring method, device, terminal and storage medium | |
| JP4826355B2 (en) | Vehicle surrounding display device | |
| KR102174863B1 (en) | Autonomous Vehicle Exterior Display Interaction Apparatus and Method | |
| CN115489536A (en) | Driving assistance method, system, equipment and readable storage medium | |
| CN111277956A (en) | Method and device for collecting vehicle blind area information | |
| CN115131749A (en) | Image processing apparatus, image processing method, and computer-readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |