CN111856606B - Forward-looking intelligent driving auxiliary device and method based on infrared thermal imaging - Google Patents
Forward-looking intelligent driving auxiliary device and method based on infrared thermal imaging Download PDFInfo
- Publication number
- CN111856606B CN111856606B CN201910708437.3A CN201910708437A CN111856606B CN 111856606 B CN111856606 B CN 111856606B CN 201910708437 A CN201910708437 A CN 201910708437A CN 111856606 B CN111856606 B CN 111856606B
- Authority
- CN
- China
- Prior art keywords
- image
- infrared
- thermal imaging
- gray scale
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001931 thermography Methods 0.000 title claims abstract description 48
- 238000000034 method Methods 0.000 title claims abstract description 22
- 238000012545 processing Methods 0.000 claims abstract description 21
- 238000003384 imaging method Methods 0.000 claims description 20
- 230000005540 biological transmission Effects 0.000 claims description 6
- 241001465754 Metazoa Species 0.000 claims description 5
- 238000009432 framing Methods 0.000 claims description 4
- 238000001514 detection method Methods 0.000 abstract description 16
- 230000000007 visual effect Effects 0.000 abstract description 5
- 230000008569 process Effects 0.000 description 6
- 230000000694 effects Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 238000001914 filtration Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000003331 infrared imaging Methods 0.000 description 3
- 230000005855 radiation Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 230000004313 glare Effects 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 239000003086 colorant Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 229910052732 germanium Inorganic materials 0.000 description 1
- GNPVGFCGXDBREM-UHFFFAOYSA-N germanium atom Chemical compound [Ge] GNPVGFCGXDBREM-UHFFFAOYSA-N 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000002604 ultrasonography Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01V—GEOPHYSICS; GRAVITATIONAL MEASUREMENTS; DETECTING MASSES OR OBJECTS; TAGS
- G01V8/00—Prospecting or detecting by optical means
- G01V8/10—Detecting, e.g. by using light barriers
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J5/48—Thermography; Techniques using wholly visual means
- G01J5/485—Temperature profile
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01J—MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
- G01J5/00—Radiation pyrometry, e.g. infrared or optical thermometry
- G01J2005/0077—Imaging
Landscapes
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Geophysics (AREA)
- General Life Sciences & Earth Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Theoretical Computer Science (AREA)
- Mechanical Engineering (AREA)
- Image Processing (AREA)
- Traffic Control Systems (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
The invention discloses a forward-looking intelligent driving auxiliary device and a method based on infrared thermal imaging, wherein the method comprises the steps of obtaining the current ambient temperature and the infrared temperature gray scale value of a target; obtaining a brightness adjustment weight of image output according to the current ambient temperature and the infrared temperature gray scale value of the target; performing brightness adjustment on all gray image data according to the brightness adjustment weight; processing the brightness adjustment weight and the infrared temperature gray scale value of the target into thermal imaging data, and converting the thermal imaging data into a gray scale image source image; establishing an image sample library; identifying a target in the gray scale image source image according to the image sample library; and outputting the identification result through the whole vehicle control bus. The invention not only can solve the problem that the driver has limited visual field in the scene of weak light or no light, but also can realize the identification and detection of the obstacle, and can solve the problem of dazzle discomfort caused by the fact that the opponent runs the vehicle to turn on the high beam.
Description
Technical Field
The invention relates to an intelligent driving auxiliary device, in particular to a forward-looking intelligent driving auxiliary device and method based on infrared thermal imaging.
Background
The infrared thermal imaging is a technology based on passive infrared light receiving and post-processing imaging, is applied to a forward-looking intelligent driving assistance system (ADAS), and makes up for driving scenes with weak visible light, and truly assists all-weather imaging.
The advanced driving assistance system (Advanced Driver Assistance System), abbreviated as ADAS, is an active safety technique for collecting environmental data inside and outside a vehicle at a first time by using various sensors mounted on the vehicle, and performing technical processes such as identification, detection and tracking of static and dynamic objects, so that a driver can perceive a possible danger at the fastest time to draw attention and improve safety. The sensors used by ADAS mainly include cameras, radar, laser, ultrasound, etc., and can detect light, heat, pressure, or other variables used to monitor the state of an automobile, typically located on front and rear bumpers, side mirrors, steering column interiors, or windshields of the vehicle. Early ADAS technologies were primarily passive alarms, which alert drivers to notice abnormal vehicle or road conditions when a vehicle detects a potential hazard. Active intervention is also common to the latest ADAS technologies.
The existing forward-looking intelligent driving auxiliary system mainly simulates the characteristics of human eyes to complete a series of algorithms, namely, a camera based on visible light is used for detecting the condition of a road ahead in the driving process, and the functions of target detection, pedestrian detection, lane line detection and the like are realized through a specific algorithm, so that the driving condition of an automobile is combined, and an alarm is given to a driver when an abnormal condition occurs.
However, the front-view ADAS system acquires images based on a camera of visible light, and then realizes a detection process through an identification algorithm, so that the quality of the images has a great influence on a judgment result, and the images completely lose effect under the condition of no visible light.
The current common use is that the system architecture form of a field programmable gate array FPGA and a digital signal processor DSP realizes the image recognition function of an infrared imaging system, wherein the FPGA completes the work of infrared imaging or image preprocessing and the like, and the DSP is responsible for the functions of image recognition and the like. The circuit structure of the mode is huge, the hardware structure is complex, and the cost is high.
In addition, the design of the existing infrared thermal imaging equipment is basically integrated, the detector, the image processing and the image output interface are concentrated on one equipment, the equipment can directly output images, and when the image identification processing is needed, the images are converted into digital video data through a special circuit and then are sent to a rear processor or other computer systems for image identification. This is a complex and costly circuit.
Disclosure of Invention
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the disclosure as claimed.
Aiming at the problems, the invention not only can solve the problem that the visual field of a driver is limited under the scene of weak light or no light, but also adds an identification algorithm of target detection to realize the identification detection of obstacles and prompt the driver of the vehicle, and can solve the problem of glare discomfort caused by the fact that the other side drives a vehicle to turn on a high beam, ensure the driving safety of the driver, and solve the problems of complex structure, high cost and the like of an infrared image imaging and identification circuit in the prior art by adopting a simple hardware structure.
The invention discloses a forward-looking intelligent driving assisting method based on infrared thermal imaging, which is characterized in that,
step one, acquiring the current ambient temperature and the infrared temperature gray scale value of a target;
step two, obtaining a brightness adjustment weight of image output according to the current ambient temperature and the infrared temperature gray scale value of the target;
step three, brightness adjustment is carried out on all gray image data according to the brightness adjustment weight;
step four, processing the brightness adjustment weight and the infrared temperature gray scale value of the target into thermal imaging data, and converting the thermal imaging data into a gray scale image source image;
step five, establishing an image sample library;
step six, identifying targets in the gray scale image source images according to the image sample library;
and step seven, outputting the identification result through a whole vehicle control bus.
Preferably, the invention further discloses a forward-looking intelligent driving assisting method based on infrared thermal imaging, which is characterized in that,
in the second step, further includes:
and counting the temperature distribution according to the infrared temperature gray scale value of the current ambient temperature, and finding out the intermediate value of the infrared temperature gray scale value in the maximum distribution area, wherein the weight is obtained by comparing the current ambient temperature with the ambient temperature value corresponding to the intermediate value.
Preferably, the invention further discloses a forward-looking intelligent driving assisting method based on infrared thermal imaging, which is characterized in that,
in the sixth step, further includes:
and framing the target in the current image according to the identification result, and performing image superposition processing.
Preferably, the invention further discloses a forward-looking intelligent driving assisting method based on infrared thermal imaging, which is characterized in that,
the image sample library is formed by marking the video or the image with the target to be identified according to the data of the frames.
Preferably, the invention further discloses a forward-looking intelligent driving assisting method based on infrared thermal imaging, which is characterized in that,
the image sample library comprises a vehicle, a person, an animal and an obstacle picture target.
The invention also discloses a forward-looking intelligent driving auxiliary device based on infrared thermal imaging, which is characterized by comprising the following components:
an infrared detector assembly that receives an infrared light signal from a forward looking object;
the special interface of the FPGA is connected with the output end of the infrared detector assembly and is used for processing the infrared light signals into data suitable for transmission;
the LVDS serializer is connected with the output end of the special interface of the FPGA and is used for adding the infrared light signals into LVDS serial data;
an image processor connected to the output end of the LVDS serializer, for imaging and recognizing the received LVDS serial data and outputting the same, and configured to:
acquiring the current ambient temperature and the infrared temperature gray scale value of the target;
obtaining a brightness adjustment weight of image output according to the current ambient temperature and the infrared temperature gray scale value of the target;
brightness adjustment is carried out on all gray image data according to the brightness adjustment weight value, and the gray image data are output;
processing the brightness adjustment weight and the infrared temperature gray scale value of the target into thermal imaging data, and converting the thermal imaging data into a gray scale image source image;
establishing an image sample library;
identifying a target in the gray scale image source image according to the image sample library;
and outputting the identification result through the whole vehicle control bus.
Preferably, the invention further discloses a forward-looking intelligent driving auxiliary device based on infrared thermal imaging, which is characterized in that,
and the image processor outputs an image superposition processing result to a whole vehicle display system, and outputs the identification result to the whole vehicle control bus.
Preferably, the invention further discloses a forward-looking intelligent driving auxiliary device based on infrared thermal imaging, which is characterized in that,
the infrared detector assembly includes a 14-bit detector.
Preferably, the invention further discloses a forward-looking intelligent driving auxiliary device based on infrared thermal imaging, which is characterized in that,
the control bus comprises a CAN bus.
Preferably, the invention further discloses a forward-looking intelligent driving auxiliary device based on infrared thermal imaging, which is characterized in that,
the image sample library is obtained by marking objects to be identified on provided video or images according to data of frames, and the image sample library contains vehicle, human, animal and obstacle picture objects.
The device and the method adopting the structure solve the problems of complex structure, high cost and the like of the infrared image imaging and identifying circuit in the prior art by using a simple hardware structure, and obtain better effect on infrared image imaging and identifying.
Drawings
Embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. Reference will now be made in detail to the preferred embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts. Furthermore, although terms used in the present disclosure are selected from publicly known and commonly used terms, some terms mentioned in the present disclosure may be selected by the applicant at his or her discretion, the detailed meanings of which are described in relevant parts of the description herein. Furthermore, it is required that the present disclosure is understood, not simply by the actual terms used but by the meaning of each term lying within.
The above and other objects, features and advantages of the present invention will become apparent to those skilled in the art from the following detailed description of the present invention with reference to the accompanying drawings.
FIG. 1 is a schematic diagram of the modular composition of the infrared thermal imaging based forward-looking intelligent driving assistance device of the present invention;
FIG. 2 is a flow chart of a thermal imaging image process employed in the present invention;
fig. 3 is a flow chart of an object detection algorithm employed in the present invention.
Reference numerals
11-infrared detector assembly
12-interface special for FPGA
13-LVDS serializer
14-image processor
Detailed Description
This specification discloses one or more embodiments that incorporate the features of the invention. The disclosed embodiments merely exemplify the invention. The scope of the invention is not limited to the disclosed embodiments. The invention is defined by the appended claims.
Reference in the specification to "one embodiment," "an example embodiment," etc., means that a particular feature, structure, or characteristic may be included in the described embodiments, but that all embodiments may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Furthermore, when a particular feature, mechanism, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to effect such feature, mechanism, or characteristic in connection with other embodiments whether or not explicitly described.
Moreover, it is to be understood that the spatial descriptions (e.g., above, below, above, left, right, below, top, bottom, vertical, horizontal, etc.) used herein are for illustrative purposes only, and that the actual implementation of the structures described herein may be spatially arranged in any orientation or manner.
All objects in nature radiate infrared rays when their temperature is higher than absolute zero (i.e. -273 ℃). The radiation energy and the distribution according to the wavelength are determined by the surface temperature of the object. The infrared detector receives the infrared rays, and then the image processor processes the details of the target to realize imaging, which is called infrared thermal imaging.
The image processing algorithm in the system is to display the obtained digital signals and the brightness of the images into the spatial distribution images of the targets on the plane through filtering and amplifying according to the content (temperature data) of the electric signals output to the image processor by the detector, and finally present the road condition scene in front to the vehicle owners in the form of gray level diagram on the vehicle-mounted display screen.
The infrared is mainly used for aiming at the weak condition of visible light, a special target detection algorithm is integrated in the system for better driving experience and safety guarantee, and targets in the front infrared sensing range can be detected and early-warned while imaging is carried out, so that driving safety is guaranteed. The algorithm model is trained based on the image data of infrared thermal imaging, so that the recognition rate is improved to a great extent.
The system consists of a thermal imaging system based on infrared radiation and image recognition for thermal imaging.
Infrared thermal imaging section: the infrared optical lens structure can project infrared rays aiming at the heat radiation wave band to the infrared detector, and the germanium sheet in front of the lens can filter visible light to only allow infrared light to pass through, so that the infrared imaging effect is guaranteed. The detector receives the infrared ray and converts the infrared ray into an electric signal of a temperature gray scale value, which is called infrared original data, and the rear-end image processor receives the original data and then sends the processed data to the image generation algorithm for imaging through the filtering and enhancing algorithm, so that a gray scale image based on brightness information is obtained.
Image generation rationale: the 14-bit detector data is filtered by a filtering algorithm to remove blind pixels and invalid data, normal phase metadata is guaranteed to be given to a subsequent imaging algorithm, and the imaging algorithm converts 14-bit original data into 8-bit YUV data capable of being imaged.
Visual recognition portion: based on the R_CNN neural network model, training an algorithm model by adopting a special image sample library of infrared thermal imaging to obtain a visual recognition algorithm model of infrared thermal imaging, transmitting video data of thermal imaging into the algorithm model, namely outputting a recognition result of a trained target object, and then framing the target result in a display system for display through image superposition processing.
In order to enable the whole vehicle to have more prompt and driving experience on the identification result, the identification result is also sent to the CAN bus in the system, and the identification result is shared with other systems of the whole vehicle.
As shown in fig. 1, the infrared thermal imaging-based forward-looking intelligent driving assistance device of the present invention includes an infrared detector assembly 11, an FPGA dedicated interface 12, an LVDS serializer 13, an image processor 14, and outputs thereof.
The image processor 14 performs image imaging, image recognition and image output on the data, and outputs the recognition result and the imaging to a display system and a control bus of the whole vehicle, respectively.
The invention completes the data acquisition by using a single device, namely, infrared data is independently acquired by using an infrared detector assembly 11, the infrared data is processed into data suitable for transmission through an FPGA special interface 12, then the data is converted into LVDS serial data suitable for transmission by an LVDS serializer 13, the data is received by a controller of an image processor 14 and then deserialized, and then the data is input into an image processor with an ISP for imaging and image recognition processing.
The special interface of the invention is realized by adopting FPGA, and the realization of clock generation, data analysis of detector communication and some control time sequences are as follows: reset, configuration, synchronization, OCC, etc.
Referring to fig. 2, the thermal imaging process of the present invention is further described in detail as follows:
for example, the detector is 14 bits, and the output temperature ranges from 0 to (2) 14 -1) the image processor 14 counts in which area the data range output by the infrared detector assembly 11 is most, and finds out the distribution relation;
step 23, the image processor 14 finds the intermediate value of the infrared temperature gray scale value in the maximum distribution area;
in this step, since the infrared detector assembly 11 senses a temperature signal, the temperature sensing of the detector 11 is usually within a certain range, such as 100 ℃, which is not fixed in the actual environment, such as optional (0 ℃ to 100 ℃) according to the environmental temperature distribution or optional (-40 ℃ to 60 ℃), and when the target scene is imaged, the image processor 14 performs identification processing on the respective space occupation ratio of the temperature to achieve good imaging effect, the difference of the temperature is reflected on the brightness value of the image, and the processing is weakened when the temperature range is exceeded for the non-main target.
it should be noted that, the image has several common expression modes, namely RGB, that is, red, green and blue three primary colors, and another is called YUV, that is, brightness and color difference, wherein Y is brightness and UV, and the color difference is expressed by the Y. Whereas for gray scale images, so-called black and white images, only luminance data, Y values, and no UV values are present.
The infrared thermal imaging is to convert the temperature gray scale value output by the infrared detector into a brightness value, namely a Y value according to a corresponding relation, so that imaging can be realized;
28, performing image superposition processing, and marking an identified target on the video;
and step 29, outputting video through the vehicle-mounted video output terminal, namely imaging according to the adjusted gray scale.
In summary, the image processing algorithm in the system displays the obtained digital signal and the brightness of the image into the spatial distribution image of each target on the plane through filtering and amplifying according to the content (temperature data) of the electric signal output to the image processor by the detector, and finally presents the road condition scene in front to the vehicle owner in the form of gray level diagram on the vehicle-mounted display screen.
The infrared detector assembly 11 receives the infrared rays and converts the infrared rays into electric signals with temperature gray scale values, which are called infrared original data, the image processor 14 receives the original data, filters and enhances the processed data through an algorithm, and an image generating algorithm images the processed data to form a gray scale image source image based on brightness information.
The infrared is mainly used for aiming at the weak condition of visible light, and in order to achieve better driving experience and safety guarantee, the system is also integrated with a target detection algorithm, and targets in the front infrared sensing range can be identified, detected and early-warned while imaging is carried out, so that driving safety is guaranteed, and the targets in the infrared image are identified through the target detection algorithm.
Meanwhile, the specific steps of the target detection flow are given in fig. 3:
and step 35, outputting the identification result through the whole vehicle control bus.
In the flow shown in fig. 3, in order to make the whole vehicle have more prompt and driving experience on the recognition result, the recognition result is also sent to the whole vehicle control bus including the CAN bus in the system, so as to be shared by other systems of the whole vehicle.
In summary, the invention adopts a simple hardware structure to solve the problems of complex circuit structure, high cost and the like of infrared image imaging and recognition in the prior art, not only can solve the problem of limited visual field of a driver in a scene with weak light or no light, but also adds a recognition algorithm of target detection to realize recognition detection of obstacles, prompt a vehicle owner in time, and simultaneously can solve the problem of glare discomfort caused by driving a high beam on the other side of the vehicle, thereby guaranteeing the driving safety of the driver.
In order to meet the longer-distance transmission requirement of the vehicle gauge on the reliability of transmission, the camera part and the controller part are connected by adopting a FARKA connector by using the POC technology, and the reliability is high.
The previous description of the preferred embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles described herein may be applied to other embodiments without the use of the inventive faculty. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (9)
1. A forward-looking intelligent driving assisting method based on infrared thermal imaging is characterized in that,
step one, acquiring the current ambient temperature and the infrared temperature gray scale value of a target;
step two, obtaining a brightness adjustment weight of image output according to the current ambient temperature and the infrared temperature gray scale value of the target;
step three, brightness adjustment is carried out on all gray image data according to the brightness adjustment weight;
step four, processing the brightness adjustment weight and the infrared temperature gray scale value of the target into thermal imaging data, and converting the thermal imaging data into a gray scale image source image;
step five, establishing an image sample library;
step six, identifying targets in the gray scale image source images according to the image sample library;
step seven, outputting the identification result through a whole vehicle control bus;
wherein, in the second step, further comprises:
and counting temperature distribution according to the infrared temperature gray scale value of the current ambient temperature, finding out the intermediate value of the infrared temperature gray scale value in the maximum distribution area, and comparing the brightness adjustment weight value with the ambient temperature value corresponding to the intermediate value according to the current ambient temperature.
2. The infrared thermal imaging-based forward-looking intelligent driving assistance method according to claim 1, characterized in that,
in the sixth step, further includes:
and framing the target in the current image according to the identification result, and performing image superposition processing.
3. The infrared thermal imaging-based forward-looking intelligent driving assistance method according to claim 2, characterized in that,
the image sample library is formed by marking the video or the image with the target to be identified according to the data of the frames.
4. A forward-looking intelligent driving assistance method based on infrared thermal imaging as claimed in claim 3, wherein,
the image sample library comprises a vehicle, a person, an animal and an obstacle picture target.
5. An infrared thermal imaging-based forward-looking intelligent driving assistance device, which is characterized by comprising:
an infrared detector assembly that receives an infrared light signal from a forward looking object;
the special interface of the FPGA is connected with the output end of the infrared detector assembly and is used for processing the infrared light signals into data suitable for transmission;
the LVDS serializer is connected with the output end of the special interface of the FPGA and is used for adding the infrared light signals into LVDS serial data;
and an image processor connected to the output end of the LVDS serializer, imaging and recognizing the LVDS serial data and outputting the same, wherein the image processor is configured to:
acquiring the current ambient temperature and the infrared temperature gray scale value of the target;
obtaining a brightness adjustment weight of image output according to the current ambient temperature and the infrared temperature gray scale value of the target;
brightness adjustment is carried out on all gray image data according to the brightness adjustment weight value, and the gray image data are output;
processing the brightness adjustment weight and the infrared temperature gray scale value of the target into thermal imaging data, and converting the thermal imaging data into a gray scale image source image;
establishing an image sample library;
identifying a target in the gray scale image source image according to the image sample library;
and outputting the identification result through the whole vehicle control bus.
6. The infrared thermal imaging-based forward-looking intelligent driving assistance device according to claim 5, characterized in that,
and the image processor outputs an image superposition processing result to a whole vehicle display system, and outputs the identification result to the whole vehicle control bus.
7. The infrared thermal imaging-based forward-looking intelligent driving assistance device according to claim 5, characterized in that,
the infrared detector assembly includes a 14-bit detector.
8. The infrared thermal imaging-based forward-looking intelligent driving assistance device according to claim 6, characterized in that,
the control bus comprises a CAN bus.
9. The infrared thermal imaging-based forward-looking intelligent driving assistance device according to claim 5, characterized in that,
the image sample library is obtained by marking objects to be identified on provided video or images according to data of frames, and the image sample library contains vehicle, human, animal and obstacle picture objects.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910708437.3A CN111856606B (en) | 2019-08-01 | 2019-08-01 | Forward-looking intelligent driving auxiliary device and method based on infrared thermal imaging |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910708437.3A CN111856606B (en) | 2019-08-01 | 2019-08-01 | Forward-looking intelligent driving auxiliary device and method based on infrared thermal imaging |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111856606A CN111856606A (en) | 2020-10-30 |
| CN111856606B true CN111856606B (en) | 2023-06-06 |
Family
ID=72970586
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910708437.3A Active CN111856606B (en) | 2019-08-01 | 2019-08-01 | Forward-looking intelligent driving auxiliary device and method based on infrared thermal imaging |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111856606B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115376092B (en) * | 2022-10-21 | 2023-02-28 | 广州万协通信息技术有限公司 | Image recognition method and device in auxiliary driving |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102323275A (en) * | 2011-08-10 | 2012-01-18 | 宁波大学 | Imaging detection device of infrared chalcogenide glass internal macroscopic defect |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101791931B (en) * | 2009-02-03 | 2012-12-19 | 上海保隆汽车科技股份有限公司 | Tire pressure monitoring system |
| CN101852869A (en) * | 2010-05-18 | 2010-10-06 | 中国民航大学 | Airfield runway foreign object detection system |
| US10324226B2 (en) * | 2017-02-23 | 2019-06-18 | Witricity Corporation | Foreign object detection using infared sensing |
| CN109308817A (en) * | 2018-09-03 | 2019-02-05 | 山东大学 | Position is shared in construction tunnel and traffic safety hidden danger early warning system and method |
-
2019
- 2019-08-01 CN CN201910708437.3A patent/CN111856606B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102323275A (en) * | 2011-08-10 | 2012-01-18 | 宁波大学 | Imaging detection device of infrared chalcogenide glass internal macroscopic defect |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111856606A (en) | 2020-10-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7105754B2 (en) | IMAGING DEVICE AND METHOD OF CONTROLLING IMAGING DEVICE | |
| US8767074B2 (en) | System and method of assisting visibility of driver | |
| US20070115138A1 (en) | Onboard imaging apparatus | |
| JP7566761B2 (en) | Imaging device | |
| JP6803989B2 (en) | Solid-state image sensor and its driving method | |
| JP2015232442A (en) | Image processor and vehicle front monitoring device | |
| US11943549B2 (en) | Imaging apparatus and electronic equipment | |
| JP2010257282A (en) | Obstacle detection device and vehicle with the device mounted thereon | |
| CN113525234A (en) | Auxiliary driving system device | |
| KR102493027B1 (en) | Imaging device, driving method thereof, and electronic device | |
| JP5951785B2 (en) | Image processing apparatus and vehicle forward monitoring apparatus | |
| CN111856606B (en) | Forward-looking intelligent driving auxiliary device and method based on infrared thermal imaging | |
| WO2017158829A1 (en) | Display control device and display control method | |
| KR102614494B1 (en) | Non-identical camera based image processing device | |
| JP5904825B2 (en) | Image processing device | |
| KR101030210B1 (en) | Obstacle recognition system for vehicle and method thereof | |
| CN109076168B (en) | Control device, control method, and computer-readable medium | |
| KR102339358B1 (en) | Vehicular device for improving visibility in adverse weather conditions | |
| WO2022219874A1 (en) | Signal processing device and method, and program | |
| WO2022249562A1 (en) | Signal processing device, method, and program | |
| CN212332517U (en) | Camera of backing a car with warning function of backing a car | |
| CN118636792A (en) | Downhole scene 360-degree looking-around system and method | |
| JP2007015460A (en) | Obstacle position indicating system and method of indicating obstacle position | |
| CN116442908A (en) | Control method and system of electronic rearview mirror of vehicle and vehicle | |
| KR20160059696A (en) | Image system for vehicle and image processing method using the same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |