KR101803908B1 - Integrated monitoring system and method for error diagnosis of vehicle using isobus and images - Google Patents

Abstract

Provided are an integrated monitoring system for diagnosing a defect of a vehicle, and an integrated monitoring method therefor using an ISOBUS and an image. According to one embodiment of the present invention, the integrated monitoring system for diagnosing a defect of a vehicle using an ISOBUS and an image comprises: an image obtaining unit obtaining an image from a plurality of cameras installed in a vehicle; a data obtaining unit obtaining warning data warning a defective state of the vehicle from a plurality of ECUs mounted in the vehicle; a display unit displaying the image and the warning data; and a control unit controlling the related image of a vehicle portion related to the warning data to be displayed on the display unit with the warning data together.

Description

Technical Field [0001] The present invention relates to an integrated monitoring system and method for diagnosing faults of a vehicle using ISOBUS and an image,

The present invention relates to an integrated monitoring system and method for diagnosing a fault diagnosis of a vehicle using an ISOBUS and an image. More particularly, the present invention relates to an ISOBUS And an integrated monitoring system and method for fault diagnosis of a vehicle using an image.

I could not see which part was broken because each part was deep inside the vehicle and I could see which part was defective after I checked the broken car at the workshop.

Recently, the failure state of each part is controlled and displayed in text form or in a user interface environment (GUI) to check the failure state.

For example, a vehicle is equipped with a fuel injection control device for controlling fuel injection of an engine by analyzing information detected from various sensors installed in the vicinity of the engine, a door sensor installed on the vehicle body, and the like to control the actuator of the chassis And electronic control units (ECUs) such as a body control unit.

The electronic control unit is provided with a function of analyzing data sensed by various sensors and diagnosing the failure of the corresponding module. Specifically, when a failure is detected, a function (for example, blinking of the lamp) for driving a display lamp for warning of the failure content to a predetermined display pattern is executed. At this time, the display pattern becomes a Diagnostic Trouble Code (DTC) set differently depending on the failure state.

However, since the driver is informed of the failure of the vehicle in the form of an error code using a warning lamp on the instrument panel or an external diagnostic device, the driver can check the failure immediately, but it is difficult to know the exact region and state of the failure.

Korean Registered Patent No. 1232639 (Announced on March 23, 2013) Korean Registered Patent No. 1398724 (Announced 2014.05.27)

The present invention provides an integrated monitoring system and method for diagnosing a fault diagnosis of a vehicle using an ISOBUS and an image, which can effectively confirm a failure state by providing an image of a failure site together with the failure warning data of the vehicle .
The problems to be solved by the present invention are not limited to the above-mentioned problems, and other matters not mentioned can be clearly understood by those skilled in the art from the following description.

delete

According to another aspect of the present invention, there is provided an integrated monitoring system for diagnosing a fault diagnosis of a vehicle using ISOBUS and an image, comprising: an image acquiring unit acquiring an image from a plurality of cameras installed in a vehicle; A data acquiring unit for acquiring warning data for warning of a failure state of the vehicle from a plurality of ECUs mounted on the vehicle; A display unit for displaying the image and alert data; And a control unit for controlling the display unit to display an associated image of the vehicle site related to the warning data together with the warning data.
The image acquiring unit may acquire an infrared image from at least one infrared camera.
The data acquiring unit may acquire data from the plurality of ECUs via a CAN line.
The data acquiring unit may acquire data according to the ISO 11783 standard.
In addition, when the warning data corresponds to a preset emergency situation, the control unit may control at least one of the related image or the warning data to flicker on the display unit.
The apparatus may further include a communication unit for wirelessly transmitting the related image and warning data to an external diagnostic apparatus.
The apparatus may further include an image analyzer for analyzing the image to detect an event.
The image analyzer may set an area of interest in the image and derive feature information of the detected event in the area of interest.
According to an aspect of the present invention, there is provided an integrated monitoring method for diagnosing a fault diagnosis of a vehicle using an ISOBUS and an image, the method comprising: acquiring an image from a plurality of cameras installed in the vehicle; Obtaining warning data warning of a fault condition of the vehicle; Extracting an associated image of a vehicle part related to the warning data from among the acquired images; And displaying the related image together with the warning data to provide the warning to the driver of the vehicle.
The method may further include wirelessly transmitting the related image and alert data to an external diagnostic device.
Other specific details of the invention are included in the detailed description and drawings.

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

According to the present invention, it is possible to integrate the warning data collected through the network with the image collected through the camera, and to provide the driver with fault information and the fault site accurately.
In addition, it is possible to shorten the trouble diagnosis and repair time by transmitting the information on the failure and the failure area obtained by integrating the warning data collected through the network and the image collected through the camera to the AS center or the maintenance person of the remote place .

delete

1 is a conceptual diagram of an integrated monitoring system for diagnosing a fault diagnosis of a vehicle using ISOBUS and an image according to an embodiment of the present invention.
2 is a block diagram of an integrated monitoring system for diagnosing faults of a vehicle using ISOBUS and an image according to an embodiment of the present invention.
3 is a diagram illustrating information provided through an integrated monitoring system for diagnosing faults of a vehicle using an ISOBUS and an image according to an embodiment of the present invention.
FIG. 4 is a block diagram of a diagnostic apparatus for receiving information provided through an integrated fault monitoring diagnosis system using an ISOBUS and an image according to an exemplary embodiment of the present invention. Referring to FIG.
5 is a block flow diagram of an integrated monitoring method for diagnosing a fault diagnosis of a vehicle using an ISOBUS and an image according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention and the manner of achieving them will become apparent with reference to the embodiments described in detail below with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Is provided to fully convey the scope of the invention to those skilled in the art, and the invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout the specification.
Although the first, second, etc. are used to describe various elements, components and / or sections, it is needless to say that these elements, components and / or sections are not limited by these terms. These terms are only used to distinguish one element, element or section from another element, element or section. Therefore, it goes without saying that the first element, the first element or the first section mentioned below may be the second element, the second element or the second section within the technical spirit of the present invention.
The terminology used herein is for the purpose of illustrating embodiments and is not intended to be limiting of the present invention. In the present specification, the singular form includes plural forms unless otherwise specified in the specification. As used herein, the terms "comprises" and / or "made of" means that a component, step, operation, and / or element may be embodied in one or more other components, steps, operations, and / And does not exclude the presence or addition thereof.
Unless defined otherwise, all terms (including technical and scientific terms) used herein may be used in a sense commonly understood by one of ordinary skill in the art to which this invention belongs. Also, commonly used predefined terms are not ideally or excessively interpreted unless explicitly defined otherwise.
Hereinafter, the present invention will be described in more detail with reference to the accompanying drawings.
1 is a conceptual diagram of an integrated monitoring system for diagnosing a fault diagnosis of a vehicle using ISOBUS and an image according to an embodiment of the present invention. 2 is a block diagram of an integrated monitoring system for diagnosing faults in a vehicle using ISOBUS and an image according to an embodiment of the present invention. FIG. 3 is a diagram illustrating information provided through an integrated monitoring system for diagnosing faults in a vehicle using an ISOBUS and an image according to an embodiment of the present invention.
Referring to FIG. 1, an integrated fault monitoring system 100 (hereinafter, referred to as 'fault diagnosis integrated monitoring system') using an ISOBUS and an image according to an embodiment of the present invention includes a plurality of cameras The image and the data are respectively obtained from the vehicle control unit 20 and a plurality of ECUs (Electronic Control Unit) 30 mounted on the vehicle, and the driver and the user of the vehicle 10 display the image and data together, Make it easy to check the status.
Here, the plurality of ECUs 30 are installed in the engine room 11 of the vehicle 10 and include an EMS (Engine Management System), a TCU (Telecommunication Control Unit), a BMS (Battery Management System), an ECS ), A smart cruise control (SCC), and the like, and is connected to the fault diagnosis integrated monitoring system 100 through a CAN (Controller Area Network) line. The CAN was originally developed as an automotive network protocol in the 1980s, and has excellent performance and low cost, and is designated by the ISO as the serial communication protocol ISO 11898 international standard, and its application to other industrial fields is being actively applied. In addition to CAN communication, LIN (Local Interconnect Network) and MOST (Media Oriented Systems Transport) can be used.
The plurality of ECUs 30 may be installed in the equipment-specific connection machine 13 connected to the vehicle 10 to control the functions thereof and may be connected to the fault diagnosis integrated monitoring system 100 via the CAN line. Particularly, when the special purpose vehicle connection machine 13 connected to the vehicle 10 is an agricultural machine, the fault diagnosis integrated monitoring system 100 obtains data from a plurality of ECUs 30 according to the ISO standard ISO 11783 of the agricultural machine can do. ISO 11783, also referred to as ISOBUS, defines standards and specifications applicable to agricultural machines (eg, tractors, etc.) and associated work machines based on network communication technology.
Each ECU 30 starts communication when the vehicle is started (IGN ON), transmits data through each node through CAN communication, notifies the occurrence of an event when an event occurs Data can be transmitted. At this time, not only the warning data indicating the failure of the vehicle but also various faults caused by communication problems can be included.
Here, the plurality of cameras 20 are installed in the engine room 11 of the vehicle 10, the room 12 of the vehicle 10, the specially equipped vehicle connecting machine 13 connected to the vehicle 10, And the image may include not only a still image but also a moving image. Here, a CCD (Charge Coupled Device) module or a CMOS (Complementary Metal Oxide Semiconductor) module can be used as an imaging device. It is possible to acquire an image using at least four cameras 20 in order to acquire an in-vehicle image. Particularly, the camera 20 mounted on the lower part of the illuminance of the engine room 11 inside the bonnet can easily acquire images by using an infrared camera or a thermal camera having good low-illuminance characteristics. For example, it may be an infrared camera including an infrared sensor module for detecting infrared rays. The infrared sensor module is preferably a near-infrared sensor module for sensing near-infrared rays, but it is not limited thereto, and may be a far-infrared sensor module for detecting far-infrared rays. The photographed image can be converted into a compressed format in which the image is compressed to facilitate data transmission. The image data in the form of a compressed format may have various formats such as MPEG (Moving Picture Experts Group) -1 or MPEG-4.
Referring to FIG. 2, the fault diagnosis integrated monitoring system 100 may include an image acquisition unit 110, a data acquisition unit 120, a display unit 130, and a control unit 140. The fault diagnosis integrated monitoring system 100 may further include an image analysis unit 115, a communication unit 150, an interface unit 155, and a storage unit 160.
The image acquiring unit 110 acquires images from a plurality of cameras 20 installed in the vehicle 10. At this time, the image obtaining unit 110 may obtain an infrared image from at least one infrared camera. The image acquired by the image acquisition unit 110 may be displayed on the display unit 130 together with warning data or the like through the process of the control unit 140. [
The data acquisition unit 120 acquires data representing the state of the vehicle 10 from a plurality of ECUs 30 mounted on the vehicle 10. [ In particular, the data acquiring unit 120 acquires warning data from the plurality of ECUs 30 to warn of a failure state. The data acquisition unit 120 can acquire data from the plurality of ECUs 30 via the CAN line. The data acquisition unit 120 may acquire data according to the ISO 11783 standard in the case of an agricultural machine or the like.
The display unit 130 displays the image acquired by the image acquisition unit 110 and the data acquired by the data acquisition unit 120. Particularly, the display unit 130 displays the relevant image of the vehicle part related to the warning data together with the warning data, so that the driver or the like of the vehicle 10 can effectively check the failure state of the vehicle 10. [ For example, as shown in FIG. 3, the display unit 130 may include a display module to display an image of a vehicle part related to the warning data together with warning data (cooling water pipe high temperature) on the screen 132 of the display module Can be displayed. The display unit 130 may further include an alarm module (not shown) such as a speaker for audibly warning the driver as well as a display module for displaying an image. The alarm module generates an event according to the warning data to the driver, and these events may include voice alarms of different volume depending on the degree of danger.
The control unit 140 controls the overall fault diagnosis diagnosis monitoring system 100 and controls the display unit 130 to display the related image of the vehicle part related to the warning data obtained from the ECU 30 together with the warning data . In particular, when the warning data corresponds to a preset emergency (for example, an engine fire), the control unit 140 may control the display unit 130 to flicker at least one of the related image or warning data . For example, the control unit 140 may blink the warning phrase "coolant pipe high temperature" displayed on the screen 132 of FIG. 3 or blink the associated image displayed above the phrase. In addition, the control unit 140 may control to simultaneously flash both the related image and warning data, or to sequentially flash related image and warning data.
The communication unit 150 wirelessly transmits the image and data to the external diagnostic apparatus 200. In particular, the communication unit 150 wirelessly transmits the relevant image of the vehicle site related to the warning data to the external diagnostic apparatus 200 together with the warning data and the DTC, thereby enabling the remote maintenance engineer to shorten the trouble diagnosis and repair time can do. Here, the communication unit 150 can communicate with the external diagnostic device 200 based on the short-range wireless communication or the long-distance wireless communication. In general, there will be more cases of communication based on short-range wireless communication. At this time, the near field wireless communication can use one of Bluetooth, Zigbee, Near Field Communication (NFC), and Wibree. Wireless LAN, Wireless MAN, WiMAX, Long Term Evolution (LTE), or the like may be used to transmit signal information based on long-distance wireless communication. Of course, it should be apparent to those skilled in the art that other wireless communication schemes may be employed without being limited to the above-mentioned near-field and long-distance wireless communication.
The image analyzing unit 115 analyzes an image acquired by the image acquiring unit 110 to detect an event. Here, an event includes all situations in which image data of the image can be changed, for example, a specific situation such as a rise in temperature or a fire occurs. For example, the image analysis unit 115 can detect an event using feature extraction and feature extraction for extracting feature information from an image input from the image acquisition unit 110. For example, a Haar-like feature using a sum of weighted products using a difference in the sum of pixel values (pixels) between two or more adjacent blocks in a local feature of an image, ) Can be applied. To extract the difference between the sum of the pixel values of neighboring blocks in extracting the Hahn-like feature, a mask considering a simple square feature can be used.
In addition, the image analyzer 115 may set the region of interest in the image and derive the feature information of the detected event in the region of interest. Specifically, the image analyzer 115 may set a region of interest (ROI) in the image and detect a specific pattern having an area having a predetermined brightness or more in the ROI as an event.
For example, the image analysis unit 115 may detect at least one of brightness data or color data of a region of interest, and may convert at least one of the detected brightness data or color data to a threshold brightness Data, and threshold color data. For example, when smoke is generated in the engine room 11 of the vehicle 10, the brightness data of the image may decrease, and the image analysis unit 115 may determine that the brightness data exceeds the threshold value of the preset brightness data It is possible to detect an event of occurrence of smoke in the event of a change.
As another example, if there is color data of a certain level of red or yellow in the region of interest, the image analysis unit 115 may determine that the proportion of the red and yellow data in the color data is It is possible to detect an event that a fire in the engine room 11 of the vehicle 10 has occurred.
The event detected by the image analysis unit 115 is transmitted to the driver of the vehicle 10 or to the external diagnostic apparatus 200 and is used as a basic data for quickly determining the information, .
The interface unit 155 directly connects to the external diagnostic device 200 and the like and can diagnose an abnormality of the vehicle even when an error such as a communication network failure of the vehicle, a collision,
The storage unit 160 stores various types of information. For example, the image acquired by the image acquisition unit 110, the event information according to the analysis of the image analysis unit 115, the data acquired by the data acquisition unit 120, the information received through the communication unit 150, And information to be processed by the control unit 140, and may store reference information and the like stored in a database in order to determine the failure of the vehicle 10. [ The storage unit 160 may be a hard disk, a flash memory, a Compact Flash card, an SD card (Secure Digital Card), an SM card (Smart Media Card), a MMC (Multimedia Card) Output module can be provided inside the fault diagnosis integrated monitoring system 100 according to an embodiment of the present invention or may be provided in a separate device.
The fault diagnosis integrated monitoring system 100 according to an exemplary embodiment of the present invention may be mounted on a vehicle as a separate module, but may be integrally mounted on a vehicle such as a black box or OBD (Onboard Diagnostics) installed in a vehicle. In such a case, the fault diagnosis integrated monitoring system 100 may be modularized and mounted inside a black box or the like.
FIG. 4 is a block diagram of a diagnostic apparatus for receiving information provided through an integrated fault monitoring diagnosis system using an ISOBUS and an image according to an exemplary embodiment of the present invention. Referring to FIG.
4, the diagnostic apparatus 200 includes a diagnostic input unit 210, a diagnostic communication unit 220, a diagnostic control unit 230, a diagnostic output unit 240, a diagnostic storage unit 250, a diagnostic interface unit 260, A diagnosis authentication unit 270, and the like.
The diagnosis input unit 210 is an input unit through which a user of the diagnostic apparatus 200 inputs and selects information, and can input various command information. In addition, the display may serve as the diagnostic input 210 if the display provides a touch screen function.
The diagnostic communication unit 220 is used to communicate with the fault diagnosis integrated monitoring system 100 wirelessly. Specifically, the diagnosis communication unit 220 and the communication unit 150 of the fault diagnosis integrated monitoring system 100 communicate with each other through a network or the like . The diagnosis communication unit 220 can receive relevant image and warning data from the fault diagnosis integrated monitoring system 100 and can transmit and receive various information to the fault diagnosis integrated monitoring system 100.
The diagnostic control unit 230 controls the diagnostic apparatus 200 and the respective components thereof. In addition, the diagnosis control unit 230 may display the related image and alert data generated and transmitted by the fault diagnosis integrated monitoring system 100 and output the same to the diagnostic output unit 240.
The diagnostic output unit 240 serves to display and provide information to a user of the diagnostic device 200 or the like. The diagnostic output unit 240 may be an audible display means such as a warning sound or a voice or a visual display means. For example, the diagnostic output unit 240 may include a speaker capable of outputting a sound, and may include a display module such as an LCD monitor or an LED capable of displaying characters, symbols, and the like.
The diagnostic storage unit 250 stores various information and data received from the outside. In particular, the diagnosis storage unit 250 can receive and store the information and data generated by the integrated fault diagnosis monitoring system 100.
The diagnostic interface unit 260 directly connects to the fault diagnosis integrated monitoring system 100 and detects the occurrence of an error such as a communication network failure of the vehicle, 10) can be diagnosed. When the diagnostic interface unit 260 is connected to the interface unit 155 of the fault diagnosis integrated monitoring system 100, only the authentication of the diagnostic authentication unit 270, which will be described later, have.
The diagnosis authentication unit 270 serves to authenticate the authority to access the fault diagnosis integrated monitoring system 100. [ For example, the user of the diagnostic apparatus 200 authenticates the right to access the fault diagnosis integrated monitoring system 100 through the indexer or password of the unique authentication module through the diagnostic authentication unit 270.
The diagnostic device 200 may be a general mobile communication terminal, a terminal capable of 2G / 3G / 4G, a WiBro wireless network service, a Palm personal computer, a personal digital assistant (PDA) ), A wireless application protocol phone (WAP phone), and the like, all of which may include a wired / wireless home appliance / communication device having a user interface for accessing a network, an interface for wireless LAN connection such as an IEEE 802.11 wireless LAN network card, Or the like. In addition, the diagnostic device 200 may be a communication device such as a server, a computer, a notebook, a tablet, or the like.
Each component of the fault diagnosis diagnosis monitoring system 100 and the diagnostic apparatus 200 may be implemented by software or hardware such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC) . ≪ / RTI > However, the components are not limited to software or hardware, and may be configured to be in an addressable storage medium and configured to execute one or more processors. The functions provided in the components may be implemented by a more detailed component or may be implemented by a single component that performs a specific function by combining a plurality of components.
5 is a block flow diagram of an integrated monitoring method for diagnosing a fault diagnosis of a vehicle using an ISOBUS and an image according to an embodiment of the present invention.
Referring to FIG. 5, an integrated monitoring method for diagnosing a fault diagnosis of a vehicle using ISOBUS and an image according to an embodiment of the present invention includes acquiring images from a plurality of cameras installed in the vehicle, (S20), and displays the related image together with the warning data (S30 (S30)). The warning data is displayed on the display unit ) To the driver of the vehicle. Then, the related image and warning data are wirelessly transmitted to an external diagnostic apparatus (S40), thereby reducing the time and effort required for the AS center or the mechanic of the remote place to diagnose the failure of the vehicle.
Meanwhile, an integrated monitoring method for diagnosing a fault diagnosis of a vehicle using ISOBUS and an image according to an embodiment of the present invention can be implemented as a single module by software and hardware, and the embodiments of the present invention described above can be implemented as a program And can be implemented in a general-purpose computer that operates the program using a computer-readable recording medium. The computer-readable recording medium is implemented in the form of a carrier wave such as a ROM, a floppy disk, a magnetic medium such as a hard disk, an optical medium such as a CD or a DVD, and a transmission through the Internet. In addition, the computer-readable recording medium may be distributed to a network-connected computer system so that computer-readable codes may be stored and executed in a distributed manner.
While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, You will understand. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive.

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

delete

100: Fault diagnosis integrated monitoring system
110: image acquisition unit 115: image analysis unit
120: data acquisition unit 130: display unit
140: control unit 150: communication unit
155: interface unit 160: storage unit

Claims (6)

An image acquiring unit acquiring images from a plurality of cameras respectively installed in an engine room of a vehicle, an interior of the vehicle, and an agricultural machine connected to the vehicle;
The method of claim 1, further comprising: detecting an event by analyzing the image, setting an area of interest in the image, detecting brightness data and color data of the area of interest, An image analyzer for detecting an event of occurrence of a fire when the specific gravity of red and yellow data in the color data is greater than a predetermined threshold specific gravity;
From the plurality of ECUs mounted on the vehicle via the CAN line, warning data warning a fault condition of the vehicle in accordance with the ISO 11898 standard, and from the ECU mounted on the agricultural machine via the CAN line, A data acquiring unit acquiring warning data warning a fault condition of the agricultural machine;
A display unit for displaying the image and alert data; And
And a control unit for controlling the display unit to display an associated image of the vehicle site related to the warning data together with the warning data. The method according to claim 1,
The image acquiring unit may acquire,
An integrated monitoring system for diagnosing faults in a vehicle using ISOBUS and images to acquire infrared images from at least one infrared camera. The method according to claim 1,
Wherein,
And controlling at least one of the related image or the warning data to flicker on the display unit when the warning data corresponds to a preset emergency situation. The method according to claim 1,
Further comprising a communication unit for wirelessly transmitting the related image and warning data to an external diagnostic apparatus. A method of controlling an automotive vehicle, comprising: acquiring images from a plurality of cameras respectively provided in an engine room of a vehicle, an interior of the vehicle, and an agricultural machine connected to the vehicle; Obtaining warning data warning a fault condition and acquiring warning data from an ECU mounted on the agricultural machine via a CAN line to warn the fault condition of the agricultural machine according to the ISO 11783 standard;
Extracting an associated image of a vehicle part related to the warning data from among the acquired images; And
And displaying the related image together with the warning data to provide the driver of the vehicle with the alarm data. 6. The method of claim 5,
And wirelessly transmitting the related image and warning data to an external diagnostic device.

Images