KR102040774B1 - Around view monitoring system and responsing method for self-propelled type agricultural machinery in emergency using the same - Google Patents

Abstract

According to the present invention, an AVM system comprises: an external camera acquiring a near field image and a far field image around self-propelled agricultural machinery at the same time; internal cameras respectively installed in an engine room of the self-propelled agricultural machinery, the interior of the self-propelled agricultural machinery, a trunk of the self-propelled agricultural machinery, and agricultural machinery connected to the self-propelled agricultural machinery; an image analyzing part which detects an event by analyzing an internal image acquired by the internal cameras; a data acquisition part acquiring warning data for warning of a fault condition through a CAN line; a display part selectively displaying a near field image and a far field image acquired by the external camera, various events detected through the image analyzing part, and warning data acquired through the data acquisition part; and a control part for performing control such that a warning sound is emitted when there is an event detection or the acquisition of warning data. Accordingly, for an agricultural work vehicle such as a tractor, not only the near work environment but also the remote work environment can be monitored, such that the convenience of work can be improved. In addition, by providing an image and information of a fault area of the self-propelled agricultural machinery together, the surrounding working environment and the fault condition can be confirmed at the same time, and since an emergency situation can be effectively responded, safe and efficient work can be achieved.

Description

AGM System and Self-propelled Agricultural Machinery Emergency Response Method Using the Same {AROUND VIEW MONITORING SYSTEM AND RESPONSING METHOD FOR SELF-PROPELLED TYPE AGRICULTURAL MACHINERY IN EMERGENCY USING THE SAME}

The present invention relates to an AVM system and a self-propelled agricultural machine emergency response method using the same, and more particularly, to effectively monitor the working environment when working on a farm tractor or the like and to ensure safety in the event of various events. The present invention relates to an AVM system capable of responding quickly to an emergency, and a self-propelled agricultural machine emergency response method using the same.

When working with a self-propelled agricultural machine, the size of the device is very large and the damage to the work environment or the damage of life around the work environment is frequently caused due to the obstruction of the device by the configuration attached to the device. In particular, there is a large obstacle around the work environment, if not found, the tractor, etc. may be overturned.

Normal cars such as passenger cars are equipped with devices such as parking assistance systems, so they can monitor the surrounding environment, but self-propelled agricultural machines are not provided with a means to monitor the surrounding environment. In addition, since the self-propelled agricultural machinery can perform safe and precise work by monitoring remote distances, it is not difficult to apply a parking assistance system of a general vehicle displaying only a near image.

On the other hand, recently, the failure status of each component of the vehicle is controlled and displayed in text form or the user interface environment (GUI) to check the failure status. For example, a self-propelled agricultural machine analyzes the information detected from various sensors installed around the engine to control the fuel injection of the engine, the actuator of the chassis and the like by the information of the door sensor installed in the vehicle body, etc. Many kinds of Electronic Control Units (ECUs), such as a body control device for controlling a vehicle, are mounted.

In addition, the electronic control unit is provided with a function of diagnosing a failure of the corresponding module by analyzing data sensed by various sensors. Specifically, when a failure is detected, a function (for example, blinking of a lamp) is driven to drive a display lamp warning the content of the failure in a predetermined display pattern. In this case, the display pattern is a diagnostic trouble code (DTC) that is set differently according to a failure state.

However, since the driver of the self-provided agricultural machine is notified in the form of a warning light of an instrument panel or an error code using an external diagnosis device, the driver can immediately confirm the failure, but it is difficult to know the exact location and state of the failure.

Republic of Korea Patent No. 1232639 (announced 13 February 2013) Republic of Korea Patent No. 1398724 (Announcement on May 27, 2014)

The present invention has been made in view of the above-described problems, and an object of the present invention is to provide an AVM system capable of monitoring not only a short-range working environment but also a long working environment in a self-propelled agricultural machine and a method for responding to an autonomous farming machine emergency using the same. In providing.

In addition, an object of the present invention is to provide an image and information of the failure site of the self-propelled agricultural machine at the same time to check the surrounding work environment and failure status, and can effectively respond to emergency conditions AVM system and self-propelled agricultural machine using the same To provide an emergency response method.

Problems to be solved by the present invention are not limited to the above-mentioned problems, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

An AVM system according to the present invention for achieving the above object is an external camera for acquiring near and far images at the same time around the self-propelled agricultural machinery; Internal cameras respectively installed in an engine room of the self-propelled agricultural machine, the interior of the self-propelled agricultural machine, the trunk of the self-propelled agricultural machine, and an agricultural machine connected to the self-propelled agricultural machine; An image analyzer configured to detect an event by analyzing an internal image acquired from the internal camera; A data acquisition unit for acquiring warning data for warning a failure state through a CAN line; A display unit for selectively displaying a near field image and a far field image obtained by the external camera, various events detected through the image analyzing unit, and warning data obtained through the data obtaining unit; And a controller configured to control to emit an alert sound when the event is detected or the alert data is acquired.

The image analyzer sets a region of interest in the internal image, detects brightness data and color data of the region of interest, and the brightness data exceeds a threshold of preset brightness data. Detects an event called the occurrence of smoke in the case of fluctuation, detects an event of the occurrence of fire when the specific gravity of the red and yellow data among the color data is greater than a predetermined threshold specific gravity, and the data acquisition unit detects the event through the CAN line. Acquire warning data from the plurality of ECUs mounted on the self-propelled agricultural machine to warn the failure state of the self-propelled agricultural machine according to the ISO 11898 standard, and obtain the ISO 11783 standard from the ECU mounted on the agricultural machine through the CAN line. According to the warning data can be obtained to warn the failure state of the agricultural machinery.

The image analyzer may determine whether there is a moving object in the image by analyzing object information existing in the near and far images when the warning data is acquired, the smoke occurrence event is detected, or the fire occurrence event is detected. The controller may be configured to transmit the warning sound to a predetermined volume or less when it is determined that the moving object exists in the near field image obtained from the external camera according to a result of analyzing the object information, and obtained from the external camera. When the moving object exists in the far image, the warning sound may be controlled to be transmitted at a predetermined volume or more.

And a storage unit for storing an emergency contact list corresponding to the detection of the smoke occurrence event, the detection of the fire occurrence event, or the acquisition of the warning data; And a communication unit configured to perform short-range or long-distance communication with the outside, wherein the control unit includes an emergency contact number corresponding to each situation when the smoke occurrence event is detected, the fire occurrence event is detected, or the warning data is acquired. The communication unit may be controlled to automatically connect to the network.

On the other hand, the emergency response method of the self-propelled agricultural machine according to the present invention for achieving the above object comprises the steps of simultaneously acquiring near-field image and remote image of the external of the self-propelled agricultural machine; Acquiring an internal image from an internal camera installed in an engine room of the self-propelled agricultural machine, the interior of the self-propelled agricultural machine, the trunk of the self-propelled agricultural machine, and an agricultural machine connected to the self-propelled agricultural machine; Analyzing an internal image obtained from the internal camera to detect an event, setting a region of interest in the internal image, detecting brightness data and color data of the region of interest, and the brightness data Detecting an event called the occurrence of smoke when the fluctuation exceeds the threshold value of the set brightness data, Detecting an event called the occurrence of fire when the specific gravity of the red and yellow data of the color data is greater than the predetermined threshold specific gravity ; Acquire warning data from the plurality of ECUs mounted on the self-propelled agricultural machine via the CAN line to warn the failure state of the self-propelled agricultural machine according to the ISO 11898 standard, and from the ECU mounted on the agricultural machine via the CAN line. Acquiring warning data to warn of a failure state of the agricultural machine according to ISO 11783 standard; Selectively displaying the near field image, the far field image, the smoke occurrence event, the fire occurrence event, and the warning data; And transmitting an alarm sound when there is a detection of the smoke occurrence event, the detection of the fire occurrence event, or the acquisition of the warning data.

Determining whether there is a moving object in the image by analyzing object information existing in the near and far images when the warning data is acquired, the smoke occurrence event is detected, or the fire occurrence event is detected; And when it is determined that the moving object exists in the near field image acquired from the external camera according to the analysis result of the object information, the warning sound is transmitted to a predetermined volume or less, and the moving object in the far field image obtained from the external camera. If present, the step of transmitting the warning sound at a predetermined volume or more; may further include.

The method may further include storing an emergency contact list corresponding to the detection of the smoke occurrence event, the detection of the fire occurrence event, or the acquisition of the warning data; And when there is a detection of the smoke occurrence event, the detection of the fire occurrence event, or the acquisition of the warning data, searching the emergency contact list and connecting the wireless communication to an emergency contact number corresponding to each situation. can do.

According to the AVM system and self-propelled agricultural machine emergency response method using the same, the self-propelled agricultural machine for agricultural work such as a tractor can monitor not only the near work environment but also the remote work environment, thereby improving the convenience of work. In addition, by providing images and information of the failure site of the self-propelled agricultural machine can be confirmed at the same time and the working environment and the failure status at the same time, it is possible to effectively respond to the emergency situation can achieve safer and more efficient work.

1 shows a camera arrangement of an AVM system in accordance with the present invention.
2 is a conceptual diagram of an AVM system according to an embodiment of the present invention.
3 is a block diagram of an AVM system according to an embodiment of the present invention.
4 is a diagram illustrating information displayed through the AVM system according to the present invention.
5 is a block diagram illustrating a configuration of a diagnostic apparatus for wirelessly connecting to an AVM system and receiving related information according to an embodiment of the present invention.
6 is a flow chart showing the flow of the self-propelled agricultural machinery emergency response method according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Advantages and features of the present invention and methods for achieving them will be apparent with reference to the embodiments described below in detail with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various forms. It is provided to fully convey the scope of the invention to those skilled in the art, and the present invention is defined only by the scope of the claims. Like reference numerals refer to like elements throughout.

Although the first, second, etc. are used to describe various elements, components and / or sections, these elements, components and / or sections are of course not limited by these terms. These terms are only used to distinguish one element, component or section from another element, component or section. Therefore, the first device, the first component, or the first section mentioned below may be a second device, a second component, or a second section within the technical spirit of the present invention.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and / or “made of” refers to a component, step, operation, and / or element that includes one or more other components, steps, operations, and / or elements. It does not exclude existence or addition.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used in a sense that can be commonly understood by those skilled in the art. In addition, the terms defined in the commonly used dictionaries are not ideally or excessively interpreted unless they are specifically defined clearly.

Hereinafter, the present invention will be described in more detail with reference to the accompanying drawings.

1 illustrates an external camera arrangement of an AVM system according to the present invention, and FIG. 2 is a conceptual diagram of an AVM system 100 according to an embodiment of the present invention. As shown in Figures 1 and 2, the self-propelled agricultural machine is provided with a plurality of external cameras (110-1 to 110-4) and a plurality of internal cameras 20. In FIGS. 1 and 2, four external cameras and three internal cameras are provided, but fewer or more cameras may be provided. In the present specification, the vehicle to which the present invention is applied is specifically described as a self-propelled agricultural machine, but it is apparent to those skilled in the art that the present invention can be applied to vehicles other than the agricultural machine. Therefore, the self-propelled agricultural machine described in the claims should also be broadly interpreted as a vehicle, for example, a truck, a passenger car, an SUV, etc., which can be used for agriculture in connection with an "agricultural machine".

The external cameras 110-1 to 110-4 collect images of the near area A _SD and the far area A _LD at a corresponding location in real time. The boundary between the near area A _SD and the far area A _LD may be appropriately adjusted by user setting or camera performance. Preferably, the near area A _SD is an area between 0 m and 2 m, and the far area A _LD may be an area of 2 m or more.

Among the external cameras, the front camera 110-1 photographs images of the near area A _SD and the far area A _{LD in} front of the self-propelled agricultural machine, and transfers each image to the controller of the AVM system 100. The left camera 110-2 captures images of the near area A _SD and the far area A _{LD on the} left side of the self-propelled agricultural machine, and transfers each image to the controller of the AVM system 100, and the rear camera ( 110-3 captures images of the near area (A _SD ) and the far area (A _LD ) behind the self-propelled agricultural machine and transfers each image to the controller of the AVM system 100, and the right camera 110-4. ) _Takes an image of the near area (A _SD ) and the far area (A _LD ) on the right side of the self-propelled agricultural machine and transfers each image to the controller of the AVM system 100.

On the other hand, the internal camera 20, unlike the camera (110-1 to 110-4) to shoot the surrounding environment of the self-propelled agricultural machinery engine room 11, indoor 12, self-propelled agricultural machinery of the self-propelled agricultural machinery It is installed in a special vehicle connecting machine (13) connected to the.

The external cameras 110-1 to 110-4 and the internal camera 20 may further include a camera adjusting unit (not shown) for adjusting the field of view, and the camera adjusting unit of each of the cameras 110-1 to 110-4. By rotating or repositioning, images of the near area A _SD and the far area A _LD may be selectively acquired. Similarly, the camera adjuster may perform rotation or position adjustment of the internal camera 20.

In this case, the images acquired by the cameras 110-1 to 110-4 and 20 include not only a still image but also a video. Each camera 110-1 to 110-4, or 20, may be a charge coupled device (CCD) module or a complementary metal oxide semiconductor (CMOS) module as an imaging device.

The internal camera 20 may be mounted on the inside of the bonnet and at an area of low illuminance of the engine room 11, and at this time, an infrared camera or a thermal imaging camera having good low illuminance characteristics may be used to facilitate image acquisition.

For example, an infrared camera including an infrared sensor module for detecting infrared rays may be further installed. The infrared sensor module is preferably a near infrared sensor module for detecting near infrared rays, but is not limited thereto, and may be a far infrared sensor module for detecting far infrared rays. The captured image may be converted into a compressed format in which the image is compressed to facilitate data transmission. Image data in the form of a compressed format may have various formats such as Moving Picture Experts Group (MPEG) -1 or MPEG-4.

2, the AVM system 100 according to an embodiment of the present invention is a plurality of external cameras (110-1 to 110-4) and the internal camera 20 and the self-propelled agricultural machine installed on a self-propelled agricultural machine Acquire images and data from a plurality of ECU (Electronic Control Unit; 30) mounted on each, and display the images and data together, so that the driver or user of the self-propelled agricultural machine such as a tractor grasps near and far images in real time. At the same time, it is easy to check the fault condition.

Here, the plurality of ECU 30 is installed in the engine room 11 of a self-propelled agricultural machine such as a tractor, and includes an engine management system (EMS), a telecommunication control unit (TCU), a battery management system (BMS), and an ECS ( It is installed in Electronic Power Steering (SCC), Smart Cruise Control (SCC), etc. and controls its function, and is connected to the AVM system 100 through a controller area network (CAN) line. The CAN was originally developed as a network protocol for self-provided agricultural machinery in the 1980s, and has high performance and low cost, and is designated as an ISO 11898 international standard for serial communication protocol by ISO. have. In addition to such CAN communication, a local interconnect network (LIN), a media oriented systems transport (MOST), or the like may be used.

In addition, the plurality of ECU 30 may be installed in a special vehicle connection machine 13 connected to a self-propelled agricultural machine such as a tractor, and the like, and control the function thereof, and may be connected to the AVM system 100 through a CAN line. In particular, when the vehicle connecting machine 13 connected to the self-propelled agricultural machine is an agricultural machine, the AVM system 100 may obtain data from the plurality of ECUs 30 by ISO 11783 which is an ISO standard of the agricultural machine. . ISO 11783 is also called ISOBUS, and this international standard defines standards and specifications applicable to agricultural machinery (eg tractors, etc.) and their accompanying working machines, based on network communication technology.

Then, each ECU 30 starts communication when the power is supplied to the start of the self-propelled agricultural machine (IGN ON), and transmits data through the respective node through CAN communication, and if an event occurs, an event is generated. Data can be sent to notify the fact. In this case, as well as the warning data indicating whether the self-propelled agricultural machinery has failed, it may include various failures caused by communication problems.

The external cameras 110-1 to 110-4 acquire an image of the surrounding environment of the self-propelled agricultural machine, but the internal camera provided separately from the engine room 11, the indoor 12 of the self-propelled agricultural machine, and often It is installed on a special vehicle connecting machine 13 connected to the agricultural type machine, etc. to take an image, but the image may include not only a still image but also a moving image.

3 is a block diagram of an AVM system 100 according to an embodiment of the present invention, and FIG. 4 is a diagram illustrating information provided through the AVM system 100 according to the present invention.

Referring to FIG. 3, the AVM system 100 may include an image acquirer 110, a data acquirer 120, a display 130, and a controller 140. In addition, the AVM system 100 may further include an image analyzer 115, a communication unit 150, an interface unit 155, and a storage unit 160.

The image acquisition unit 110 acquires an image from a plurality of external cameras 110-1 to 110-4 installed in a self-propelled agricultural machine such as a tractor. The images obtained from the plurality of external cameras 110-1 to 110-4 may be images of a remote region as well as a near region of the working environment. As described above, the plurality of external cameras 110-1 to 110-4 for acquiring the plural images may be implemented to selectively or simultaneously acquire the near image and the far image by having a plurality of lenses having different angles of view.

In addition, the image acquisition unit 110 may secure the image data for checking the state of the self-propelled agricultural machinery through the internal camera 20. In this case, the internal camera 20 may be an infrared camera, and the image acquisition unit 110 acquires an infrared image from at least one infrared camera, and then displays the display unit 130 along with warning data through processing of the controller 140. Can be displayed.

The data acquisition unit 120 obtains data indicating the state of the self propellered agricultural machine from the plurality of ECUs 30 mounted on the self propellered agricultural machine. In particular, the data acquisition unit 120 obtains warning data warning a failure state from the plurality of ECUs 30. The data acquisition unit 120 may acquire data from the plurality of ECUs 30 through the CAN line. In addition, the data acquisition unit 120 may obtain data according to the ISO 11783 standard in the case of agricultural machinery.

The display unit 130 may display an image acquired by the image acquisition unit 110, data acquired by the data acquisition unit 120, and short-range and long-range images obtained through the external cameras 110-1 to 110-4. Display.

The display unit 130 displays the related image of the self-propelled agricultural machine part related to the warning data together with the warning data, so that the driver of the self-propelled agricultural machine and the like can effectively check the failure state of the self-propelled agricultural machine. For example, as shown in FIG. 3, the display unit 130 includes a display module such that the self-propelled agricultural machine part associated with the warning data together with warning data (coolant pipe high temperature) on the screen 132 of the display module. Can be displayed.

The display unit 130 may further include an alarm module (not shown), such as a speaker that not only visually warns the driver but also a display module for displaying an image. The alarm module generates an event according to the warning data to the driver, and these events may include voice warnings of different volumes according to the degree of danger. In particular, the alarm module may change the volume of the warning sound according to an analysis result of whether there is a moving object (eg, a person) in the near and far image data acquired from the external cameras 110-1 to 110-4. .

In detail, the image analyzing unit 115 acquires the fault condition warning data to be described below, and detects the smoke occurrence event or the detection of the fire occurrence event in the near field acquired from the external cameras 110-1 to 110-4. Object information existing in the image and the remote image is analyzed to determine whether there is a moving object in the image.

When the moving object exists in the near field image according to the object information analysis result of the image analyzing unit 115, the controller 140 controls the alarm module to transmit a warning sound at a predetermined volume or less. On the contrary, when there is a moving object in the far image, a warning sound is transmitted at a predetermined volume or more.

This can prevent damage to surrounding lives when there is a risk of blasting, etc. in the event of a problem with the self-propelled agricultural machinery, and can identify abnormal conditions of the self-propelled agricultural machinery even when the driver is far from the self-propelled agricultural machinery. It becomes possible.

The controller 140 controls the AVM system 100 as a whole and selectively displays the near and far images acquired from the plurality of external cameras 110-1 to 110-4 according to a user command.

In addition, the controller 140 controls the display unit 130 to display the related image of the self-propelled agricultural machine part related to the warning data obtained from the ECU 30 together with the smoke / fire event or the warning data.

In particular, when the warning data corresponds to a preset emergency situation (for example, an engine fire, etc.), the controller 140 may control the display 130 to blink at least one of the related image or the warning data. .

For example, as shown in FIG. 4, the controller 140 selectively displays the near and far images according to a user command, and at the same time blinks the warning phrase "coolant pipe high temperature" or the self-propelled agricultural machine image. You can mark the relevant marker to flash above. In addition, the controller 140 may control to sequentially display the related image and the warning data.

As shown in FIG. 4, the display unit 130 selects an image display area A1 for displaying images acquired from the plurality of external cameras 110-1 to 110-4, a near field image, and a far field image. Area A2, a failure display area A3 indicating an emergency occurrence point, a thermal image display area A4 obtained by the internal camera 20, and a warning message display area A5. In this case, the image display area A1 may be partitioned to display as many images as the number of external cameras.

The communication unit 150 wirelessly transmits the image and the data to an external diagnostic device. In particular, the communication unit 150 wirelessly transmits the related image of the self-propelled agricultural machine part related to the warning data together with the warning data or the DTC to an external diagnosis device, thereby significantly reducing the time for the trouble diagnosis and repair of the remote site. can do. Here, the communication unit 150 may communicate with an external diagnostic device based on short range wireless communication or long range wireless communication. In general, there will be more cases of communicating based on near field communication. In this case, the short range wireless communication may use one of Bluetooth, Zigbee, Near Field Communication (NFC), and Wibree. When transmitting signal information based on long range wireless communication, a wireless LAN, a wireless man, a WiMAX, a long term evolution (LTE), or the like may be used. Of course, it will be apparent to those skilled in the art that the present invention is not limited to the short range and long range wireless communication, and other wireless communication schemes may be adopted.

The image analyzer 115 detects an event by analyzing an image acquired by the image acquirer 110. Here, the event (event) is the presence of obstacles or dangerous objects included in the images obtained from the plurality of external cameras (110-1 to 110-4) and the internal camera 20, the presence or absence of moving objects, self-propelled agricultural machinery This includes all situations that can change the image data of the image, such as changing the temperature of the temperature, occurrence of a fire.

For example, the image analyzer 115 may detect an event by using feature extraction and feature extracted to extract feature information from an image input from the image acquirer 110. When analyzing the images of the external cameras 110-1 to 110-4, when the movement of the corresponding object is detected through feature extraction of the object of interest, it may be determined as a moving object such as a person or an animal. In addition, it is possible to recognize the characteristics of the size, structure and the like, to recognize buildings, self-propelled agricultural machinery and the like.

In particular, the image analyzer 115 may detect various events as follows based on the image input from the internal camera 20. For example, as a local feature of an image, a Haal-like feature using a sum of weighted products using a difference or a sum of a sum of pixels (pixel) values between two or more adjacent blocks as a local feature of an image. ) Can be applied. In order to obtain a difference of sum of pixel values between neighboring blocks when extracting a Haal-like feature, a mask considering a simple rectangular feature may be used.

In addition, the image analyzer 115 may set a region of interest in the image and derive feature information of an event detected in the region of interest. In detail, the image analyzer 115 may set a region of interest (ROI) in the image and detect a specific pattern including an area having a predetermined brightness or more in the region of interest as an event.

For example, the image analyzer 115 may detect at least one of brightness data or color data of the ROI, and at least one data of the detected brightness data or color data may be threshold brightness. The event can be detected by comparison with the data and the threshold color data. For example, when the smoke occurs in the engine room 11 of the self-propelled agricultural machine, the brightness data of the image may decrease, and the image analyzer 115 may exceed the threshold value of the preset brightness data. In the case of a change, an event of occurrence of smoke can be detected.

As another example, if there is more than a certain level of red and yellow color data in the region of interest, it may be determined that the flame is spreading. The image analyzer 115 may determine the specific gravity of the red and yellow data in the color data. When greater than the predetermined threshold specific gravity, it is possible to detect an event that a fire of the engine room 11 of the self-propelled agricultural machine has occurred.

The event detected by the image analysis unit 115 is transmitted to the driver of the self-propelled agricultural machine or an external diagnostic device, and is a basic data for quickly determining information, which helps to establish a quick response action.

The interface unit 155 directly connects to an external diagnostic device and the like, so that an error of the self-propelled agricultural machine can be diagnosed even when an error such as a communication network failure of a self-provided agricultural machine, a collision occurrence, a failure of a warning light, etc. do.

The storage unit 160 stores various kinds of information. For example, near and far images acquired from the plurality of external cameras 110-1 to 110-4 and 20, event information according to the analysis of the image analyzer 115, and data acquired by the data acquirer 120. Store information received through the communication unit 150, information processed by the control unit 140, and the like to store reference information stored in a database to determine the failure of the self-propelled agricultural machine.

Furthermore, the storage 160 may store in advance an emergency contact list corresponding to the detection of a smoke occurrence event, the detection of a fire occurrence event, or the acquisition of a failure state warning data. For example, when a smoke or fire occurrence event is detected, an emergency contact list such as a hospital or 119 may be stored, and when a failure state warning data is obtained, a contact list such as a repair center according to a failure area may be stored. Can be.

In this regard, the control unit 140 displays an emergency contact list through the display unit 130 when there is a detection of a smoke occurrence event, a detection of a fire occurrence event, or acquisition of a failure state warning data, or an emergency contact corresponding to each situation. The communication unit 150 may be controlled to automatically connect the number.

The storage unit 160 may include a hard disk, a flash memory, a compact flash card (CF), a secure digital card (SD), a smart card (SM), a multimedia card (MMC), or a memory stick. As a module capable of inputting / outputting, a module may be provided inside the AVM system 100 according to an embodiment of the present invention or may be provided in a separate device.

The AVM system 100 according to an embodiment of the present invention may be mounted in a self-propelled agricultural machine as a separate module, but may be integrally mounted in a black box or an on board doagnostics (OBD) installed in the self-propelled agricultural machine. have. In this case, the AVM system 100 may be modularized and mounted inside the black box.

5 is a block diagram of a diagnostic apparatus for receiving information provided through an AVM system according to an embodiment of the present invention.

5, the diagnostic apparatus 200 includes a diagnostic input unit 210, a diagnostic communication unit 220, a diagnostic control unit 230, a diagnostic output unit 240, a diagnostic storage unit 250, and a diagnostic interface unit 260. , The diagnostic authentication unit 270 may be included.

The diagnostic input unit 210 is an input unit for inputting and selecting information by a user of the diagnostic apparatus 200, and may input various command information. In addition, when the display provides a touch screen function, the display may serve as the diagnostic input unit 210.

The diagnostic communication unit 220 serves to wirelessly communicate with the AVM system 100. Specifically, the diagnostic communication unit 220 and the communication unit 150 of the AVM system 100 are connected through a network. The diagnostic communication unit 220 may receive related images and warning data from the AVM system 100, and may also transmit and receive various information with the AVM system 100.

The diagnostic control unit 230 controls the diagnostic apparatus 200 and its components. In addition, the diagnostic controller 230 may display the related image and the warning data generated and transmitted by the AVM system 100 and output the same to the diagnostic output unit 240.

The diagnostic output unit 240 serves to display and provide information to a user of the diagnostic apparatus 200. The diagnostic output unit 240 may be an audible display means such as a warning sound or a voice or a visual display means. For example, the diagnostic output unit 240 may include a speaker capable of outputting sound, and may also include a display module such as an LCD monitor or an LED that can display characters, symbols, and the like.

For example, when there is a moving object in the remote image data as a result of the analysis of the image analyzer 115, and the fire event occurs, the diagnostic output unit 240 may transmit a warning sound of a predetermined volume or more.

The diagnostic storage unit 250 stores various information and data received from the outside, and in particular, may receive and store various information and data produced by the AVM system 100. Furthermore, the diagnostic storage unit 250 stores an emergency contact list related to various events and warning data. When an event or warning data is generated, the diagnostic storage unit 250 reads the corresponding list from the diagnostic storage unit 250 and outputs the diagnostic output unit 240. ) Can be displayed.

The diagnostic interface unit 260 directly connects with the AVM system 100. The diagnostic interface unit 260 uses the diagnostic apparatus 200 even when an error such as a communication network failure of a self-propelled agricultural machine, a collision occurrence, a failure of a warning light, or the like occurs. To diagnose the abnormality of agricultural machinery. When the diagnostic interface unit 260 connects to the interface unit 155 of the AVM system 100 only through authentication of the diagnostic authentication unit 270 described later, information security, confidentiality, and the like can be maintained.

The diagnostic authentication unit 270 authenticates the authority to access the AVM system 100. For example, the diagnostic authentication unit 270 authenticates a user of the diagnostic apparatus 200 to access the AVM system 100 through an indexer or a password of a unique authentication module.

The diagnostic apparatus 200 is a general mobile communication terminal, 2G / 3G / 4G, a WiBro wireless network service terminal, Palm Personal Computer (PDA), Personal Digital Assistant (PDA), smart phone (Smart phone) ), Which may mean all wired and wireless home appliances / communication devices having a user interface for accessing a network such as a WAP phone (WAP phone) and an interface for accessing a wireless LAN such as an IEEE 802.11 WLAN network card. It may be provided with a device. In addition, the diagnostic apparatus 200 may be a communication device such as a server, a computer, a laptop, a tablet, or a device including the same.

Up to now, each component of the AVM system 100 and the diagnostic apparatus 200 may refer to software or hardware such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). Can be. However, the components are not limited to software or hardware, and may be configured to be in an addressable storage medium and may be configured to execute one or more processors. The functions provided in the above components may be implemented by more detailed components, or may be implemented as one component that performs a specific function by combining a plurality of components.

Figure 6 is a flow chart of a self-propelled agricultural machinery emergency response method according to the present invention. As shown in FIG. 6, the self-propelled agricultural machine emergency response method according to the present invention first acquires a short-range image and a long-range image of the outside of the self-propelled agricultural machine (S310). Specifically, the external cameras (110-1 to 110-4) provided on the outside of the self-propelled agricultural machinery collects the image of the near area (A _SD ) and the far area (A _LD ) in real time at the corresponding location, the near area ( The boundary between A _SD ) and the far area (A _LD ) may be properly adjusted by user setting or camera performance. Preferably, the near area A _SD is an area between 0 m and 2 m, and the far area A _LD may be an area of 2 m or more.

In addition, an internal image is obtained inside the self-propelled agricultural machine. That is, an internal image is acquired from an internal camera installed inside the self-propelled agricultural machine (S320). Specifically, the internal image is obtained from an internal camera installed in the engine room of the self-propelled agricultural machine, the interior of the self-propelled agricultural machine, the trunk of the self-propelled agricultural machine, and the agricultural machine connected to the self-propelled agricultural machine.

Thereafter, the internal image obtained from the internal camera is analyzed, and various events (fire and smoke) are detected (S330). In detail, an event is detected by analyzing an internal image obtained from an internal camera, and a region of interest is set in the internal image, brightness and color data of the region of interest are detected, and the brightness data. Detects an event of occurrence of smoke when is changed beyond a threshold of preset brightness data, and detects an event of occurrence of fire when the specific gravity of red and yellow data among the color data is greater than a predetermined threshold specific gravity do.

After event detection is completed, warning data for warning of a fault condition is obtained from the ECU through the CAN line (S340). Specifically, warning data is obtained from the plurality of ECUs mounted on the self-propelled agricultural machine through the CAN line to warn the failure state of the self-propelled agricultural machine according to the ISO 11898 standard, and mounted on the agricultural machine via the CAN line. Warning data is obtained from the ECU to warn the failure state of the agricultural machine according to the ISO 11783 standard.

When the detection of the smoke / fire occurrence event and the acquisition of the warning data are made, the near-field / distant image, various events (smoke-fired event, fire-fired event) and the warning data acquired from an external camera are selectively displayed (S350). .

When there is a detection of a smoke occurrence event, a detection of a fire occurrence event, or acquisition of warning data, a warning sound is sent (S360).

In this case, when there is an acquisition of warning data, detection of a smoke occurrence event, or detection of a fire occurrence event, determining whether there is a moving object in the image by analyzing object information existing in the near and far images and analyzing the object information. According to the result, when it is determined that the moving object exists in the near image acquired from the external camera, a warning sound is sent out below the preset volume, and when the moving object exists in the far image acquired from the external camera, the warning sound is above the preset volume. The method may further include transmitting.

The method may further include storing an emergency contact list corresponding to detection of a smoke occurrence event, detection of a fire occurrence event, or acquisition of warning data. The step of storing the emergency contact list may be performed before the near field image and the far image acquisition step (S310) outside the self-propelled agricultural machine.

When there is a detection of a smoke occurrence event, a detection of a fire occurrence event, or acquisition of warning data, the method may further include searching for an emergency contact list and connecting the wireless communication to an emergency contact number corresponding to each situation.

According to another embodiment of the self-propelled agricultural machine emergency response method, obtaining near and far images through a plurality of cameras (110-1 to 110-4) arranged in front, rear, left and right of the self-propelled agricultural machinery, image display Receiving a user command related to the step, selectively extracting near and far images based on the user command, obtaining warning data, extracting the self-propelled agricultural machine image related to the warning data, and the near / far image And displaying the selected image and the warning data related image based on the user command.

The user command related to the image display is implemented by the display unit 130 provided in the self-propelled agricultural machine, such as a touch screen, generated by the user touching the text or picture displayed on the touch screen, and may be transmitted to the controller 140. have. When the display unit 130 is not implemented as a touch screen, a user input unit such as a button may be further provided.

Selectively extracting near and far images based on a user command may selectively obtain near and far images by changing viewpoints of the plurality of cameras 110-1 to 110-4.

In the step of acquiring the warning data, the data indicating unit 120 may acquire data indicating the state of the self propellered agricultural machine from the plurality of ECUs 30 mounted on the self propellered agricultural machine. In particular, the data acquisition unit 120 obtains warning data warning a failure state from the plurality of ECUs 30. The data acquisition unit 120 may acquire data from the plurality of ECUs 30 through the CAN line. In addition, the data acquisition unit 120 may obtain data according to the ISO 11783 standard in the case of agricultural machinery.

Extracting the self-propelled agricultural machine image related to the warning data includes an internal camera 20 installed in the engine room 11, the interior 12 of the self-propelled agricultural machine 12, and the vehicle linking machine 13 connected to the self-propelled agricultural machine 13. Image acquisition can be achieved through.

The remote or near field image acquired by the external camera and the warning data related image obtained by the internal camera may be displayed to the user through the display unit 130.

Meanwhile, the exemplary embodiment may further include wirelessly transmitting the related image and warning data to an external diagnostic device, which is a time and effort for a remote AS center or a mechanic to diagnose a failure of a self-propelled agricultural machine. Etc. can be reduced.

The AVM system and self-propelled agricultural machine emergency response method using the same according to an embodiment of the present invention can be implemented as a single module by software and hardware, and the above-described embodiments of the present invention can be written as a program that can be executed in a computer. The computer may be implemented in a general-purpose computer for operating the program using a computer-readable recording medium. The computer-readable recording medium may be implemented in the form of a carrier wave such as a magnetic medium such as a ROM, a floppy disk, a hard disk, an optical medium such as a CD or a DVD, and a transmission through the Internet. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

Although embodiments of the present invention have been described above with reference to the accompanying drawings, those skilled in the art to which the present invention pertains may implement the present invention in other specific forms without changing the technical spirit or essential features thereof. I can understand that. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive.

100: AVM system
110: image acquisition unit
110-1 to 110-4: external camera
115: image analysis unit
120: data acquisition unit
130: display unit
140: control unit
150: communication unit
155: interface unit
160: storage unit

Claims (7)

An external camera for simultaneously acquiring near-field and far-field images around the self-propelled agricultural machinery;
Internal cameras respectively installed in an engine room of the self-propelled agricultural machine, the interior of the self-propelled agricultural machine, the trunk of the self-propelled agricultural machine, and an agricultural machine connected to the self-propelled agricultural machine;
An image analyzer configured to detect an event by analyzing an internal image acquired from the internal camera;
A data acquisition unit for acquiring warning data for warning a failure state through a CAN line;
A display unit for selectively displaying a near field image and a far field image obtained by the external camera, various events detected through the image analyzing unit, and warning data obtained through the data obtaining unit; And
And a control unit which controls to transmit an alarm sound when there is detection of the event or acquisition of the warning data.
The image analyzing unit analyzes object information existing in the near and far images when there is an acquisition of warning data, detection of a smoke occurrence event or detection of a fire occurrence event, and determines whether there is a moving object with motion in the image.
If it is determined that there is a moving object with movement in the near field image acquired from the external camera according to a result of analysis of the object information, the control unit transmits the warning sound to a predetermined volume or less, and obtained from the external camera. When there is a moving object with movement in the far image, the warning sound is controlled to be transmitted at a predetermined volume or more, thereby preventing damage to surrounding people.
The display unit,
And an image display area, an image selection area, a failure display area, a thermal image display area, and a warning message display area, wherein the image display area is divided by the number of the external cameras provided in the self-propelled agricultural machine. The method of claim 1,
The image analyzer sets a region of interest in the internal image, detects brightness data and color data of the region of interest, and changes the brightness data beyond a threshold of preset brightness data. If the event of occurrence of smoke is detected, and if the specific gravity of the red and yellow data of the color data is greater than the predetermined threshold specific gravity, the event of the occurrence of fire is detected,
The data acquisition unit obtains warning data warning a failure state of the self-propelled agricultural machine according to ISO 11898 standard from a plurality of ECUs mounted on the self-propelled agricultural machine through the CAN line, and uses the CAN line for the agricultural AVM system for acquiring warning data from a ECU mounted in a machine to warn of a failure state of the agricultural machine according to the ISO 11783 standard. The method of claim 2,
A storage unit for storing an emergency contact list corresponding to detection of the smoke occurrence event, detection of the fire occurrence event, or acquisition of the warning data; And
Further comprising; a communication unit for performing short-range or long-distance communication with the outside,
And the control unit controls the communication unit to automatically connect to an emergency contact number corresponding to each situation when the detection of the smoke occurrence event, the detection of the fire occurrence event, or the acquisition of the warning data occurs. delete delete delete delete

Images