KR102521841B1 - System and method for detecting the occurrence of danger according to surrounding environment of the moving object - Google Patents

Abstract

Provided are a system and a method for detecting the occurrence of danger according to a surrounding environment of a moving object. The system for detecting the occurrence of danger according to a surrounding environment of a moving object according to various embodiments of the present invention comprises: an integrated monitoring module disposed in a moving object to collect sensor data corresponding to the moving object; and a control server using the sensor data collected by the integrated monitoring module to detect the occurrence of danger for the moving object, wherein the control server sets criteria for detecting danger based on the surrounding environment of the moving object, and detects the occurrence of danger for the moving object by using the status of the moving object, which is determined based on the collected sensor data, and the set criteria. Accordingly, the present invention can prevent the occurrence of danger for the moving object.

Description

Danger detection system and method according to the surrounding environment of a moving object

Various embodiments of the present invention relate to a system and method for detecting danger according to the surrounding environment of a moving object, and more specifically, to a danger occurrence according to the surrounding environment of a moving object for detecting the occurrence of danger to the moving object in consideration of the surrounding environment of the moving object. It relates to a detection system and method.

With the recent development of information and communication technology, advanced agricultural technologies grafted with information and communication technology have appeared in the agricultural field. For example, there are smart farms such as indoor vertical farms and smart barns, and precision agriculture that optimizes the efficiency of agricultural production management.

However, most of the agricultural fields grafted with information and communication technology are confined to producing and managing agricultural products, and an appropriate technology grafted with information and communication technology has not yet been proposed for managing agricultural machinery for vehicles used in agriculture. .

On the other hand, recently, urban farms with good accessibility from cities have appeared, but most farms are still located in areas far from cities or in deprived areas. In the case of these areas, since there are many unpaved roads or bare land, the risk of accidents of agricultural machinery for vehicles is inevitably greater than that of cities.

In fact, accidents at agricultural sites across the country, which are steadily increasing every year, are frequently caused by various causes such as aging of the agricultural population, inexperience in operating agricultural machinery, carelessness, drinking or driving alone. 3 years) Out of 422 agricultural machinery accidents, 255 cases were overturning accidents, accounting for a whopping 60.4%.

In particular, since agricultural machinery overturning accidents often occur during independent operation, in case of an agricultural machinery accident, it is necessary to report the accident directly, and if the driver is injured, it is left unattended until a witness appears and a dangerous situation may occur. there is

In order to solve this problem, a device capable of managing agricultural machinery, such as a black box, has been proposed, but this is nothing more than recording data after an accident or at the time of an accident. That is, there is a problem in that it is difficult to prevent an accident from occurring or to respond quickly when an accident occurs by preventing or warning an accident in advance before it occurs.

In addition, devices capable of managing agricultural machines, such as black boxes, only detect the occurrence of accidents for only one agricultural machine on which the device is installed, and a control system that monitors the occurrence of accidents for multiple agricultural machines. There is a problem in that it is difficult to efficiently manage risk occurrence for a plurality of agricultural machines in that there is no (eg, an accident monitoring system for a plurality of agricultural machines located in a specific area).

The problem to be solved by the present invention is to determine the operation of the moving object using sensor data on the moving object collected using an integrated monitoring module installed in the moving object, and accordingly, to determine the possibility of danger to the moving object or to determine whether or not danger has occurred. However, by determining the possibility of danger to the moving object or determining whether or not the risk occurs according to the surrounding environment of the moving object, not only can the occurrence of danger to the moving object be prevented, but also an environment in which quick response is possible in case of danger to the moving object is established. An object of the present invention is to provide a system and method for detecting danger according to the surrounding environment of a moving object.

Another problem to be solved by the present invention is not only detecting the occurrence of danger for each of a plurality of moving objects, but also detecting the occurrence of danger for a plurality of moving objects located in a specific area, thereby a plurality of moving objects (eg, located in a specific area). An object of the present invention is to provide a system and method for detecting danger according to the surrounding environment of a mobile body, which can provide an environment in which risk occurrence monitoring for a plurality of moving bodies can be performed more efficiently.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

A risk detection system according to the surrounding environment of a mobile body according to an embodiment of the present invention for solving the above problems is provided in a mobile body and includes an integrated monitoring module that collects sensor data corresponding to the mobile body and the integrated monitoring module. and a control server that detects the occurrence of danger to the moving object using sensor data collected via Danger to the moving object can be detected using the determined state of the moving object and the set risk detection criterion.

In various embodiments, the integrated monitoring module includes a camera sensor for generating image data including the surrounding environment of the mobile body as the surrounding environment of the mobile body is photographed, and the control server generates image data through the camera sensor. Information on the surrounding environment of the moving object by analyzing the obtained image data - The information on the surrounding environment of the moving object is information on the topography of the road in the area where the moving object is located, information on surrounding objects, and information on the weather. It is possible to extract - including -, generate surrounding environment information for the mobile body using the extracted information about the surrounding environment of the mobile body, and set a risk detection criterion for the mobile body using the generated surrounding environment information. there is.

In various embodiments, the control server generates surrounding environment information for the first moving object by using first image data including the surrounding environment of the first moving object, and sets a predetermined value around the location of the first moving object. When one or more second movable bodies are located within the range, one or more second image data including the surrounding environment of the one or more second movable bodies is collected from a camera sensor provided in the one or more second movable bodies, and the collected one or more second image data is collected. Based on the information about the surrounding environment of the one or more second moving objects extracted by analyzing the above second image data, the generated surrounding environment information for the first moving object may be corrected.

In various embodiments, the control server grids a predetermined area to generate a plurality of grids, and each of the plurality of grids generated from a plurality of integrated monitoring modules provided in a plurality of moving objects located in the predetermined area A plurality of sensor data for a location corresponding to is collected, and information about the surrounding environment extracted from the collected plurality of sensor data is recorded in each of the generated plurality of grids, thereby creating a grid map of the surrounding environment for the predetermined area. and loads information about the surrounding environment recorded in a grid corresponding to a point where the mobile body is located among a plurality of grids included in the generated grid map of the surrounding environment when it is desired to detect the occurrence of danger to the mobile body. Thus, a risk detection criterion for the moving object may be set.

In various embodiments, the control server records information about the surrounding environment extracted based on sensor data for a position corresponding to any one of the plurality of grids in the one grid, When two or more sensor data are collected from each of two or more movable bodies for a position corresponding to a lattice of , sensor data collected from a movable body located at the shortest distance from a position corresponding to any one of the lattices among the two or more movable bodies Only the information about the surrounding environment extracted based on the grid can be recorded in any one of the grids.

In various embodiments, the control server records information about the surrounding environment extracted based on sensor data for a position corresponding to any one of the plurality of grids in the one grid, When two or more sensor data are collected from each of two or more moving objects for a position corresponding to the lattice of , only the information about the surrounding environment extracted based on the last sensor data among the two or more collected sensor data is recalled. Can be written on either grid.

In various embodiments, the integrated monitoring module is provided at an arbitrary position of the moving body and includes an inertial measurement sensor including a gyro sensor, an acceleration sensor, and a geomagnetic sensor, and the control server collects data from the inertial measurement sensor. Using the 9 degree of freedom values for the moving object - the 9 degree of freedom values include XYZ-axis angular velocity values, XYZ-axis acceleration values, and XYZ-axis geomagnetism values - determining the posture of the moving object as a state of the moving object, Based on the determined posture of the moving object, the occurrence of danger to the moving object is detected, and accident data for each of a plurality of environments - The accident data for each of the plurality of environments is related to an accident that occurred in each of a plurality of environments including different conditions. Based on the information, an average posture value of the moving body when danger to the moving bodies is detected in each of the plurality of environments is calculated, and the calculated average posture value of the moving body is used in each of the plurality of environments. A plurality of corresponding danger detection criteria is defined, a danger detection criterion corresponding to the surrounding environment of the moving object is selected from among the defined plurality of risk detection criteria, and based on the selected risk detection criteria and the determined posture of the moving object It is possible to detect danger to the moving object.

In various embodiments, the control server sets a risk detection criterion for a specific area based on information about the surrounding environment of the specific area, and detects a predetermined number or more of the plurality of moving objects located in the specific area. When danger is detected, the danger detection level for the at least one second moving object may be raised by correcting the danger detection criteria for the set specific area.

In various embodiments, the control server corrects a risk detection criterion for at least one second movable object located within a predetermined range around the location of the first movable object when a danger to the first movable object is detected. Thus, the risk occurrence detection level for the at least one second moving object may be increased.

A method for detecting danger according to the surrounding environment of a mobile body according to another embodiment of the present invention for solving the above problems is a risk according to the surrounding environment of a mobile body including an integrated monitoring module and a control server for detecting the occurrence of danger to the mobile body. A method performed through an occurrence detection system, comprising: collecting sensor data corresponding to the moving object through the integrated monitoring module and detecting an occurrence of danger to the moving object using the collected sensor data; , In the step of detecting the occurrence of danger to the moving object, a risk detection criterion is set based on the surrounding environment of the moving object, and the state of the moving object determined based on the collected sensor data and the set risk detection criterion are used. and detecting the occurrence of danger to the moving object.

Other specific details of the invention are included in the detailed description and drawings.

According to various embodiments of the present invention, an operation of the moving object is determined using sensor data on the moving object collected using an integrated monitoring module installed in the moving object, and accordingly, the possibility of danger to the moving object is determined or whether or not danger has occurred. However, by determining the possibility of danger to the moving object or determining whether or not the danger occurs according to the surrounding environment of the moving object, it is possible to prevent the occurrence of danger to the moving object, and to create an environment in which a quick response is possible in case of danger to the moving object. There are advantages to building it.

In addition, by not only detecting the occurrence of danger to each of a plurality of moving objects, but also detecting the occurrence of danger to a plurality of moving objects located in a specific area, the risk to a plurality of moving objects (eg, multiple moving objects located in a specific area) It has the advantage of being able to provide an environment in which occurrence monitoring can be performed more efficiently.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

1 is a diagram illustrating a risk detection system according to the surrounding environment of a moving object according to an embodiment of the present invention.
2 is a diagram illustrating the configuration of an integrated monitoring module installed in a mobile body in various embodiments.
3 is a diagram illustrating an integrated monitoring module packaged as one module in various embodiments.
4 is a diagram exemplarily illustrating a form in which an integrated monitoring module is installed in a mobile body in various embodiments.
5 is a diagram illustrating a water level measurement sensor included in an integrated monitoring module in various embodiments.
6 is a diagram showing a form in which a water level measuring sensor is installed in a moving body by way of example.
7 is a diagram illustrating a warning notification output module included in an integrated monitoring module in various embodiments.
8 is a diagram illustrating a hardware configuration of a control server included in a system for detecting danger according to a surrounding environment of a moving object according to various embodiments.
9 is a flowchart of a method for detecting danger according to the surrounding environment of a moving object according to another embodiment of the present invention.
10 is a diagram exemplarily illustrating a Madgwick Filter algorithm applicable to various embodiments.
11 is a diagram for explaining a process of calculating the degree of leaning of a moving object in various embodiments.
12 is a flowchart illustrating a method of detecting danger to a moving object according to a surrounding environment of the moving object, in various embodiments.
13 to 56 exemplarily illustrate a user interface (UI) provided by a control server included in a system for detecting danger according to the surrounding environment of a moving object in various embodiments.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, only these embodiments are intended to complete the disclosure of the present invention, and are common in the art to which the present invention belongs. It is provided to fully inform the person skilled in the art of the scope of the invention, and the invention is only defined by the scope of the claims.

Terminology used herein is for describing the embodiments and is not intended to limit the present invention. In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase. As used herein, "comprises" and/or "comprising" does not exclude the presence or addition of one or more other elements other than the recited elements. Like reference numerals throughout the specification refer to like elements, and “and/or” includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various components, these components are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first element mentioned below may also be the second element within the technical spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used with meanings commonly understood by those skilled in the art to which the present invention belongs. In addition, terms defined in commonly used dictionaries are not interpreted ideally or excessively unless explicitly specifically defined.

The term "unit" or "module" used in the specification means a hardware component such as software, FPGA or ASIC, and "unit" or "module" performs certain roles. However, "unit" or "module" is not meant to be limited to software or hardware. A “unit” or “module” may be configured to reside in an addressable storage medium and may be configured to reproduce one or more processors. Thus, as an example, a “unit” or “module” may refer to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. Functions provided within components and "units" or "modules" may be combined into smaller numbers of components and "units" or "modules" or may be combined into additional components and "units" or "modules". can be further separated.

The spatially relative terms "below", "beneath", "lower", "above", "upper", etc. It can be used to easily describe a component's correlation with other components. Spatially relative terms should be understood as including different orientations of elements in use or operation in addition to the orientations shown in the drawings. For example, if you flip a component that is shown in a drawing, a component described as "below" or "beneath" another component will be placed "above" the other component. can Thus, the exemplary term “below” may include directions of both below and above. Components may also be oriented in other orientations, and thus spatially relative terms may be interpreted according to orientation.

In this specification, a computer means any kind of hardware device including at least one processor, and may be understood as encompassing a software configuration operating in a corresponding hardware device according to an embodiment. For example, a computer may be understood as including a smartphone, a tablet PC, a desktop computer, a laptop computer, and user clients and applications running on each device, but is not limited thereto.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Although each step described in this specification is described as being performed by a computer, the subject of each step is not limited thereto, and at least a part of each step may be performed in different devices according to embodiments.

1 is a diagram illustrating a risk detection system according to the surrounding environment of a moving object according to an embodiment of the present invention.

Referring to FIG. 1, a system for detecting danger according to the surrounding environment of a moving object according to an embodiment of the present invention includes a control server 100, an integrated monitoring module 200, a user terminal 300, and a network 400. can do.

Here, the danger occurrence detection system according to the surrounding environment of the moving object shown in FIG. 1 is according to an embodiment, and its components are not limited to the embodiment shown in FIG. 1, and can be added, changed, or deleted as necessary. can

In one embodiment, the control server 100 may detect danger to the plurality of moving objects 10 .

In various embodiments, the control server 100 may collect sensor data for each of the plurality of moving objects 10 from the integrated monitoring module 200 provided in each of the plurality of moving objects 10 located in a predetermined area, and , It is possible to determine the state of each of the plurality of movable objects 10 based on the collected sensor data, and to detect the occurrence of danger to the plurality of movable objects 10 based on the determined state.

At this time, the control server 100 sets the risk detection criteria for each of the plurality of moving objects 10 based on the surrounding environment of each of the plurality of moving objects 10, and sets the risk detection criteria for each of the plurality of moving objects 10 and It is possible to detect the occurrence of danger for each of the plurality of movable bodies 10 by using the state of each of the plurality of movable bodies 10 .

Here, the state of the moving object 10 determined by the control server 100 is the motion of the moving object 10 determined based on the inertial data (eg, the movement, attitude, movement speed and position of the moving object 10, etc.) and It may include, but is not limited to, the water level and driving stability of the mobile body 10 determined based on the water level data, and the state of the mobile body 10 may be considered to detect danger to the mobile body 10. It can be applied in any condition.

In addition, although the state determination operation of the moving object 10 is described here as being performed by the control server 100 based on sensor data collected from the integrated monitoring module 200, it is not limited thereto, and the integrated monitoring module 200 ) The state determination operation of the mobile body 10 is performed through the computing device 210 provided in the control server 100, and result data derived as the state determination operation of the mobile body 10 is performed from the computing device 210 is performed. ) can be implemented in the form of collection.

In one embodiment, the integrated monitoring module 200 may be provided and installed on at least a portion of the mobile body 10 and may collect sensor data of the mobile body 10 . To this end, the integrated monitoring module 200 may include a computing device 210, a sensor module 220, a power source 230, a communication module 240, and a warning notification output module 250. However, it is not limited thereto.

Here, in the integrated monitoring module 200, components included in the integrated monitoring module 200 may be packaged into one module, as shown in FIG. 3, in consideration of convenience of installation and management of the mobile body 10. And, as shown in FIG. 4, it may be installed at any position of the movable body 10, but is not limited thereto.

More specifically, referring to FIG. 2 , first, the computing device 210 may control operations of various components included in the integrated monitoring module 200 . For example, the computing device 210 may be a micro controller unit (MCU), but is not limited thereto.

Here, the sensor data for the moving object 10 is data measured through the sensor module 220 to be described later, for example, inertial data, water level data, position data, temperature data, humidity data, and image data for the moving object 10. It may include, but is not limited to.

Next, the sensor module 220 may collect sensor data for the moving object 10 . To this end, the sensor module 220 may include an inertial measurement sensor 221, a water level measurement sensor 222, a position sensor 223, a temperature/humidity sensor 224, and a camera sensor 225, but is limited thereto. It doesn't work.

Here, a plurality of sensors included in the sensor module 220 may be packaged and installed as one set. At this time, only at least one sensor among a plurality of sensors may be selectively packaged according to the type of the movable body 10 . For example, when the moving object 10 is a tractor, an inertial measurement sensor 221, a position sensor 223, a temperature/humidity sensor 224, and a camera sensor 225 among a plurality of sensors are packaged as one set for integrated monitoring. It can be built into the module 200. In addition, when the moving object 10 is a control vehicle, the inertial measurement sensor 221, the water level measurement sensor 222, the position sensor 223, the temperature/humidity sensor 224, and the camera sensor 225 are set as one set. It may be packaged and embedded in the integrated monitoring module 200.

In addition, in some cases, when the water level measurement sensor 222 is implemented in the form shown in FIG. 5 , other sensors except for the water level measurement sensor 222 (eg, the inertial measurement sensor 221, the position sensor 223, The temperature/humidity sensor 224 and the camera sensor 225) may be packaged as a set and embedded in the integrated monitoring module 200, and the water level measurement sensor 222 may be an independent module from the integrated monitoring module 200. It can be implemented and connected to the integrated monitoring module 200 through short-range wireless communication (eg, Bluetooth). However, it is not limited thereto.

As described above, when the water level measurement sensor 222 is implemented independently of the integrated monitoring module 200, the water level module 222C for collecting water level data based on the water level value measured from the water level measurement sensor 222 is further provided. The water level module 222C and the integrated monitoring module 200 may be connected through short-range wireless communication and the water level data collected through the water level module 222C may be transmitted to the integrated monitoring module 200. .

The inertial measurement sensor 221 may generate inertial data for the moving object 10 . For example, the inertial measurement sensor 221 may be a 9-axis inertial measurement sensor including a gyro sensor 221A, an acceleration sensor 221B, and a geomagnetic sensor 221C, and the gyro sensor 221A, the acceleration sensor 221B and Inertial data for the moving object 10 may be measured using the geomagnetic sensor 221C.

Here, the inertial data for the moving object 10 may be values of 9 degrees of freedom measured through a 9-axis inertial measurement sensor. For example, the inertial data for the moving object 10 is a 3-axis (XYZ-axis) angular velocity value measured using the gyro sensor 221A, a 3-axis (XYZ-axis) acceleration value measured using the acceleration sensor 221B, and geomagnetism It may include, but is not limited to, the 3-axis (XYZ-axis) geomagnetic value measured using the sensor 221C, and the value generated by processing the 9-degree-of-freedom value (eg, obtained by integrating the XYZ-axis angular velocity angular value, a velocity value obtained by integrating the XYZ-axis acceleration, a position change amount obtained by integrating the XYZ-axis velocity value, etc.) may be further included.

The water level measurement sensor 222 may be installed in the mobile body 10 (eg, a control vehicle loaded with a control solution) for loading the liquid material, and the water level for the liquid material loaded in the mobile body 10 data can be generated. For example, the water level sensor 222 may be installed in a control vehicle loaded with a control solution, measure a level value for the control solution loaded in the control vehicle, and generate level data including the measured level value. there is.

In various embodiments, the water level measurement sensor 222 may measure a water level value for the control solution loaded on the moving object 10 using an electrostatic method. To this end, the water level measuring sensor 222 may include a tube tube 222A and a plurality of capacitive sensors 222B, as shown in FIG. 5 .

The tube pipe (222A) is provided on the movable body 10 and is connected to the control solution tank for loading the control solution, so that the control solution loaded in the control solution tank may be introduced.

At this time, the tube pipe (222A) has one side (eg, the lower end of the tube pipe (222A)) connected to the lower end of the control solution tank, and the other side (eg, the upper end of the tube pipe (222A)) is connected to the upper end of the control solution tank Depending on, it may be implemented so that the control solution loaded in the control solution tank is introduced to the same height as the water level of the control solution loaded in the control solution tank, but is not limited thereto.

A plurality of capacitive sensors (222B) may be provided on the outer surface of the tube tube (222A), it is possible to detect the control solution flowing into the tube tube (222A) through an electrostatic method. For example, the plurality of capacitive sensors 222B may measure the capacitance inside the tube tube 222A, and the capacitance value inside the tube tube 222A fluctuates as the control solution flows into the tube tube 222A. By detecting that, it is possible to detect the control solution flowing into the tube tube (222A). To this end, in order to measure the capacitance inside the tube tube (222A), each of the plurality of capacitive sensors (222B) may be composed of two capacitive sensors as a pair, and the two capacitive sensors are the tube tube ( 222A), the capacitance value inside the tube 222A can be measured by being arranged facing each other at the same height. As an example, the plurality of capacitive sensors 222B may be capacitive touch sensors, but are not limited thereto.

Here, the plurality of capacitive sensors 222B are disposed on the outer surface of the tube tube 222A, and the position, number and number of the plurality of capacitive sensors 222B are disposed on the outer surface of the tube tube 222A. The arrangement interval between the electrostatic sensors 222B is at least one of the length of the tube 222A, the capacity of the control solution tank provided in the moving body 10, the weight of the moving body 10, and the average water level level for the moving body 10. can be determined based on one

In one example, the number of the plurality of capacitive sensors (222B) may be determined in proportion to the weight of the length of the tube tube (222A), the capacity of the control solution tank and the mobile body (10).

As another example, an average water level level for the moving object 10 is calculated based on sensor data collected for a predetermined period of time, and the capacitive sensor 222B is provided at every first interval within a predetermined range around the calculated average water level level. , and the capacitive sensor 222B may be disposed at every second interval wider than the first interval except for a predetermined range.

As another example, an average water level for the movable body 10 is calculated based on sensor data collected for a predetermined period of time, and the position of the tube tube 222A corresponding to the calculated average water level level is based on one blackout. An expression sensor 222B may be disposed, and the remaining capacitive sensors 222B are arranged at predetermined intervals in the vertical direction centering on the capacitive sensor 222B as a reference, but the capacitive sensor 222B as a reference ), that is, the distance disposed in proportion to the distance from the position of the tube pipe 222A corresponding to the average water level level may be set wide.

As another example, for the purpose of detecting danger according to the water level of the moving object 10, the center of the position of the tube pipe 222A corresponding to the water level that is a criterion for detecting danger (risk determination standard) Thus, the capacitive sensor 222B may be disposed at every first interval within a predetermined range, and the capacitive sensor 222B may be disposed at every second interval wider than the first interval outside the predetermined range.

However, the arrangement position, number, and interval of the plurality of capacitive sensors 222B are not limited thereto, and may be freely set and arranged according to a user's request using the integrated monitoring module 200 .

The position sensor 223 may generate position data of the moving object 10 . For example, the location sensor 223 may be a GPS sensor, and may generate location data including information about location coordinates by measuring location coordinates of the moving object 10 through the GPS sensor, but is not limited thereto. .

Here, it is described that the position sensor 223 is included in the integrated monitoring module 200 to generate position data for the moving object 10, but is not limited thereto, and the integrated monitoring module 200 is a separate position sensor ( 223), it can be connected to the user terminal 300 of the occupant on the mobile body 10 through the network 400, and is generated through a location sensor (eg, GPS) provided in the user terminal 300. It can be implemented in the form of collecting location data.

The temperature/humidity sensor 224 may measure temperature and humidity data of the mobile body 10 or an environment surrounding the mobile body 10 .

The camera sensor 225 may generate image data about the moving object 10 or the moving direction of the moving object 10 by capturing the moving object 10 or the moving direction of the moving object 10 . For example, the camera sensor 225 may be a black box installed on the mobile body 10, but is not limited thereto, and the integrated monitoring module 200 does not include a separate camera sensor 225 and is installed on the mobile body 10. By using the camera module provided in the user terminal 300 of the occupant on board, the moving object 10 or the moving direction of the moving object 10 is photographed to generate image data for the moving object 10 or the moving direction of the moving object 10. can do.

Next, the power source 230 includes components included in the integrated monitoring module 200 (eg, the computing device 210, the sensor module 220, the communication module 240, and the warning notification output module 250) can supply power for driving. For example, the power source 230 may be a battery provided inside the integrated monitoring module 200, but is not limited thereto.

Next, the communication module 240 may be connected to an external device/device (eg, the user terminal 300 and the control server 100) of the integrated monitoring module 200 through the network 400, and the sensor module ( 220) may be provided to an external device/device. For example, the communication module 240 may be an LTE module, but is not limited thereto.

Here, the user terminal 300 is a wireless communication device that ensures portability and mobility, and includes navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System) , PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminal, smartphone ( It may include all types of handheld-based wireless communication devices such as Smartphone, Smartpad, Tablet PC, etc., but is not limited thereto, and the infotainment system provided in the mobile body 10 can be

Also, here, the network 400 may refer to a connection structure capable of exchanging information between nodes such as a plurality of terminals and servers. For example, the network 400 includes a local area network (LAN), a wide area network (WAN), a world wide web (WWW), a wired and wireless data communication network, a telephone network, a wired and wireless television communication network, and the like. can do.

In addition, here, the wireless data communication networks are 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), 5GPP (5th Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi (Wi-Fi) Fi), Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), RF (Radio Frequency), Bluetooth network, A near-field communication (NFC) network, a satellite broadcasting network, an analog broadcasting network, a digital multimedia broadcasting (DMB) network, etc. may be included, but is not limited thereto.

Next, the warning notification output module 250 may output a warning notification in response to detection of danger to the moving object 10 .

For example, the control server 100 may detect the occurrence of danger to the moving object 10 based on information about the state of the moving object 10, and based on whether or not the occurrence of danger to the moving object 10 is detected, If there is a high possibility of danger to the moving object 10 or it is determined that danger has occurred to the moving object 10, the warning notification output module 250 through the computing device 210 included in the integrated monitoring module 200 A control command for controlling the operation of the controller may be output so that a warning notification is output. Here, the warning notification output module 250 may include a warning light as shown in FIG. 7 , but is not limited thereto, and the warning notification output module 250 further includes a speaker for outputting a warning notification in the form of a voice. can include In addition, the integrated monitoring module 200 does not have a separate warning notification output module 250, and based on the result of determining the danger to the moving object 10, there is a high possibility that danger will occur in the moving object 10 or the moving object When it is determined that danger has occurred in (10), it may be provided to the user terminal 300 so that a warning notification is output through the user terminal 300. Hereinafter, with reference to FIG. 8 , the hardware configuration of the control server 100 that performs a method for detecting danger according to the surrounding environment of a moving object will be described.

8 is a diagram illustrating a hardware configuration of a control server included in a system for detecting danger according to a surrounding environment of a moving object according to various embodiments.

Referring to FIG. 8, in various embodiments, the control server 100 includes one or more processors 110, a memory 120 for loading a computer program 151 executed by the processor 110, and a bus ( 130), a communication interface 140, and a storage 150 for storing the computer program 151. Here, only the components related to the embodiment of the present invention are shown in FIG. 8 . Accordingly, those skilled in the art to which the present invention pertains can know that other general-purpose components may be further included in addition to the components shown in FIG. 8 .

The processor 110 controls the overall operation of each component of the control server 100 . The processor 110 includes a Central Processing Unit (CPU), a Micro Processor Unit (MPU), a Micro Controller Unit (MCU), a Graphic Processing Unit (GPU), or any type of processor well known in the art of the present invention. It can be.

Also, the processor 110 may perform an operation for at least one application or program for executing a method according to embodiments of the present invention, and the control server 100 may include one or more processors.

In various embodiments, the processor 110 may temporarily and/or permanently store signals (or data) processed in the processor 110 (RAM: Random Access Memory, not shown) and ROM (ROM: Read -Only Memory, not shown) may be further included. In addition, the processor 110 may be implemented in the form of a system on chip (SoC) including at least one of a graphics processing unit, RAM, and ROM.

Memory 120 stores various data, commands and/or information. Memory 120 may load computer program 151 from storage 150 to execute methods/operations according to various embodiments of the present invention. When the computer program 151 is loaded into the memory 120, the processor 110 may perform the method/operation by executing one or more instructions constituting the computer program 151. The memory 120 may be implemented as a volatile memory such as RAM, but the technical scope of the present disclosure is not limited thereto.

The bus 130 provides a communication function between components of the control server 100 . The bus 130 may be implemented in various types of buses such as an address bus, a data bus, and a control bus.

The communication interface 140 supports wired and wireless Internet communication of the control server 100 . Also, the communication interface 140 may support various communication methods other than Internet communication. To this end, the communication interface 140 may include a communication module well known in the art. In some embodiments, communication interface 140 may be omitted.

The storage 150 may non-temporarily store the computer program 151 . When a danger detection process according to the surrounding environment of the mobile body is performed through the control server 100, the storage 150 may store various types of information necessary to provide the danger detection process according to the surrounding environment of the mobile body.

The storage 150 may be a non-volatile memory such as read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, or the like, a hard disk, a removable disk, or a device well known in the art. It may be configured to include any known type of computer-readable recording medium.

Computer program 151 may include one or more instructions that when loaded into memory 120 cause processor 110 to perform methods/operations in accordance with various embodiments of the invention. That is, the processor 110 may perform the method/operation according to various embodiments of the present disclosure by executing the one or more instructions.

In one embodiment, the computer program 151 collects sensor data corresponding to the moving object through an integrated monitoring module and uses the collected sensor data to detect an occurrence of danger to the moving object. It may include one or more instructions for performing a risk occurrence detection method according to.

Steps of a method or algorithm described in connection with an embodiment of the present invention may be implemented directly in hardware, implemented in a software module executed by hardware, or implemented by a combination thereof. A software module may include random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any form of computer readable recording medium well known in the art to which the present invention pertains.

Components of the present invention may be implemented as a program (or application) to be executed in combination with a computer, which is hardware, and stored in a medium. Components of the present invention may be implemented as software programming or software elements, and similarly, embodiments may include various algorithms implemented as data structures, processes, routines, or combinations of other programming constructs, such as C, C++ , Java (Java), can be implemented in a programming or scripting language such as assembler (assembler). Functional aspects may be implemented in an algorithm running on one or more processors. Hereinafter, with reference to FIGS. 9 to 12 , a method of detecting danger according to the surrounding environment of a moving object performed by the control server 100 will be described.

9 is a flowchart of a method for detecting danger according to the surrounding environment of a moving object according to another embodiment of the present invention.

Referring to FIG. 9 , in step S110 , the control server 100 may collect sensor data for the moving object 10 . For example, the control server 100 may be connected to the integrated monitoring module 200 provided in the mobile body 10 through the network 400 and may collect sensor data measured through the integrated monitoring module 200. there is.

In various embodiments, the control server 100 may collect inertial data on the moving object 10 as sensor data on the moving object 10 .

More specifically, the control server 100 outputs a control command for controlling the inertial measurement sensor 221 among a plurality of sensors included in the sensor module 220 of the integrated monitoring module 200, so that the inertial measurement sensor 221 The gyro sensor 221A, acceleration sensor 121B, and geomagnetic sensor 221C included in may operate to measure inertial data of the moving object 10, and the inertial measurement sensor 221 may measure the moving object 10. ) of inertial data can be collected.

Here, the inertial data of the moving object 10 collected from the inertial measurement sensor 221 may be 9 degrees of freedom values including XYZ-axis angular velocity values, XYZ-axis acceleration values, and XYZ-axis geomagnetism values, but is not limited thereto, and 9 degrees of freedom Inertial information (e.g., angle data obtained by integrating XYZ-axis angular velocity, velocity data obtained by integrating XYZ-axis acceleration, position data obtained by integrating velocity data, etc.) calculated by processing degree values can include more.

In various embodiments, the control server 100 may collect water level data of the mobile body 10 as sensor data of the mobile body 10 .

More specifically, first, the control server 100 outputs a control command for controlling the water level measuring sensor 222 among the plurality of sensors included in the sensor module 20 of the integrated monitoring module 20, so that the water level measuring sensor ( 222) can be controlled to detect the control solution introduced into the tube tube 222A through a plurality of capacitive sensors 222B included in the tube tube 222A, each of the plurality of capacitive sensors 222B flows into the tube tube 222A Capacitance values measured by sensing the sprayed control solution can be collected.

Thereafter, the control server 100 may select at least one capacitive sensor 222B having a changed capacitance value from among the plurality of capacitive sensors 222B, and the tube tube tube among the selected at least one capacitive sensor 222B. The capacitive sensor 222B disposed at the highest position of 222A may be selected, the height value of the selected capacitive sensor 222B may be extracted as the water level value of the mobile body 10, and the extracted water level value It is possible to collect water level data including information about

In various embodiments, the control server 100 may collect sensor data for the moving object 10 at predetermined intervals.

For example, the control server 100 may detect the start of the operation of the specific moving object 10, and when it is determined that the operation of the specific moving object 10 has started, the control server 100 may set a preset starting time point when the operation of the specific moving object 10 is started. By outputting a control command for controlling the operation of the integrated monitoring module 200 provided in the specific moving body 10 at each period, the sensor for the specific moving body 10 is set every predetermined period from the time when the operation of the specific moving body 10 is started. data can be collected.

As another example, the integrated monitoring module 200 provided in the specific mobile unit 10 may be connected to the user terminal 300 based on short-range wireless communication (eg, Bluetooth (BLE), etc.), and received from the user terminal 300. It is possible to recognize that the user terminal 300 enters within a predetermined range based on the specific mobile body 10 based on the signal strength, and the control server 100 determines that the user terminal 300 moves within a specific mobile body 10. By outputting a control command for controlling the operation of the integrated monitoring module 200 provided in the specific mobile body 10 at each predetermined period from the time when it is determined that the mobile body 300 has entered the predetermined range based on Based on (10), sensor data for the moving object 10 may be collected at predetermined intervals from the time when it is determined that the sensor has entered within a predetermined range.

Here, the preset period is a period in which the sensor module 220 measures sensor data, and may be a value defined in advance, but is not limited thereto. Also, here, the predetermined period (sensor data measurement period) may be individually set for each of the plurality of moving objects 10 .

In addition, a predetermined period (sensor data measurement period) may be individually set for each of a plurality of sensors included in one integrated monitoring module 200 . For example, a period of measuring inertial data through the inertial measurement sensor 221 included in the sensor module 220 may be set shorter than a period of measuring water level data through the water level sensor 222, but is not limited thereto. .

In various embodiments, the control server 100 may calculate an accident risk for each of a plurality of regions based on at least one of accident data, weather data, and moving vehicle usage data of each of a plurality of regions, and the calculated accident risk Based on this, it is possible to determine a sensor data collection period for each of a plurality of regions, and sensor data can be collected from the plurality of integrated monitoring modules 200 based on the determined sensor data collection period.

More specifically, first, the control server 100 may define in advance first score data for each number of accidents, second score data for each weather, and third score data for each usage of the moving object 10 .

Thereafter, the control server 100 extracts a first score according to the number of accidents based on the accident data for a specific area and the first score data for each number of accidents, and calculates the weather data for a specific area and the second score data for each weather. Based on this, a second score according to the weather may be extracted, and a third score according to the usage of the moving object 10 may be extracted based on the moving object usage data and the third score data for each moving object usage.

Thereafter, the control server 100 may calculate the accident risk for a specific area by calculating the sum of the first score, the second score, and the third score, and by setting a period having a size inversely proportional to the calculated accident risk, It may be implemented such that sensor data is collected at shorter intervals as the risk of accidents increases, and sensor data is collected at longer intervals as the risk of accidents decreases. However, it is not limited thereto.

In step S120, the control server 100 may determine the state of the moving object 10 using sensor data on the moving object 10 collected through step S110.

In various embodiments, the control server 100 may determine an operation of the moving object 10 as a state of the moving object 10 using inertial data of the moving object 10 . Here, the operation of the movable body 10 may include the movement and posture of the movable body, but is not limited thereto.

In various embodiments, the control server 100 calculates the Euler Angle and Quaternion for the moving object 10 based on the value of the 9 degrees of freedom for the moving object 10, and the Euler angle and the quaternion. It is possible to determine the posture of the moving body 10 as a state of the moving body 10 by using.

Here, a method for calculating the Euler angle and quaternion for the moving object 10 based on inertial data (9 degree of freedom values) for the moving object 10 and estimating the posture of the moving object 10 based on the Euler angle and quaternion And various techniques are known for the method of determining, and these known techniques can be selectively applied. In this specification, based on a specific method for calculating the Euler angle and quaternion for the moving object 10 and the Euler angle and quaternion However, the specific method for estimating and determining the posture of the moving object 10 is not limited.

The method of determining the posture of the movable body 10 based on Euler angles and quaternions is not to obtain a fixed angle of the Z-Y-X axis based on a fixed frame, but continues rotation based on a frame that fluctuates every rotation without a fixed frame. This is a method of obtaining a reversely changed angle while going, which is a method capable of estimating the posture of the moving body 10 without distinguishing the front, rear, left and right directions of the moving body 10 . Therefore, the integrated monitoring module 200 including the inertial measurement sensor 121 can determine the posture of the moving object 10 regardless of where and in which direction it is installed, and thus the installation convenience of the integrated monitoring module 200 It has the advantage of increasing degrees of freedom.

In various embodiments, the control server 100 determines the posture of the moving object 10 based on the 9 degrees of freedom value, which is inertial data for the moving object 10, but uses a Madgwick Filter algorithm (eg, FIG. 10 ). ), it is possible to filter noise such as acceleration shaking from the inertial data of the moving object 10 by correcting the 9 degree of freedom value for the moving object 10 based on There is an advantage in that more accurate results can be derived by determining the posture of the moving object 10 using inertial data.

In various embodiments, the control server 100 may calculate the degree of leaning of the moving object 10 in a specific direction as a state of the moving object 10 when the moving object 10 includes a plurality of wheels.

Generally, in the case of a tractor, the front wheel is smaller than the rear wheel, the body is high, and the weight is heavy, so it has a high center of gravity. Accordingly, even if the tractor is slightly tilted in a certain direction, it can easily overturn, roll over, or fall. There is a problem with being able to.

In consideration of this point, the control server 100 determines the degree of leaning of the tractor in a specific direction as a state of the tractor, thereby more accurately determining the danger and risk of overturning, overturning, and falling accidents of the tractor can do.

More specifically, referring to FIG. 11, first, when the moving body 10 (eg, a tractor) includes a plurality of wheels, the control server 100 corresponds to the moving body 10 centered on each of the plurality of wheels. The XY plane (a plane horizontal to the moving direction of the moving body 10) can be divided into a plurality of regions. For example, when the moving body 10 includes four wheels, the control server 100 may divide the XY plane corresponding to the moving body 10 into four areas centered on each of the four wheels.

Thereafter, the control server 100 may calculate the center of gravity of the moving object 10 and the inclination of each of the plurality of wheels, and may calculate the degree of leaning toward each of the plurality of areas using the calculated inclination.

In various embodiments, the control server 100 may calculate an angle corresponding to the moving direction of the moving object 10 as a state of the moving object 10 .

Typically, in evaluating the driving stability of the mobile body 10, the angle corresponding to the traveling direction of the mobile body 10 may act as the most important parameter. In particular, in the case of a tractor, unlike a general vehicle, the traveling direction Corresponding angles and overturning, overturning and falling accidents for tractors have a high relevance.

In consideration of this point, the control server 100, when the moving object 10 is a tractor, calculates the angle with respect to the moving direction of the tractor, thereby more accurately determining the risk of overturning, overturning, and falling of the tractor and whether or not the risk occurs. In addition to being able to build an environment that can be judged, there is an advantage in that the system can be lightweight and the response speed of the system can be improved by calculating only the angle for the direction of travel without determining the attitude of the tractor as a state of the tractor. .

More specifically, first, the control server 100 may determine the moving direction of the moving object 10 .

For example, the control server 100 may collect location data of the moving object 10 for a predetermined period of time, and determine the moving direction of the moving object 10 using the location data collected for a predetermined period of time.

As another example, the control server 100 may collect 9 degree-of-freedom values that are inertial data of the moving object 10 for a predetermined period of time, and the moving object 10 progresses using the 9 degree-of-freedom values collected during the predetermined period of time. direction can be determined. For example, the control server 100 may integrate the 3-axis (XYZ-axis) acceleration values collected during a predetermined period to obtain speed values in the 3-axis direction, and integrate the speed values in the 3-axis direction to obtain the 3-axis acceleration values. The amount of change in position in the direction may be obtained, and the direction having the largest amount of change in position based on the amount of change in position in the three-axis direction may be determined as the moving direction of the movable body 10 .

Thereafter, the control server 100 may calculate an angle corresponding to the moving direction of the moving object 10 by using the value of the 9 degrees of freedom for the moving object 10 . For example, when it is determined that the traveling direction of the moving object 10 is in the X-axis direction, the control server 100 may calculate an angle in the X-axis direction using an X-axis angular velocity value among nine degrees of freedom.

In various embodiments, the control server 100 determines the attitude of the moving object 10 using the inertial data of the moving object 10, but based on the attitude of the user terminal 200 of the occupant on the moving object 10 , the determined attitude of the movable body 10 can be verified.

More specifically, the control server 100 may be connected to the user terminal 300 (eg, a smart phone) through the network 400, and the user terminal 300 may be connected to the user from a sensor (eg, an inertial measurement sensor) provided in the user terminal 300. Inertial data for the terminal 300 (eg, 9 degree of freedom values for the user terminal 300) may be collected.

Thereafter, the control server 100 may determine the posture of the user terminal 300 based on the inertial data of the user terminal 200, and compare the posture of the user terminal 300 with the posture of the mobile body 10. , verification of the attitude of the moving object 10 can be performed.

For example, the control server 100 determines the angle corresponding to the moving direction of the moving object 10 determined using the inertial data of the moving object 10 and the user terminal 300 determined using the inertial data of the user terminal 300 When the angle corresponding to the moving direction of the moving object has a difference within an error range, it can be determined that the angle corresponding to the moving direction of the moving object 10 is accurately calculated using the inertial data of the moving object 10 .

In various embodiments, the control server 100 may use the water level data to determine the level of the control solution loaded on the moving object 10. For example, water level data according to water level values may be defined in advance in the control server 100, and based on the water level data according to the water level values, the water level values included in the water level data collected from the water level measurement sensor 222 The water level corresponding to can be determined as the water level level for the control solution loaded on the moving body (10). However, it is not limited thereto.

In various embodiments, the control server 100 may determine the stability of the operation of the moving object 10 as a state of the moving object 10 based on the water level of the disaster prevention solution loaded on the moving object 10. .

In various embodiments, the control server 100 may determine the stability of the operation of the moving object 10 using Equation 1 below.

<Equation 1>

Here, the loading amount of the pesticide solution according to the water level is provided in the mobile body 10, and based on the capacity of the disaster prevention solution tank for loading the pesticide solution and the water level determined for the mobile body 10, indirectly according to the water level It may mean the load of the calculated control capacity, and the normalized weight of the mobile body 10 may mean a value obtained by normalizing the weight of the mobile body 10 to a value within a predetermined range (eg, a value within the range of 0 to 1). However, it is not limited thereto.

The weight (normal drag) of the moving body 10 is a parameter that maintains the running track of the moving body 10, and the loading amount (water level) of the control capacity is a parameter that leaves the running track of the moving body 10. The main cause of the movable body 10 loaded with the pesticide solution leaving the running track is when the centrifugal force created by the pesticide solution is higher than the frictional force of the vehicle maintaining the running track.

Therefore, the control server 100 determines the stability of the operation of the moving object 10 based on the water level of the control solution corresponding to the weight of the control solution and the weight of the moving object 10, thereby overturning, overturning and falling the control vehicle. It is possible to build an environment that can more accurately determine the risk of accidents and whether or not they occur.

In step S130, the control server 100 may detect danger to the moving object 10 based on the state of the moving object 10 determined through step S120. For example, the control server 100 may classify the moving object 10 into a safe state, a caution state, a dangerous state, or an accident state based on the state of the moving object 10, but is not limited thereto.

In various embodiments, the control server 100 may set a risk detection criterion based on the surrounding environment of the moving object 10, and the occurrence of danger to the moving object 10 based on the state of the moving object 10 and the risk detection criterion. can detect Hereinafter, it will be described in more detail with reference to FIG. 12 .

12 is a flowchart illustrating a method of detecting danger to a moving object according to a surrounding environment of the moving object, in various embodiments.

Referring to FIG. 12 , in step S210 , the control server 100 may generate surrounding environment information for the moving object 10 .

In various embodiments, the control server 100 may generate surrounding environment information about the moving object 10 based on image data obtained by capturing the surrounding environment of the moving object 10 .

More specifically, the control server 100 may collect image data generated as the surrounding environment of the mobile body 10 is photographed through the camera sensor 225 provided in the mobile body 10, and the collected image data It is possible to extract information about the surrounding environment of the moving object 10 through analysis, and generate information about the surrounding environment of the moving object 10 using the extracted information about the surrounding environment.

In various embodiments, the control server 100 may extract information about the surrounding environment of the moving object 10 by analyzing image data including the surrounding environment of the moving object 10 using a pre-learned image analysis model. there is.

Here, the pre-learned image analysis model is a model learned according to a supervised learning method using a plurality of image data labeled with information about the surrounding environment as learning data, and using specific image data as input data to obtain result data As , it may be a model for extracting information about the surrounding environment for specific image data.

In addition, here, the information on the surrounding environment is information on the topography of the road in the area where the moving object 10 is located (eg, whether it is a paved/unpaved road, width, curvature, angle of inclination of the road, etc.), information on surrounding objects Information (eg, dynamic/static objects located around the moving object 10 (eg, obstacles, other moving objects, pedestrians, etc.)) and weather information may be included, but are not limited thereto.

An image analysis model (e.g., a neural network) is composed of one or more network functions, and one or more network functions may be composed of a set of interconnected computational units, which may be generally referred to as 'nodes'. These 'nodes' may also be referred to as 'neurons'. One or more network functions include at least one or more nodes. Nodes (or neurons) that make up one or more network functions can be interconnected by one or more 'links'.

Within the image analysis model, one or more nodes connected through a link may form a relative relationship of an input node and an output node. The concept of an input node and an output node is relative, and any node in an output node relationship with one node may have an input node relationship with another node, and vice versa. As described above, the input node to output node relationship can be created around the link. More than one output node can be connected to one input node through a link, and vice versa.

In a relationship between an input node and an output node connected through one link, the value of the output node may be determined based on data input to the input node. Here, a node interconnecting an input node and an output node may have a weight. The weight may be variable, and may be variable by a user or an algorithm in order to perform a function desired by the image analysis model. For example, when one or more input nodes are interconnected by respective links to one output node, the output node is set to a link corresponding to values input to input nodes connected to the output node and respective input nodes. An output node value may be determined based on the weight.

As described above, in the video analysis model, one or more nodes are interconnected through one or more links to form an input node and output node relationship in the video analysis model. Characteristics of the image analysis model may be determined according to the number of nodes and links in the image analysis model, a relationship between the nodes and links, and a weight value assigned to each link. For example, when there are two image analysis models having the same number of nodes and links and different weight values between the links, the two image analysis models may be recognized as different from each other.

Some of the nodes constituting the image analysis model may form one layer based on distances from the first input node. For example, a set of nodes having a distance of n from the first input node may constitute n layers. The distance from the first input node may be defined by the minimum number of links that must be passed through to reach the corresponding node from the first input node. However, the definition of such a layer is arbitrary for explanation, and the order of a layer in an image analysis model may be defined in a method different from that described above. For example, a layer of nodes may be defined by a distance from a final output node.

An initial input node may refer to one or more nodes to which data is directly input without going through a link in relation to other nodes among nodes in the image analysis model. Alternatively, in a relationship between nodes based on a link in an image analysis model network, it may mean nodes that do not have other input nodes connected by a link. Similarly, the final output node may refer to one or more nodes that do not have an output node in relation to other nodes among nodes in the image analysis model. Also, the hidden node may refer to nodes constituting an image analysis model other than the first input node and the last output node. An image analysis model according to an embodiment of the present invention may have more nodes of an input layer than nodes of a hidden layer close to an output layer, and may be an image analysis model in which the number of nodes decreases as the node progresses from the input layer to the hidden layer. can

An image analysis model may include one or more hidden layers. A hidden node of a hidden layer may use outputs of previous layers and outputs of neighboring hidden nodes as inputs. The number of hidden nodes for each hidden layer may be the same or different. The number of nodes of the input layer may be determined based on the number of data fields of the input data and may be the same as or different from the number of hidden nodes. Input data input to the input layer may be operated by a hidden node of the hidden layer and may be output by a fully connected layer (FCL) that is an output layer.

In various embodiments, the image analysis model may be a deep learning model.

A deep learning model (e.g., a deep neural network (DNN)) may refer to an image analysis model including a plurality of hidden layers in addition to an input layer and an output layer. It can identify latent structures, i.e., the latent structures of a photograph, text, video, audio, or music (e.g., what objects are in the photo; content and emotions, etc.).

Deep neural networks include convolutional neural networks (CNN), recurrent neural networks (RNNs), auto encoders, generative adversarial networks (GANs), and restricted Boltzmann machines (RBMs). boltzmann machine), deep belief network (DBN), Q network, U network, Siamese network, etc., but is not limited thereto.

In various embodiments, the network function may include an autoencoder. Here, the auto encoder may be a type of artificial neural network for outputting output data similar to input data.

An auto-encoder may include at least one hidden layer, and an odd number of hidden layers may be disposed between input and output layers. The number of nodes of each layer may be reduced from the number of nodes of the input layer to an intermediate layer called the bottleneck layer (encoding), and then expanded symmetrically with the reduction from the bottleneck layer to the output layer (symmetrical to the input layer). Nodes of the dimensionality reduction layer and the dimensionality restoration layer may or may not be symmetrical. Also, an autoencoder can perform non-linear dimensionality reduction. The number of input layers and output layers may correspond to the number of remaining sensors after preprocessing of input data. In the auto-encoder structure, the number of hidden layer nodes included in the encoder may decrease as the distance from the input layer increases. If the number of nodes in the bottleneck layer (the layer with the fewest nodes located between the encoder and decoder) is too small, a sufficient amount of information may not be conveyed, so more than a certain number (e.g., more than half of the input layer, etc.) ) may be maintained.

The neural network may be trained using at least one of supervised learning, unsupervised learning, and semi-supervised learning. The learning of neural networks is to minimize errors in the output. More specifically, learning of the neural network repeatedly inputs training data to the neural network, calculates the output of the neural network for the training data and the error of the target, and converts the error of the neural network to the output of the neural network in a direction to reduce the error. This is the process of updating the weight of each node of the neural network by performing backpropagation from the layer to the input layer.

First, in the case of teacher learning, each learning data is labeled with the correct answer (ie, labeled learning data), and in the case of comparative teacher learning, the correct answer may not be labeled in each learning data. That is, for example, learning data in the case of teacher learning regarding data classification may be data in which each learning data is labeled with a category. Labeled training data is input to the neural network, and an error may be calculated by comparing an output (category) of the neural network and a label of the training data.

Next, in the case of comparative history learning for data classification, an error may be calculated by comparing input learning data with a neural network output. The calculated error is back-propagated in a reverse direction (ie, from the output layer to the input layer) in the neural network, and the connection weight of each node of each layer of the neural network may be updated according to the back-propagation. The amount of change in the connection weight of each updated node may be determined according to a learning rate. The neural network's computation of input data and backpropagation of errors can constitute a learning cycle (epoch). The learning rate may be applied differently according to the number of iterations of the learning cycle of the neural network. For example, a high learning rate may be used in the early stage of neural network training to increase efficiency by allowing the neural network to quickly obtain a certain level of performance, and a low learning rate may be used in the late stage to increase accuracy.

In neural network learning, generally, training data can be a subset of real data (ie, data to be processed using the trained neural network), and therefore, errors for training data are reduced, but errors for real data are reduced. There may be incremental learning cycles. Overfitting is a phenomenon in which errors for actual data increase due to excessive learning on training data. For example, a phenomenon in which a neural network that has learned a cat by showing a yellow cat does not recognize that it is a cat when it sees a cat other than yellow may be a type of overfitting. Overfitting can act as a cause of increasing the error of machine learning algorithms. Various optimization methods can be used to prevent such overfitting. In order to prevent overfitting, methods such as increasing training data, regularization, and omitting some nodes of a network in the process of learning may be applied.

In various embodiments, the control server 100 analyzes image data of the surrounding environment of the moving object 10 and generates surrounding environment information about the moving object 10 based on the information about the extracted surrounding environment. It is possible to correct surrounding environment information for the moving object 10 based on image data of other moving objects located at a point adjacent to (10).

Information on the surrounding environment of a specific mobile body is extracted by analyzing sensor data collected through the integrated monitoring module 200 provided in at least a part of the specific mobile body, and objects (eg, buildings, fixed objects) around the specific mobile body are extracted. Static objects such as obstacles, other moving objects, dynamic objects such as people, etc.) are present, the objects located around the specific moving body can be recognized through the integrated monitoring module 200 provided in the specific moving body, In the case of the area beyond the objects, sensor data collection is impossible for the area beyond the objects because it is blocked by the objects. A shaded area may occur that is not recognized by the user.

In consideration of this point, the control server 100 analyzes the image data of the surrounding environment of the moving object 10 and generates surrounding environment information about the moving object 10 based on the information about the extracted surrounding environment, A shadow area may be removed from the surrounding environment information of the moving object 10 by correcting the surrounding environment information of the moving object 10 based on image data of other moving objects located at a point adjacent to the moving object 10 .

More specifically, the control server 100 generates surrounding environment information for the first moving object by using the first image data including the surrounding environment of the first moving object, in a predetermined range around the position of the first moving object. One or more second image data including the environment of the one or more second movable bodies may be collected from the camera sensors 225 provided in the one or more second movable bodies, and the one or more second image data may be analyzed to analyze the one or more second image data. It is possible to extract information about the surrounding environment of the second moving object, and correct the surrounding environment information about the first moving object based on the extracted information about the surrounding environment of one or more second moving objects.

In various embodiments, the control server 100 uses the surrounding environment information for the moving object 10 as the surrounding environment grid for the moving object 10 to set a risk detection criterion for the moving object 10 located within a predetermined area. Maps can be pre-built.

More specifically, first, the control server 100 may grid a predetermined area. For example, the control server 100 may generate a plurality of grids by dividing a predetermined area into predetermined sizes.

Thereafter, the control server 100 collects a plurality of sensor data for positions corresponding to each of a plurality of grids from a plurality of integrated monitoring modules 200 provided in each of a plurality of movable bodies 10 located within a predetermined area. can For example, the control server 100 may collect first image data obtained by photographing a location corresponding to the first grid from the first moving object 10 located on a first grid among a plurality of grids, and may be located on a second grid. Second image data obtained by capturing a location corresponding to the second grid may be collected from the second movable body 10 that moves.

Thereafter, the control server 100 may analyze a plurality of sensor data to extract information about a plurality of surrounding environments, and record the extracted information about a plurality of surrounding environments in each of a plurality of grids, thereby providing information in a predetermined area. A grid map of the surrounding environment can be created. For example, the control server 100 may record information about the surrounding environment extracted by analyzing the first image data of the location corresponding to the first grid in the first grid, and the location corresponding to the second grid. A grid map of the surrounding environment for a predetermined area may be generated by recording information about the surrounding environment extracted by analyzing the second image data captured in the second grid in the second grid.

In various embodiments, when there are two or more sensor data for a location corresponding to a specific grid, the control server 100 transmits information about the surrounding environment extracted using only one of the two or more sensor data to a specific grid. can be recorded

For example, the control server 100 records the surrounding environment information extracted based on the sensor data for the location corresponding to any one of the plurality of grids in any one grid, and the location corresponding to any one grid. When two or more sensor data are collected from two or more mobile bodies 10, based on the sensor data collected from the mobile body 10 located at the shortest distance from the position corresponding to any one of the grids among the two or more mobile bodies 10 Only the information about the surrounding environment extracted by can be recorded in any one grid. That is, in collecting sensor data for a specific location, the closer the distance to the specific location is, the more accurate sensor data can be collected. By using the sensor data collected from the moving object 10 located at , by extracting and recording information about the surrounding environment for a specific grid, it is possible to create a grid map of the surrounding environment in which more accurate information is recorded.

As another example, the control server 100 records the surrounding environment information extracted based on the sensor data for the location corresponding to any one of the plurality of grids in any one grid, but the location corresponding to any one grid. In the case where two or more sensor data are collected from two or more mobile bodies 10, only the information about the surrounding environment extracted based on the sensor data collected last among the two or more sensor data can be recorded in any one grid. . That is, the control server 100 may generate a surrounding environment grid map in which more accurate information is recorded by extracting and recording the most up-to-date information on a specific grid.

As another example, the control server 100 records ambient environment information extracted based on sensor data for a location corresponding to any one grid among a plurality of grids in any one grid, and When two or more sensor data are collected from two or more movable bodies 10 with respect to a location, only information about the surrounding environment extracted based on the sensor data collected using a relatively high-performance sensor among the two or more movable bodies 10 is selected. Can be recorded on one grid. For example, when image data is collected from two or more movable bodies 10 with respect to a specific grid, the control server 100 provides relatively high performance based on the performance of the camera sensors 225 provided in each of the two or more movable bodies 10. Only the information about the surrounding environment extracted based on the image data generated by the camera sensor 225 may be recorded in any one grid.

Thereafter, when the control server 100 wants to set a risk detection criterion for a specific moving object 10 located within a predetermined area, a specific moving object among a plurality of grids included in the surrounding environment grid map corresponding to the predetermined area ( 10) may select a grid corresponding to the location, load information on the surrounding environment recorded on the selected grid, and use the loaded information on the surrounding environment to determine the risk detection criteria for a specific moving object 10 can be set.

In step S220, the control server 100 may set a risk detection criterion for the moving object 10 based on the surrounding environment information about the moving object 10 generated through step S210.

In various embodiments, the control server 100 may predefine risk detection criteria for each environment for the purpose of setting risk detection criteria based on the surrounding environment of the moving object 10, and Danger detection criteria for the moving object 10 may be set based on the risk detection criteria for each environment and the current surrounding environment of the moving object 10 .

More specifically, first, the control server 100 may calculate an average posture of the moving body when danger to moving bodies is detected in each of a plurality of environments based on accident data for each of a plurality of environments, and average posture of the moving body. A plurality of risk detection criteria corresponding to each of a plurality of environments may be defined in advance using a value.

Here, the accident data for each environment may be data including information about accidents occurring in each of a plurality of environments including different conditions. For example, accident data for each environment may include information on an accident that occurred on an unpaved road, information about an accident that occurred on a paved road, and information about an accident that occurred on a slope having a predetermined angle, but is not limited thereto. don't

For example, the control server 100 calculates an average angle value corresponding to the moving direction of the moving object 10 when danger to the moving objects located on the unpaved road is detected based on the information on the accident that occurred on the unpaved road. It can be calculated, and a risk detection criterion on an unpaved road can be defined using an average angle value corresponding to the traveling direction of the moving object 10 .

In addition, the control server 100 controls information about the moving objects 10 when danger to the moving objects located on the slope having a predetermined angle is detected based on the information on the accident that occurred on the ramp having a predetermined angle. An average value of the degree of leaning in a specific direction may be calculated, and a risk detection criterion on a slope having a predetermined angle may be defined using the average value of the degree of leaning of the moving object 10 in a specific direction.

In addition, the control server 100 averages angles corresponding to the moving direction of the moving object 10 when danger to the moving objects located in the rainy environment is detected based on the information on the accident occurring in the rainy environment. A value can be calculated, and a risk detection criterion in a rainy environment can be defined using an average angle value corresponding to the traveling direction of the moving object 10 . However, it is not limited thereto.

Thereafter, the control server 100 analyzes the image data of the surrounding environment of the moving object 10 and based on the information about the extracted surrounding environment, among the plurality of risk detection criteria corresponding to each of the plurality of environments, the moving object ( In 10), a risk detection criterion corresponding to the surrounding environment can be selected.

In various embodiments, the control server 100 may define risk detection criteria for detecting danger to the moving object 10 in advance, and the surroundings extracted from image data captured of the surrounding environment of the moving object 10. Predefined risk detection criteria can be calibrated based on information about the environment.

For example, the control server 100 sets the danger detection criterion for the moving object 10 as “moving object 10” by dividing 0 to 90°, which is an angle in the traveling direction of the moving object 10, into four based on the paved flat road. The angle corresponding to the direction of travel is less than 22.5° - safe state, greater than 22.5° and less than 45° - caution, greater than 45° and less than 67.5° - dangerous state, greater than 67.5° - accident state” can be defined in advance. .

Thereafter, the control server 100 assumes that the moving object 10 is located on a paved ramp having an angle of 20° based on information about the surrounding environment extracted from image data of the surrounding environment of the moving object 10. If it is determined, the danger detection criterion for the moving object 10 is set as “the angle corresponding to the moving direction of the moving object 10 is 37.5° or less-safe Condition, more than 37.5° and less than 55° - caution, more than 55° and less than 72.5° - dangerous state, more than 72.5° - accident state”.

In various embodiments, the control server 100 sets a risk detection criterion for a specific area based on surrounding environment information of the specific area, but sets a predetermined number or more mobile bodies 10 among a plurality of mobile bodies 10 located in the specific area. ), the risk detection level for a specific region can be raised by correcting the risk detection criteria for the specific region.

In addition, the control server 100, when a danger to the first movable body 10 is detected, is directed to at least one second movable body 10 located within a predetermined range around the position of the first movable body 10. The danger detection level for at least one second moving object 10 may be raised by correcting the danger detection standard for the second moving object 10 .

Here, the danger occurrence detection level for a specific area or moving object 10 may mean an index indicating how strong danger is to be detected for a specific area or moving object 10 . For example, the higher the danger detection level, the higher the danger detection level, and the lower the danger detection level, the lower the danger detection level.

That is, raising the risk detection level means lowering the risk detection criteria for detecting danger so that the danger to a specific area or a specific moving object 10 is more sensitively detected. Adjusting the level downward may mean that the occurrence of danger to a specific area or a specific moving object 10 is sensed more insensitively by raising a danger detection criterion for detecting occurrence of danger. However, it is not limited thereto.

For example, the control server 100 has a risk detection criterion for a specific area: “The angle corresponding to the moving direction of the moving object 10 is less than 22.5 ° - safe state, more than 22.5 ° and less than 45 ° - caution state, more than 45 ° 67.5 ° or less-dangerous condition, greater than 67.5°-in case of an accident”, when danger is detected for more than a predetermined number of moving objects 10 among a plurality of moving objects 10 located in a specific area, By raising the risk detection standard, “the angle corresponding to the moving direction of the moving object 10 is less than 15°-safe state, more than 15° and less than 30°-caution state, more than 30° and less than 45°-dangerous state, more than 45° By correcting to "accident occurrence state", the risk occurrence detection level can be adjusted upward.

In various embodiments, the control server 100 sets a risk detection criterion for a specific region based on surrounding environment information of the specific region, but corrects the risk detection criterion for a specific region using accident data for the specific region. can

To this end, the control server 100 may build a big data system in advance using accident data for a plurality of moving objects 10 .

For example, first, the control server 100 may collect accident data for a plurality of moving objects 10 and store the collected accident data according to a big data-based data storage method, thereby constructing a big data system. there is.

Here, the big data-based data collection and storage methods include various technologies that are already known (e.g., big data collection technologies (log collection, crawling, sensing, RSS, Open API) and big data storage technologies (NoSQL, File System, Cloud, network)), and these various known technologies can be selectively applied according to circumstances, and the present specification does not specifically disclose a method for collecting and storing data based on big data performed by the computing device 100. .

Thereafter, the control server 100 may extract accident data corresponding to a specific region from the built big data system.

Here, as for the method of extracting specific data from pre-stored big data, various technologies are already known (e.g., big data processing technology (a method of extracting result values from a large database through a query, a method of extracting result values through real-time processing) )), and these various known technologies can be selectively applied according to circumstances, and the present specification does not specifically disclose the big data processing method performed by the computing device 100.

Thereafter, the control server 100 may calculate an average value of accident data corresponding to a specific area, and the average value of the calculated accident data (eg, operation of the moving object 10 when an accident occurs with respect to the moving object 10). and average posture values) may be used to calibrate a risk detection criterion for a specific area.

Here, it is described that the control server 100 builds a big data system and uses it to calibrate the risk determination criteria corresponding to a specific area, but this is just one example, and in some cases, the big data system is used. It is possible to extract accident data for each condition such as region, weather, farmland, etc., and use the extracted accident data to calibrate the risk judgment standard for each condition.

In step S230, the control server 100 may detect the occurrence of danger to the moving object 10 based on the risk detection criterion set through step S220.

Here, detecting the occurrence of danger to the moving object 10 is not only determining whether an accident of overturning, overturning, or falling with respect to the moving object 10 has actually occurred, but also an accident of overturning, overturning, or falling with respect to the moving object 10. may include detecting whether a dangerous condition is likely to occur.

In various embodiments, the control server 100 may classify the moving object 10 into a safe state, a caution state, a dangerous state, or an accident state based on the posture of the moving object 10 and the risk detection criteria.

For example, the control server 100 may calculate the degree of leaning of the moving object 10 in a specific direction, and place the moving object 10 in a safe state and a caution state based on the degree of leaning in the specific direction and the risk determination criterion. , it can be classified as a dangerous state or an accident state.

As another example, the control server 100 may calculate an angle corresponding to the direction of movement of the moving object 10, and safely move the object 10 based on the angle corresponding to the direction of movement of the moving object 10 and the risk determination criterion. It can be classified as state, state of caution, state of danger, or state of accident. For example, the control server 100 classifies the moving object 10 into a safe state when the angle corresponding to the traveling direction of the moving object 10 is 22.5° or less, and the angle corresponding to the traveling direction of the moving object 10 exceeds 22.5°. If the angle is less than 45 °, the mobile body 10 is classified as a warning state, and if the angle corresponding to the traveling direction of the mobile body 10 exceeds 45 ° and is less than 67.5 °, the mobile body 10 is classified as a dangerous state, and the moving body 10 is classified as a dangerous state. When the angle corresponding to the traveling direction exceeds 67.5°, the moving object 10 may be classified as an accident occurrence state.

In various embodiments, the control server 100 warns the occupant of the moving object 10 about the occurrence of danger when danger is detected in the moving object 10 based on the state of the moving object 10 and the risk determination criterion. Notifications can be provided. For example, when it is determined that danger to the moving object 10 is detected, the control server 100 outputs a control command for controlling the operation of the warning notification output module 250 to the warning notification output module 250. The included warning lights and speakers can be activated.

In various embodiments, the control server 100 classifies the moving object 10 into a safe state, a caution state, a dangerous state, or an accident state based on the state of the moving object 10 and the risk detection criterion. Based on the status classification result, warning notifications having different properties may be provided.

For example, when the moving object 10 is classified as a safe state, the control server 100 does not provide a separate warning notification or controls the operation of the warning light of the warning notification output module 250 to display the first color through the warning light. A signal of (e.g. green signal) can be output.

In addition, when the moving object 10 is classified as a caution state, the control server 100 notifies the occupant of the moving object 10 with a warning notification having a first attribute (eg, a signal of a second color through a warning light (eg, a warning light)). A yellow signal) may be output and a voice signal (eg, a warning sound of a first magnitude) corresponding to an attention state may be output through a speaker.

In addition, when the moving object 10 is classified as a dangerous state, the control server 100 notifies the occupant of the moving object 10 with a warning notification having a second attribute (eg, a signal of a third color through a warning light (eg, a warning light)). A red signal) may be output and a voice signal (eg, a warning sound of a second size) corresponding to a dangerous state may be output through a speaker.

In addition, when the moving object 10 is classified as an accident occurrence state, the control server 100 notifies the occupant of the moving object 10 with a warning notification having a third attribute (eg, a signal of a third color through a warning light (eg, a warning light)). : A red signal) may be output and a voice signal (eg, a warning sound of a third level) corresponding to an accident state may be output through a speaker.

As another example, the control server 100 may control the warning notification output module 250 to output a warning notification every first period when it is determined that the state of the moving object 10 is in a safe state, and When it is determined that the state is a warning state, the warning notification output module 250 may be controlled to output a warning notification every second cycle shorter than the first cycle, and when it is determined that the state of the moving object 10 is a dangerous state The warning notification output module 250 can be controlled to output a warning notification every third period shorter than the second period, and when it is determined that the state of the moving object 10 is in an accident state, the warning notification output module 250 It can be controlled to continuously output warning notifications. However, it is not limited thereto.

In various embodiments, the control server 100 detects an occurrence of danger in the first movable body 10 based on the state of the first movable body 10 and the risk determination criterion. Provides a warning notification according to the occurrence of danger, but when at least one second moving body 10 is located within a predetermined range around the point where the first moving body 10 is located, at least one second moving body 10 ), it is possible to provide a warning notification corresponding to the occurrence of danger to the occupants who board the at least one second movable body 10 regardless of the state of the ).

In various embodiments, the control server 100 may set a specific area as a dangerous area when danger is detected for a predetermined number of moving objects 10 among a plurality of moving objects 10 located in a specific area, A warning notification corresponding to occurrence of danger may be provided to all moving objects 10 that have entered a specific area, regardless of the state of the moving objects 10 .

In various embodiments, the control server 100 notifies an occupant of the moving object 10 with a warning according to the occurrence of danger when danger is detected in the moving object 100 based on the state of the moving object 10 and the risk determination criterion. It can provide guidance information on the risks that have occurred with it.

Here, the guide information includes information on how to take action and how to cope with the danger occurring in the moving object 10, and may be information provided based on augmented reality or virtual reality. Not limited to this.

In various embodiments, when the control server 100 determines that an accident has occurred in at least one moving object 10 as at least one moving object 10 among a plurality of moving objects 10 is classified as an accident occurrence state, A report reception corresponding to an accident occurring in at least one moving object 10 may be requested to the 119 Safety Report Center server.

At this time, the control server 100 determines that an accident has occurred in at least one moving object 10 and provides a warning notification to the occupant on at least one moving object 10 to report the accident from the occupant. By requesting the 119 Safety Report Center server to receive a report in response to an accident that occurred in at least one moving object only when the requested feedback is obtained or the feedback from the occupant is not obtained within a predetermined time from the time the warning notification is provided, It is possible to prevent an incorrect report submission from being requested due to an inaccurate state judgment.

In various embodiments, the control server 100 provides a warning notification according to the dangerous occurrence of the moving object 10 as the dangerous occurrence of the moving object 10 is detected, but the warning notification is based on the point where the moving object 10 is located. Attributes may be determined, and alert notifications may be provided according to the determined alert notification attributes. For example, the control server 100 detects a danger to the moving object 10 located at the first point based on the operation of the moving object 10 located at the first point, the moving object located at the first point ( 10) Provides a warning notification according to the occurrence of danger to the occupant on board, but based on at least one of driving history data and accident history data for a plurality of moving objects at the first point, a type of warning notification for the first point, It is possible to determine the warning notification properties including the provision method, size, and number of times, and based on the warning notification properties, a warning notification according to occurrence of danger can be provided to the occupants of the moving object 10 located at the first point.

The above-mentioned method for detecting danger according to the surrounding environment of the moving object has been described with reference to the flow chart shown in the drawings. For a brief explanation, a method for detecting danger according to the surrounding environment of a moving object has been illustrated and described as a series of blocks, but the present invention is not limited to the order of the blocks, and some blocks are in a different order from those shown and operated in this specification. may be performed or performed concurrently. In addition, new blocks not described in the present specification and drawings may be added, or some blocks may be deleted or changed. Hereinafter, the UI provided by the control server 100 will be described in more detail with reference to FIGS. 13 to 56 .

13 to 56 exemplarily illustrate a user interface (UI) provided by a control server included in a system for detecting danger according to the surrounding environment of a moving object in various embodiments.

First, referring to FIG. 13 , the UI provided by the control server 100 may provide overall situation information. The UI provided by the control server 100 can summarize major events generated in the monitoring-related system by topic and provide them as a list, and when the "[More]" button displayed on the UI is selected, the page moves to the corresponding menu. can

Next, referring to FIG. 14 , the UI provided by the control server 100 may provide map information. The UI provided by the control server 100 can display a central map corresponding to the business area of the business place, display changes in the map such as classification according to the type of agricultural machine, and safety/caution according to the operating state of the agricultural machine. Results can be displayed selectively, such as /hazard/accident and failure report.

Next, referring to FIG. 15 , the UI provided by the control server 100 may provide an automatic entry/exit function. In the UI provided by the control server 100, if the sensor is operating normally within the rental period and the coordinates of the location near the rental office are confirmed, the system automatically processes it as “shipment or return (receipt)” (manager manual shipment). / return is also possible). Here, the reference distance of the range for system identification of release/return through proximity to the rental office may be a radius of 100m from the center of the location of the rental office (effective identification sector: range of up to 200m across all directions), but is not limited thereto.

Next, referring to FIG. 16 , the UI provided by the control server 100 may provide a failure report reception function. The UI provided by the control server 100 enables confirmation and processing of application received from the mobile app service for farmers, management of processing status records and preservation through administrator action memos, etc., and failure reports are received. A list can be exposed as a list.

Next, referring to FIG. 17 , the UI provided by the control server 100 may include an upper notification panel. In the notification panel area at the top of the UI provided by the control server 100, it is possible to check the number of text message transmission information, the status of alert agricultural machinery, and the status of reservation reception, and it is possible to move the page by applying a shortcut link, and the dynamic panel is always fixed. function can be provided.

Next, referring to FIGS. 18 to 24 , the UI provided by the control server 100 may provide a user registration function.

For example, referring to FIG. 18 , the UI provided by the control server 100 may provide an institution registration function. The UI provided by the control server 100 provides a process of registering basic information that is the standard for the operation of the system and the owner to identify the subject (institution) using the system, and is implemented in a form in which the system can be used only after the process. It can be.

Also, referring to FIG. 19 , the UI provided by the control server 100 may provide a region classification registration function. The UI provided by the control server 100 can register/manage the administrative area classification defined by the policy, and manage and classify the area to be inquired or the administrative district of the institution to be registered. In addition, the UI provided by the control server 100 may provide a function for registering regional classification in administrative area units for inquiry or monitoring by dividing regions of main information/map information.

Also, referring to FIGS. 20 to 24 , the UI provided by the control server 100 may provide a manager registration function. The UI provided by the control server 100 may issue and manage accounts granted with permission to access and edit functions suitable for the duties of a manager performing management tasks.

For example, referring to FIG. 20 , the UI provided by the control server 100 may provide a manager code (level) registration function for classifying manager levels to be classified/identified by the system and assigning codes.

In addition, referring to FIG. 21 , the UI provided by the control server 100 may provide an authorization code (authorization information) registration function for classifying/identifying a system for controlling administrator authorization.

In addition, referring to FIG. 22 , the UI provided by the control server 100 may provide an authorization function for setting whether to allow <read, write, modify, delete> authorization for each menu of each level manager.

In addition, referring to FIG. 23 , the UI provided by the control server 100 may provide an administrator registration function capable of granting authority and issuing or managing an account.

In addition, referring to FIG. 24 , the UI provided by the control server 100 may provide a farmer member registration function. The UI provided by the control server 100 may provide a function for managing data of a subscribed member who has installed a mobile app for farmers or for a manager to directly register and manage members.

Next, referring to FIGS. 25 to 27 , the UI provided by the control server 100 may provide an agricultural machine registration function. Through the UI provided by the control server 100, it is possible to register/manage agricultural machines managed by the rental office, and it is possible to view information such as a list of registered agricultural machines, applied sensors, and rental history.

For example, referring to FIG. 25 , the UI provided by the control server 100 may provide an agricultural machine code registration function capable of registering agricultural machine equipment to be classified/identified by the system by type and type.

In addition, referring to FIG. 26 , the UI provided by the control server 100 may provide an agricultural machine classification registration function capable of registering/managing agricultural machine classification according to work steps and types of agricultural machines.

Also, referring to FIG. 27 , the UI provided by the control server 100 may provide an agricultural machine registration function.

Next, referring to FIGS. 27 to 30, the UI provided by the control server 100 may provide a sensor registration function, and through the UI provided by the control server 100, applied to the rental agricultural machinery of the rental office You can register/manage sensor equipment.

For example, referring to FIG. 28 , the UI provided by the control server 100 may provide a sensor code registration function capable of registering sensor devices to be classified/identified by the system by shape/type.

Also, referring to FIG. 29 , the UI provided by the control server 100 may provide a sensor registration function.

Also, referring to FIGS. 30 and 31 , the UI provided by the control server 100 may provide a sensor matching function capable of matching a sensor registered in the system or searching/managing information of the matched sensor.

Next, referring to FIGS. 32 and 33 , the UI provided by the control server 100 may provide a notice registration function. Through the UI provided by the control server 100, it is possible to create/register/view notices for farmers members or managers, and managers can directly write all of the notices and register posts.

For example, referring to FIG. 32 , the UI provided by the control server 100 may provide a notice registration function for farmers. Notices for farmers are notices for farmers who have joined as members, and can be used when there are contents that need to be notified, such as safety education/other notices.

Also, referring to FIG. 33 , the UI provided by the control server 100 may provide a notice registration function for administrators. Announcements for administrators may be used for the purpose of disseminating notifications to managers in charge of control tasks by registering content to be notified to administrators who manage the system.

Next, referring to FIG. 34 , the UI provided by the control server 100 may provide a firmware registration function. Sensor-related firmware can be registered/managed through the UI provided by the control server 100 . Here, the firmware may be for updating module information in the sensor.

Next, referring to FIG. 35 , the UI provided by the control server 100 may provide a mobile app registration function. Through the UI provided by the control server 100, for the management of each Android (AOS) and Apple (iOS) mobile apps for farmer members, it is possible to register and manage as intangible assets, including APK file versions and improvements. there is.

Next, referring to FIGS. 36 to 39 , the UI provided by the control server 100 may provide an education registration function. The training registration function provided through the UI provided by the control server 100 is a posting management function for providing guidance and information on the completion process, such as mandatory safety training for reservation applicants for agricultural machine rental (rental), and Relevant information such as safety education and curriculum guidance can be provided.

For example, referring to FIGS. 36 and 37, the UI provided by the control server 100 can provide an education guide registration function through which a farmer (member) can view the contents of education information on a mobile device through a manager's posting. there is.

In addition, referring to FIGS. 38 and 39 , the UI provided by the control server 100 may provide a completion status registration function capable of searching for each training and checking/managing the number of people who have completed training.

Next, referring to FIGS. 40 to 42 , the UI provided by the control server 100 may provide a reservation reception management function. The reservation reception management function provided through the UI provided by the control server 100 is to receive/confirm reservation requests such as the type of agricultural machine desired for reservation and the rental period, and to perform tasks such as approval/rejection of the reservation. It is composed of functions, and it is possible to easily check the reservation schedule, determine whether or not it is possible to rent, etc., and to visually and efficiently assign rental agricultural machines, manage schedules in the form of a calendar, easily identify reservation status, can manage

For example, referring to FIG. 40 , the UI provided by the control server 100 may provide reservation reception registration and modification functions.

In addition, referring to FIG. 41, the UI provided by the control server 100 is a reception history search function that can manually register a reservation or check reservation reception details that require approval / rejection through a mobile app in the reservation reception list menu can provide. Here, the status of approval of reservation reception can be checked in the status of reservation completion.

In addition, referring to FIG. 42, the UI provided by the control server 100 provides reservation details, rental and return processing details, and a search function, and provides a current calendar search function to easily identify and grasp the current status in the form of a calendar. can do. Here, reservation details approved by the manager may be processed as being rented.

Next, referring to FIGS. 43 to 45 , the UI provided by the control server 100 may provide a storage/receiving management function. Through the UI provided by the control server 100, it is possible to manage the arrival / departure of agricultural machines held / managed for rental, and it is possible to inquire through the setting of the date section, etc., the rental status of each agricultural machine, It is possible to check/manage the agricultural machinery scheduled to be warehousing.

For example, referring to FIGS. 43 and 44 , the UI provided by the control server 100 may provide input/output registration and modification functions.

In addition, referring to FIG. 45 , the UI provided by the control server 100 may provide a function of checking in/out details.

Next, referring to FIGS. 46 and 47 , the UI provided by the control server 100 may provide a failure reception management function. The failure reception management function provided through the UI provided by the control server 100 is a function to manage the received contents of the A/S dispatch request with the touch of a button in the civil mobile app, and to check the received contents of the report according to the failure and action management. In addition, it is possible to search classification/total according to the type of report, the manager can directly modify the processing status, and manage the processing status through memos related to action items. In addition, by providing location data where a failure has occurred, a troubleshooting team can quickly and accurately identify the location of a failure and dispatch it.

For example, referring to FIGS. 46 and 47 , the UI provided by the control server 100 may provide a function of confirming and processing reception details.

Next, referring to FIGS. 48 to 51 , the UI provided by the control server 100 may provide an alarm notification management function.

For example, referring to FIG. 48 , the UI provided by the control server 100 may provide a text message transmission history function. You can search the cumulative list of notifications sent to mobile phone numbers pre-registered in the system, such as administrators, farmer members, and acquaintances. It can provide services such as sending text messages to mobile phones about events to notify various situations or warn of dangerous situations.

Also, referring to FIG. 49 , the UI provided by the control server 100 may provide a WebNoti list function. It is possible to search the occurrence history of Web-Notice, which is operated for the purpose of providing limited identification notices within the PC device of the controller operating the control website, and search function for details related to notification/propagation of warning situations. can notify the controller with a designated Web-Notice by referring to the level of the risk level defined by determining the situation based on the data detected during operation of agricultural machinery. In addition, it can inform about the occurrence of events such as membership registration, reservation application, equipment arrival/departure, and accident data.

In addition, referring to FIG. 50 , the UI provided by the control server 100 may provide a text message sending setting function capable of setting a text message sending criterion condition value.

In addition, referring to FIG. 51 , the UI provided by the control server 100 may provide a Web-Notice setting function capable of setting whether or not to use a Web-Notice creation condition value.

Next, referring to FIG. 52, the UI provided by the control server 100 can search/report data information collected/accumulated from the sensor, and can detect data according to the date, section, etc. of the collected/accumulated data. Data status function can be provided.

Next, referring to FIGS. 53 to 56, the UI provided by the management server 100 provides accident status/rental performance/failure reception/text transmission data with search condition setting and statistical data by period/year/quarter/month. It can provide statistics management function that can search.

For example, referring to FIG. 53, the UI provided by the control server 100 is a menu capable of inquiring/reporting performance statistics of accidents that have occurred, and displays statistical results according to the date, section, etc. of accumulated data through statistical visualization. Accident situation statistics function can be provided.

In addition, referring to FIG. 54 , the UI provided by the control server 100 may provide a rental status statistics function capable of inquiry/reporting by visualizing statistics on rental performance based on rented agricultural machines.

In addition, referring to FIG. 55, the UI provided by the control server 100 can search/report statistics on the reception of A/S dispatch requests for trouble reporting applied with a button touch in the mobile app function for farmers (members). It can provide fault report statistics function.

In addition, referring to FIG. 56 , the UI provided by the control server 100 may provide a text message transmission statistics function capable of inquiring/reporting statistics of notifications sent to mobile phones.

Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains can be implemented in other specific forms without changing the technical spirit or essential features of the present invention. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

100: control server
200: integrated monitoring module
210: computing device
220: sensor module
230: power source
240: communication module
250: warning notification output module
300: user terminal
400: network

Claims (10)

An integrated monitoring module provided in the mobile body and collecting sensor data corresponding to the mobile body; and
A control server for detecting danger to the moving object using sensor data collected through the integrated monitoring module;
The control server,
A risk detection criterion is set based on the surrounding environment of the moving object, and an occurrence of danger to the moving object is detected using a state of the moving object determined based on the collected sensor data and the set risk detection criterion;
A risk detection criterion for the specific region is set based on information about the surrounding environment of the specific region, and when a risk occurrence is detected for more than a predetermined number of moving objects among a plurality of moving objects located in the specific region, the set specific region Upgrading the risk detection level for the specific region by correcting the risk detection criteria for the region,
Hazard detection system according to the surrounding environment of the moving object. According to claim 1,
The integrated monitoring module,
A camera sensor for generating image data including the surrounding environment of the mobile body as photographing the surrounding environment of the mobile body; includes,
The control server,
Information on the surrounding environment of the moving object by analyzing image data generated through the camera sensor - The information on the surrounding environment of the moving object includes information on topography of roads in the area where the moving object is located and information on surrounding objects. and weather-related information - is extracted, ambient environment information for the mobile object is generated using the extracted information about the surrounding environment of the mobile object, and information about the mobile object is generated using the generated surrounding environment information. setting the risk detection criteria;
Hazard detection system according to the surrounding environment of the moving object. According to claim 2,
The control server,
Generating environmental information about the first moving object using first image data including the surrounding environment of the first moving object;
When one or more second movable bodies are positioned within a predetermined range around the position of the first movable body, one or more second movable bodies including the surrounding environment of the one or more second movable bodies is detected from a camera sensor provided in the one or more second movable bodies. 2 image data is collected, and the information about the surrounding environment of the generated first moving object is corrected based on the information about the surrounding environment of the at least one second moving object extracted by analyzing the collected at least one second image data.
Hazard detection system according to the surrounding environment of the moving object. According to claim 1,
The control server,
A plurality of grids are generated by gridding a predetermined area, and a plurality of sensor data for positions corresponding to each of the generated plurality of grids is obtained from a plurality of integrated monitoring modules provided in a plurality of mobile bodies located in the predetermined area. And generating a grid map of the surrounding environment for the predetermined area by recording information about the surrounding environment extracted from the collected plurality of sensor data in each of the generated grids,
When it is desired to detect danger to the moving object, information on the surrounding environment recorded in a grid corresponding to a point where the moving object is located among a plurality of grids included in the generated grid map of the surrounding environment is loaded to the moving object. to set risk detection criteria for
Hazard detection system according to the surrounding environment of the moving object. An integrated monitoring module provided in the mobile body and collecting sensor data corresponding to the mobile body; and
A control server for detecting danger to the moving object using sensor data collected through the integrated monitoring module;
The control server,
A risk detection criterion is set based on the surrounding environment of the moving object, and an occurrence of danger to the moving object is detected using a state of the moving object determined based on the collected sensor data and the set risk detection criterion;
A plurality of grids are generated by gridding a predetermined area, and a plurality of sensor data for positions corresponding to each of the generated plurality of grids is obtained from a plurality of integrated monitoring modules provided in a plurality of mobile bodies located in the predetermined area. and generating a grid map of the surrounding environment for the predetermined area by recording information about the surrounding environment extracted from the collected plurality of sensor data on each of the generated grids;
Record the information about the surrounding environment extracted based on the sensor data for the position corresponding to any one of the plurality of grids in the one grid, and with respect to the position corresponding to the one grid, two When two or more sensor data are collected from each of the two or more moving objects, only information about the surrounding environment extracted based on the sensor data collected from the moving object located at the shortest distance from the location corresponding to the grid of any one of the two or more moving objects To record on any one of the grids,
Hazard detection system according to the surrounding environment of the moving object. An integrated monitoring module provided in the mobile body and collecting sensor data corresponding to the mobile body; and
A control server for detecting danger to the moving object using sensor data collected through the integrated monitoring module;
The control server,
Setting a risk detection criterion based on the surrounding environment of the moving object, and detecting the occurrence of danger to the moving object using the state of the moving object determined based on the collected sensor data and the set risk detection criterion;
A plurality of grids are generated by gridding a predetermined area, and a plurality of sensor data for positions corresponding to each of the generated plurality of grids is obtained from a plurality of integrated monitoring modules provided in a plurality of mobile bodies located in the predetermined area. And generating a grid map of the surrounding environment for the predetermined area by recording information about the surrounding environment extracted from the collected plurality of sensor data in each of the generated grids,
Record the information about the surrounding environment extracted based on the sensor data for the position corresponding to any one of the plurality of grids in the one grid, and with respect to the position corresponding to the one grid, two When two or more sensor data are collected from each of the moving objects, only the information about the surrounding environment extracted based on the sensor data collected last among the two or more sensor data collected is recorded in any one of the grids,
Hazard detection system according to the surrounding environment of the moving object. An integrated monitoring module provided in the mobile body and collecting sensor data corresponding to the mobile body; and
A control server for detecting danger to the moving object using sensor data collected through the integrated monitoring module;
The integrated monitoring module,
It is provided at an arbitrary position of the moving body and includes an inertial measurement sensor including a gyro sensor, an acceleration sensor, and a geomagnetic sensor,
The control server,
Setting a risk detection criterion based on the surrounding environment of the moving object, and detecting the occurrence of danger to the moving object using the state of the moving object determined based on the collected sensor data and the set risk detection criterion;
The mobile body as a state of the mobile body using the 9 degree-of-freedom values of the moving body collected from the inertial measurement sensor - the 9 degree-of-freedom values include XYZ-axis angular velocity values, XYZ-axis acceleration values, and XYZ-axis geomagnetism values - Determining the posture of the moving body and detecting the occurrence of danger to the moving body based on the determined posture of the moving body,
Based on accident data for each environment - the accident data for each environment includes information on accidents that have occurred in each of a plurality of environments including different conditions - Calculate an average posture of the moving body when danger is detected, and define a plurality of danger detection criteria corresponding to each of the plurality of environments using the calculated average posture of the moving body;
Selecting a risk detection criterion corresponding to the surrounding environment of the moving object from among the plurality of risk detection criteria defined above, and detecting the occurrence of danger to the moving object based on the selected risk detection criterion and the determined attitude of the moving object.
Hazard detection system according to the surrounding environment of the moving object. delete An integrated monitoring module provided in the mobile body and collecting sensor data corresponding to the mobile body; and
A control server for detecting danger to the moving object using sensor data collected through the integrated monitoring module;
The control server,
Setting a risk detection criterion based on the surrounding environment of the moving object, and detecting the occurrence of danger to the moving object using the state of the moving object determined based on the collected sensor data and the set risk detection criterion;
When danger to the first movable body is detected, the risk detection criterion for at least one second movable body located within a predetermined range centered on the position of the first movable body is corrected so that the at least one second movable body which raises the risk detection level,
Hazard detection system according to the surrounding environment of the moving object. In the method performed through a risk occurrence detection system according to the surrounding environment of a mobile body including an integrated monitoring module and a control server for detecting danger to the mobile body,
collecting sensor data corresponding to the moving object through the integrated monitoring module; and
And detecting the occurrence of danger to the moving object using the collected sensor data,
The step of detecting the occurrence of danger to the moving object,
setting a risk detection criterion based on the surrounding environment of the moving object, and detecting danger to the moving object using a state of the moving object determined based on the collected sensor data and the set risk detection criterion; and
A risk detection criterion for the specific region is set based on information about the surrounding environment of the specific region, and when a risk occurrence is detected for more than a predetermined number of moving objects among a plurality of moving objects located in the specific region, the set specific region Comprising the step of correcting the risk detection criteria for the region to increase the risk detection level for the specific region,
A method for detecting danger according to the surrounding environment of a moving object.

Images