KR102614325B1 - Electronic device for analyzing an accident event of vehicle and operating method thereof - Google Patents

Abstract

In order to generate an accident event analysis result of the vehicle, the electronic device mounted on the vehicle acquires time information at which the accident event occurred as an accident event for the vehicle equipped with the electronic device is detected, and based on the time information, Generating first surrounding environment information for an accident event, based on the visual information, obtaining second surrounding environment information for an accident event from a vehicle control device that controls the vehicle, the first surrounding environment information and the second surrounding environment Determine the complexity of information processing for analyzing an accident event based on the information, and based on the complexity, when the electronic device performs information processing, analyze the first surrounding environment information and the second surrounding environment information to determine the complexity of the information processing to analyze the accident event. If an analysis result is generated and a predetermined analysis server performs information processing based on complexity, the first surrounding environment information and the second surrounding environment information may be transmitted to the analysis server. When the electronic device transmits the first surrounding environment information and the second surrounding environment information to the analysis server, analysis results may be generated by the analysis server. In addition, various embodiments may be possible.

Description

Electronic device for analyzing vehicle accident events and its operating method {ELECTRONIC DEVICE FOR ANALYZING AN ACCIDENT EVENT OF VEHICLE AND OPERATING METHOD THEREOF}

The embodiments below relate to a method for analyzing a vehicle accident event.

Autonomous vehicles or vehicles supporting ADAS (advanced driver assistance system) use one or more sensors to recognize situations around the vehicle, and appropriate vehicle control is required for the recognized situations. For example, to recognize the situation around the vehicle, the lanes of the road and objects around the vehicle are detected. To detect lanes or objects, images can be created using the vehicle's camera, and objects within the image can be identified.

When an accident event occurs, such as when an impact is detected in a vehicle, various information processing may be required to analyze the accident event. For example, as a result of accident event analysis, an object may be recognized in a captured image, the type of object may be determined, and the direction and speed of impact may be determined. Information processing for vehicle accident event analysis may require a high computational amount, and high computational information processing may not be possible in electronic devices within the vehicle due to performance limitations.

A vehicle control device such as an ECU (electronic control unit) can obtain various information through sensors attached or installed on the vehicle, but more information may be needed for more accurate analysis of vehicle accident events.

However, technical challenges are not limited to the above-mentioned technical challenges, and other technical challenges may exist.

According to one embodiment, an accident event analysis method performed by an electronic device includes: acquiring time information at which the accident event occurred when an accident event for a vehicle equipped with the electronic device is detected; Based on the time information, generating first surrounding environment information for the accident event; Based on the visual information, obtaining second surrounding environment information about the accident event from a vehicle control device that controls the vehicle; determining a complexity of information processing for analyzing the accident event based on the first surrounding environment information and the second surrounding environment information; Based on the complexity, when the electronic device performs the information processing, generating an analysis result for the accident event by analyzing the first surrounding environment information and the second surrounding environment information; and based on the complexity, when a predetermined analysis server performs the information processing, transmitting the first surrounding environment information and the second surrounding environment information to the analysis server - the analysis result is sent to the analysis server. Created by - may include.

According to one embodiment, the accident event may be detected by the electronic device or by one or more vehicle sensors mounted on the vehicle that communicate with the vehicle control device.

According to one embodiment, the first surrounding environment information may be generated based on at least one of a photographing device, a GPS sensor, an impact detection sensor, an acceleration sensor, and a sound detection sensor connected to the electronic device.

According to one embodiment, the second surrounding environment information includes an ultrasonic sensor, a RADAR sensor, a LiDAR sensor, a steering angle sensor, an acceleration sensor, and an AVM ( It can be obtained through at least one of a camera for (around view monitor), an impact detection sensor, and a sound detection sensor.

According to one embodiment, determining the complexity includes determining whether the vehicle is parked based on the first surrounding environment information and the second surrounding environment information; If the vehicle is parked, determining the complexity to be a first level; and if the vehicle is not parked, determining the complexity to be a second level higher than the first level.

According to one embodiment, the step of determining the complexity includes determining the number of sensors involved in acquiring the first surrounding environment information or the second surrounding environment information, and the number of sensors involved in acquiring the first surrounding environment information and the second surrounding environment information. It may include determining the complexity of the information processing based on at least one of capacity and difficulty of object detection in the image included in the first surrounding environment information and the second surrounding environment information. .

According to one embodiment, the analysis result includes size information of the external object related to the accident event, type information of the external object, impact direction information of the external object, impact speed information of the external object, and occupants inside the vehicle. It may include at least one of the status information.

According to one embodiment, the analysis result is a learning model used by the electronic device to generate an analysis result for the accident event or a learning model used by the analysis server to generate an analysis result for the accident event. The analysis result may be used to retrain the learning model of the electronic device or the learning model of the analysis server.

According to one embodiment, when the analysis server performs the information processing, receiving the analysis result from the analysis server; And it may further include storing the analysis results and transmitting the analysis results to a user terminal linked to the electronic device.

According to one embodiment, when the analysis server performs the information processing, the analysis result is an entity associated with the accident event - the entity is a server of a fire department, a server of a police station, located around the vehicle, and at least one of an insurance company's server linked to the vehicle.

According to one embodiment, the vehicle may be an autonomous vehicle or a vehicle supporting an advanced driver assistance system.

According to one embodiment, the electronic device is one of a black box and a vehicle video recording system (DVRS: Drive Video Record System), and the vehicle control device is a vehicle electronic control unit (ECU: Electronic Control Unit). You can.

According to one embodiment, an electronic device that performs an accident event analysis method includes: a sensor unit for acquiring first surrounding environment information about an accident event of a vehicle on which the electronic device is mounted; a communication unit for communicating with a vehicle control device that controls the vehicle and a predetermined accident event analysis server; Memory storing computer-executable instructions; and a processor that accesses the memory and executes the instructions, wherein the instructions obtain information on the time when the accident event occurred as the accident event is detected, and execute information based on the time information. Thus, the first surrounding environment information is generated through the sensor unit, and based on the visual information, second surrounding environment information about the accident event is acquired from the vehicle control device, and the first surrounding environment information and the Determine the complexity of information processing for analyzing the accident event based on the second surrounding environment information, and when the electronic device performs the information processing based on the complexity, the first surrounding environment information and the first 2 Generate an analysis result for the accident event by analyzing surrounding environment information, and based on the complexity, when the analysis server performs the information processing, send the first surrounding environment information and the first to the analysis server. 2 It may be configured to transmit surrounding environment information - the analysis result is generated by the analysis server.

According to one embodiment, the accident event may be detected by the sensor unit or by one or more vehicle sensors mounted on the vehicle that communicate with the vehicle control device.

According to one embodiment, the sensor unit may include at least one of a photographing device, a GPS sensor, an impact detection sensor, an acceleration sensor, and a sound detection sensor connected to the electronic device.

According to one embodiment, the second surrounding environment information includes an ultrasonic sensor, a RADAR sensor, a LiDAR sensor, a steering angle sensor, an acceleration sensor, and an AVM ( It can be obtained through at least one of a camera for (around view monitor), an impact detection sensor, and a sound detection sensor.

According to one embodiment, determining the complexity includes determining whether the vehicle is parked based on the first surrounding environment information and the second surrounding environment information; If the vehicle is parked, determining the complexity to be a first level; and if the vehicle is not parked, determining the complexity to be a second level higher than the first level.

According to one embodiment, the analysis result includes size information of the external object related to the accident event, type information of the external object, impact direction information of the external object, impact speed information of the external object, and occupants inside the vehicle. It may include at least one of the status information.

According to one embodiment, an accident event analysis system for a vehicle is mounted on the vehicle and, as the accident event is detected, obtains information on the time when the accident event occurred, and based on the time information, the accident event is detected. Generate first surrounding environment information for an event, obtain second surrounding environment information for the accident event from a vehicle control device that controls the vehicle based on the visual information, and obtain the first surrounding environment information and the first surrounding environment information. 2 an electronic device that determines a complexity for analyzing the accident event based on surrounding environment information and generates an analysis result for the accident event if the complexity is below a critical level; And when the complexity exceeds the threshold level, it may include an accident event analysis server that receives the first surrounding environment information and the second surrounding environment information from the electronic device and generates an analysis result for the accident event. .

A vehicle may include a vehicle video recording system (DVRS: Drive Video Record System), for example, a black box, and information obtained through sensors attached or installed in the vehicle as well as information obtained by the vehicle video recording device. Based on this, electronic devices and systems that analyze accident events more accurately can be provided.

A system that adaptively performs information processing for analyzing an accident event in an electronic device or an accident event analysis server may be provided depending on the complexity of information processing for analyzing an accident event.

In addition, various effects that can be directly or indirectly identified through this document may be provided.

1 is a block diagram of an electronic device, according to an embodiment.
Figure 2 is a block diagram of an accident event analysis system, according to one embodiment.
Figure 3 shows a vehicle equipped with an electronic device, according to one embodiment.
Figure 4 is a flowchart for explaining the operation of an electronic device, according to an embodiment.
Figure 5 is a diagram for explaining an operation of determining the complexity of information processing for analyzing an accident event, according to an embodiment.
FIG. 6 is a diagram for explaining the operation of an electronic device when the electronic device performs information processing for accident event analysis, according to an embodiment.
FIG. 7 is a diagram illustrating the operation of an electronic device when information processing for accident event analysis is performed in an accident event analysis server, according to an embodiment.
Figure 8 is a flowchart for explaining the operation of an accident event analysis system according to an embodiment.
Figure 9 is a flowchart for explaining the operation of the accident event analysis server of the accident event analysis system, according to one embodiment.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be changed and implemented in various forms. Accordingly, the actual implementation form is not limited to the specific disclosed embodiments, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical idea described in the embodiments.

Terms such as first or second may be used to describe various components, but these terms should be interpreted only for the purpose of distinguishing one component from another component. For example, a first component may be named a second component, and similarly, the second component may also be named a first component.

When a component is referred to as being “connected” to another component, it should be understood that it may be directly connected or connected to the other component, but that other components may exist in between.

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as “comprise” or “have” are intended to designate the presence of the described features, numbers, steps, operations, components, parts, or combinations thereof, and are intended to indicate the presence of one or more other features or numbers, It should be understood that this does not exclude in advance the possibility of the presence or addition of steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art. Terms as defined in commonly used dictionaries should be interpreted as having meanings consistent with the meanings they have in the context of the related technology, and unless clearly defined in this specification, should not be interpreted in an idealized or overly formal sense. No.

Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the accompanying drawings, identical components will be assigned the same reference numerals regardless of the reference numerals, and overlapping descriptions thereof will be omitted.

<Electronic devices and systems>

Figure 1 is a block diagram of an electronic device 101 according to an embodiment.

Referring to FIG. 1 , the electronic device 101 that analyzes an accident event may include a communication unit 110, a processor 120, a memory 130, and a sensor unit 140. The electronic device 101 may be a vehicle video recording system (DVRS) or a black box, and may be attached or installed on an autonomous vehicle or a vehicle supporting an advanced driver assistance system.

According to one embodiment, the communication unit 110 is connected to the processor 120 and the memory 130 to transmit and receive data. The communication unit 110 may be connected to another external device and receive data, for example, images captured by a camera. Hereinafter, the expression "transmitting and receiving "A" may refer to transmitting and receiving "information or data representing A."

According to one embodiment, the communication unit 110 may be implemented as a circuitry within the electronic device 101. For example, the communication unit 110 may include an internal bus and an external bus. As another example, the communication unit 110 may be an element that connects the electronic device 101 and an external device. The communication unit 110 may be an interface. The communication unit 110 may receive data from an external device and transmit the data to the processor 120 and the memory 130. According to one embodiment, the processor 120 may communicate with a vehicle control device and a server for analyzing accident events through the communication unit 110. A vehicle control device is a device for controlling mechanical or electronic devices of a vehicle, and may be, for example, an ECU (electronic control unit). The vehicle control device and the server for analyzing accident events will be described in detail with reference to FIG. 2, which shows the accident event analysis system.

According to one embodiment, the processor 120 may process data received by the communication unit 110 and data stored in the memory 130. A “processor” may be a data processing device implemented in hardware that has a circuit with a physical structure for executing desired operations. For example, the intended operations may include code included in a program or computer-executable instructions. For example, data processing devices implemented in hardware include microprocessors, central processing units, processor cores, multi-core processors, and multiprocessors. , ASIC (Application-Specific Integrated Circuit), and FPGA (Field Programmable Gate Array).

According to one embodiment, the processor 120 may execute computer-readable code (e.g., software) stored in a memory (e.g., memory 130) and instructions triggered by the processor 120. there is.

According to one embodiment, the memory 130 may store data received by the communication unit 110 and data processed by the processor 120. For example, the memory 130 may store programs (or applications, software). The stored program determines the complexity of information processing for analyzing an accident event based on information acquired through the sensor unit 140 and information received from a vehicle control device (not shown), and performs information processing based on the complexity. It may be a set of syntaxes that are coded to perform or request information processing from an accident event analysis server and can be executed by the processor 120.

According to one embodiment, the memory 130 may include one or more volatile memory, non-volatile memory, random access memory (RAM), flash memory, a hard disk drive, and an optical disk drive.

According to one embodiment, the memory 130 may store a set of instructions (eg, software) that operates the electronic device 101. A set of instructions for operating the electronic device 101 may be executed by the processor 120.

According to one embodiment, the sensor unit 140 may include at least one of a photographing device, a GPS sensor, an impact detection sensor, an acceleration sensor, and a sound detection sensor. For example, the electronic device 101 may be attached or installed on the front of the vehicle, and an image in the front direction of the vehicle may be acquired with the imaging device of the sensor unit 140. However, not only images in the front direction of the vehicle can be obtained with the imaging device, but various images around the vehicle can be obtained. For example, the imaging device of the sensor unit 140 may photograph the interior of the vehicle, and images of occupants inside the vehicle may be acquired through the sensor unit 140.

As another example, the current location information of the vehicle may be acquired using the GPS sensor of the electronic device 101, or an impact may be detected using an impact detection sensor. As another example, the acceleration sensor of the electronic device 101 obtains information about the moving speed of the vehicle equipped with the electronic device 101, and the sound detection sensor obtains various information such as the passenger's voice and/or the sound of the horn of another vehicle. Sound information may be obtained.

According to one embodiment, the processor 120 of the electronic device 101 acquires information about the time when the accident event occurred as an accident event for a vehicle equipped with the electronic device 101 is detected, and based on the time information Thus, first surrounding environment information about the accident event can be generated, and second surrounding environment information can be obtained from the vehicle control device that controls the vehicle.

According to one embodiment, the processor 120 may determine the complexity of information processing for analyzing an accident event based on the first surrounding environment information and the second surrounding environment information. For example, the result of analyzing an accident event may include at least one of the size information of the external object related to the accident event, the type information of the external object, the impact direction information of the external object, the impact speed information of the external object, and the status information of the occupants inside the vehicle. One may be included, and various information processing may have to be performed to obtain the above-described information. The operation of determining the complexity of information processing for accident event analysis will be described in detail with reference to FIG. 5.

According to one embodiment, the processor 120 may determine whether to perform information processing in the electronic device 101 or the accident event analysis server based on complexity. When performing information processing in the electronic device 101, the processor 120 may generate an analysis result for an accident event by analyzing the first surrounding environment information and the second surrounding environment information. When information processing is performed in the analysis server, the processor 120 transmits the first surrounding environment information and the second surrounding environment information to the analysis server, and the analysis result may be generated by the analysis server.

According to one embodiment, the analysis result may be generated based on a learning model included in the electronic device 101 or the accident event analysis server and used to generate the analysis result for the accident event. The specific operation of generating analysis results based on the learning model will be described in detail with reference to FIGS. 6 and 9.

The operation of the processor 120 of the electronic device 101 according to various embodiments will be described in detail with reference to FIGS. 4 to 7.

Figure 2 is a block diagram of an accident event analysis system 200, according to one embodiment.

Referring to FIG. 2 , an accident event analysis system 200 according to an embodiment is shown, including the electronic device 101 described above with reference to FIG. 1 . The accident event analysis system 200 may include an electronic device 101, a vehicle control device 230 that communicates with the electronic device 101, and an accident event analysis server 250. Descriptions of the vehicle control device 230 and the accident event analysis server 250 that overlap with those described above with reference to FIG. 1 will be omitted.

As described above with reference to FIG. 1 , the vehicle control device 230 is a device for controlling mechanical or electronic devices of a vehicle and may be an ECU. The vehicle control device 230 may acquire information sensed through vehicle sensors, process it, and transmit it to the electronic device 101. For example, the vehicle control device 230 may detect the occurrence of an accident event through vehicle sensors and transmit information on the time when the accident event occurred to the electronic device 101. As another example, as the vehicle control device 230 detects an accident event, the information detected by the vehicle sensor may be transmitted to the electronic device 101 as second surrounding environment information for a predetermined period of time based on visual information.

The accident event analysis server 250 may perform information processing to analyze the accident event according to a request from the electronic device 101. The accident event analysis server 250 may include a learning model used to generate analysis results for the accident event, and the learning model included in the accident event analysis server 250 is a learning model included in the electronic device 101. The calculation speed may be faster. The operation of the accident event analysis server 250 will be described in detail with reference to FIG. 9.

The operation of the accident event analysis system 200 will be described in detail with reference to FIGS. 8 and 9.

Figure 3 shows a vehicle 300 equipped with an electronic device according to an embodiment.

Referring to FIG. 3 , a vehicle 300 including the electronic device 101 described with reference to FIG. 1 is shown. The vehicle 300 according to one embodiment may drive in an autonomous mode according to the perceived surrounding environment, even in situations where there is little or no input from the driver. In other words, the vehicle 300 may be an autonomous vehicle or a vehicle that supports an advanced driver assistance system (ADAS).

The surrounding environment may be recognized through one or more vehicle sensors attached or installed on the vehicle 300. For example, one or more automotive sensors may include an ultrasonic sensor, a RADAR sensor, a LiDAR sensor, a steering angle sensor, an acceleration sensor, a camera for an AVM (around view monitor), and an impact detection sensor. and a sound detection sensor. Vehicle sensors may be located in the front (310, 320), side mirrors (330, 340), rear (350), etc. of the vehicle, but are not limited to the examples described. For example, automotive sensors may be located on the vehicle's front grill, rear trunk, left/right wheels, etc. As another example, some of the vehicle sensors may be installed inside the vehicle 300. For example, the vehicle sensor may include a camera attached or installed inside the vehicle 300, and the vehicle control device 230 may acquire images of the occupants through the camera inside the vehicle 300. The surrounding environment may include roads, road conditions, types of lanes, presence or absence of surrounding vehicles, distance to nearby vehicles, weather, presence or absence of obstacles, etc., and is not limited to the examples described.

Among vehicle sensors, a camera may be mounted at a horizontal angle or a constant tilt angle to the vehicle 300 for parking assistance, and a camera including a fisheye lens may be used to ensure that there are no blind spots. there is. For example, the sensor may be built-in to the vehicle 300.

The vehicle 300 can recognize the surrounding environment and create an autonomous driving path suitable for the surrounding environment. Autonomous vehicles can control internal and external mechanical elements to follow an autonomous path. The vehicle 300 may periodically generate an autonomous driving route.

According to another aspect, the vehicle 300 may assist the driver's driving using an advanced driver assistance system (ADAS). ADAS includes Autonomous Emergency Braking (AEB), which slows down or stops on its own without the driver having to apply the brakes in the event of a collision, and Lane Keep Assist, which adjusts the driving direction in case of lane departure to maintain the lane. System: LKAS), Advanced Smart Cruise Control (ASCC), which automatically maintains the distance from the car in front while driving at a preset speed, and rear and side collision avoidance support that detects the risk of blind spot collision and helps change lanes safely. It may include a system (Active Blind Spot Detection: ABSD) and an Around View Monitor (AVM) system that visually shows the situation around the vehicle.

The vehicle control device 230 included in the vehicle 300 can control the mechanical or electronic devices of the vehicle 300 to drive autonomously or assist the driver's driving, and various types of controllers or sensors other than the described embodiment. You can communicate with etc.

According to one embodiment, the electronic device 101 may be a black box, and may be attached or installed in a vehicle, or built-in to the vehicle, as shown in FIG. 3. The electronic device 101 may communicate with the vehicle control device 230 through wired or wireless communication. For example, a USB dongle for Bluetooth communication may be inserted into the electronic device 101, and the electronic device 101 may perform Bluetooth-based wireless communication with the vehicle control device 230. As another example, the electronic device 101 may perform wireless communication with the vehicle control device 230 based on a cellular network (eg, LTE). As another example, when the electronic device 101 is built into the vehicle 300, it can communicate with the vehicle control device 230 by wire.

In FIGS. 1 to 3 described above and FIGS. 4 to 9 described later, the electronic device 101 is described separately from the vehicle control device 230 such as an ECU, but depending on implementation, the electronic device 101 may be used as a vehicle control device. It can be implemented as one with (230). For example, when the electronic device 101 is built into the vehicle 300, it can operate as a single device with the vehicle control device 230.

Below, with reference to FIGS. 4 to 9 , how the electronic device 101 or the accident event analysis system 200 analyzes an accident event will be described in detail.

<How to analyze accident events>

FIG. 4 is a flowchart for explaining the operation of the electronic device 101 according to an embodiment.

Steps 410 to 460 may be performed by the processor 120 of the electronic device 101 described above with reference to FIG. 1, and for concise and clear explanation, content that overlaps with the content described with reference to FIGS. 1 to 3 will be omitted. It may be omitted.

In step 410, the processor 120 may obtain information about the time of occurrence of the accident event as an accident event for a vehicle equipped with the electronic device 101 is detected.

As described above with reference to FIG. 3, the vehicle 300 equipped with the electronic device 101 may include a vehicle control device 230 for controlling mechanical or electronic devices included in the vehicle, and an accident event may occur in the vehicle. It may be detected through a vehicle sensor that communicates with the control device 230 or the sensor unit 140 included in the electronic device 101. For example, an accident event may be detected by an impact detection sensor among vehicle sensors or an impact detection sensor of the sensor unit 140.

According to one embodiment, an accident event is detected through vehicle sensors, and the vehicle control device 230 may obtain information on the time when the accident event occurred and transmit it to the electronic device 101. According to another embodiment, an accident event is detected through the sensor unit 140 of the electronic device 101, and the processor 120 may obtain information on the time when the accident event occurred.

In step 420, the processor 120 may generate first surrounding environment information about the accident event based on visual information. For example, when an accident event is detected through the sensor unit 140 of the electronic device 101, the processor 120 detects the sensor unit 140 for a predetermined period of time (e.g., 5 minutes before and after) based on the visual information. The sensed information can be generated as first surrounding environment information. As another example, when the vehicle control device detects an accident event through vehicle sensors, the processor 120 receives time information from the vehicle control device and operates the sensor unit 140 for a predetermined time based on the received time information. Information detected through the sensor can be generated as first surrounding environment information.

In step 430, the processor 120 may obtain second surrounding environment information from a vehicle control device that controls the vehicle. For example, when an accident event is detected through the sensor unit 140 of the electronic device 101, the processor 120 detects the vehicle sensor for a predetermined period of time (for example, 5 minutes before and after the impact detection time) based on the visual information. Second surrounding environment information can be obtained by requesting the sensed information to the vehicle control device. As another example, when the vehicle control device detects an accident event through vehicle sensors, the vehicle control device may transmit the information detected through the vehicle sensor to the electronic device 101 for a predetermined period of time based on visual information, and the processor 120 may obtain second surrounding environment information.

However, it is not limited to the above-described embodiments, and the first surrounding environment information and the second surrounding environment information can be obtained in various ways. For example, the vehicle control device transmits information detected through a vehicle sensor in bulk to the electronic device 101, and the processor 120 provides data for a predetermined time based on visual information among the bulk data. 2 Can be obtained from information about the surrounding environment.

In step 440, the processor 120 may determine the complexity of information processing for analyzing an accident event based on the first surrounding environment information and the second surrounding environment information. In step 450, the processor 120 may determine whether the complexity is below a critical level. The complexity determination process and the threshold level for classifying complexity will be described in detail with reference to FIG. 5.

In step 460, if the complexity is below a critical level, the processor 120 may perform information processing for accident event analysis based on the first surrounding environment information and the second surrounding environment information and generate an analysis result. As described above with reference to FIG. 1, the results of analyzing the accident event include size information of the external object related to the accident event, type information of the external object, impact direction information of the external object, impact speed information of the external object, and occupants inside the vehicle. At least one of the state information may be included, and the processor 120 may perform information processing for this. For example, the processor 120 recognizes an object involved in an accident event and determines the type of the object by detecting the forward direction of the vehicle 300 obtained through the imaging device of the sensor unit 140 of the electronic device 101. DNN-based semantic segmentation can be performed on images. However, this is only an example, and the processor 120 acquires size information of the external object related to the accident event, type information of the external object, impact direction information of the external object, impact speed information of the external object, and status information of the occupants inside the vehicle. Various methods can be used to do this.

According to one embodiment, the processor 120 may obtain status information of the occupants inside the vehicle by analyzing images of the occupants inside the vehicle obtained by the sensor unit 140 of the electronic device 101 or a vehicle sensor. For example, the processor 120 may determine that the occupant is unconscious when no movement is detected in an image inside the vehicle after a vehicle accident event occurs.

According to one embodiment, the processor 120 may use a learning model included in the electronic device 101 in analyzing an accident event, and the processor 120 uses the learning model to generate analysis results for the accident event. Specific operations will be described in detail with reference to FIG. 6 .

In step 470, the processor 120 may transmit the first surrounding environment information and the second surrounding environment information to the predetermined accident event analysis server 250 when the complexity exceeds a critical level. The analysis result including at least one of the size information of the external object related to the accident event, the type information of the external object, the impact direction information of the external object, the impact speed information of the external object, and the status information of the occupants inside the vehicle are sent to the analysis server (250) ) can be created by.

Figure 5 is a diagram for explaining an operation of determining the complexity of information processing for analyzing an accident event, according to an embodiment.

Steps 510 to 530 may be performed by the processor 120 of the electronic device 101 described above with reference to FIG. 1, and for concise and clear explanation, content that overlaps with the content described with reference to FIGS. 1 to 4 will be omitted. It may be omitted.

According to one embodiment, steps 510 to 530 include determining the complexity of information processing for analyzing an accident event based on the first surrounding environment information and the second surrounding environment information described with reference to FIG. 4 (e.g., FIG. It can correspond to step 440) of 4. In connection with the operation of determining complexity, a threshold level of complexity for determining whether to perform information processing in the electronic device 101 or the analysis server 250 (e.g., the threshold in step 450 of FIG. 4 level) is also explained.

In step 510, the processor 120 may determine whether the vehicle including the electronic device 101 is parked based on the first surrounding environment information and the second surrounding environment information. For example, in the front view image of a vehicle among the first surrounding environment information acquired for a predetermined time based on visual information, the processor 120 of the electronic device 101 detects movement above a certain level in the front view image during that time. If this does not occur, it can be determined that the vehicle is parking. As another example, the second surrounding environment information received from the vehicle control device 230 includes information about the state of the vehicle (e.g., gear is parked (P), driving (D), and parked (P) in neutral (N). ) may be included, and the processor 120 may determine that the vehicle is parked based on vehicle status information.

If it is determined that the vehicle is parked, in step 520 the processor 120 may determine a first level of complexity of information processing for analyzing the accident event. When parking, the vehicle moves relatively less than when not parking, so the amount of information processing required to recognize or extract an object from image information included in the first surrounding environment information and the second surrounding environment information may be less. Since the amount of information processing is small, the processor 120 may determine the complexity to be a relatively low first level and decide to perform information processing to analyze the accident event in the electronic device 101.

If it is determined that the vehicle is not parked, in step 530 the processor 120 may determine the complexity of information processing for analyzing the accident event to be a second level higher than the first level. When the vehicle is not parked, there is relatively more movement of the vehicle than when it is parked, and the amount of information processing required to recognize or extract an object from the image information included in the first surrounding environment information and the second surrounding environment information may be large. The processor 120 determines the complexity to be a second level higher than the first level because the amount of information processing is large, and processes the information to analyze the accident event in the predetermined accident event analysis server 250 in communication with the electronic device 101. You may decide to carry out . As described above with reference to FIG. 4 , when the analysis server 250 performs information processing, the processor 120 may transmit first surrounding environment information and second surrounding environment information to the analysis server 250 .

Referring to FIG. 5, an embodiment of determining the complexity has been described based on whether the vehicle is parked, but it is not limited to this and the complexity is determined in various ways, and the image is processed in the electronic device 101 according to the complexity. It may be determined whether to perform image processing in the analysis server 250.

For example, in the above-described embodiment, it was explained that the processor 120 determines the complexity to be high when the vehicle is not parked, but even when the vehicle is not parked, the complexity is determined to be low and the electronic device 101 detects an accident event. Information processing may be performed to analyze. According to one embodiment, the processor 120 may determine complexity according to the amount of change in the acquired images. When a vehicle is traveling at a certain speed on a highway, the amount of change between acquired images may be relatively small, and the processor 120 determines the complexity to be low even if the vehicle is not parked, and the electronic device 101 determines the accident Information processing for event analysis can be performed.

According to one embodiment, the processor 120 determines the complexity based on the sum of the capacities of the first surrounding environment information and the second surrounding environment information and performs information processing in the electronic device 101 according to a predetermined threshold. or whether to perform information processing in the analysis server 250. For example, the processor 120 may determine to perform information processing on the electronic device 101 when the sum of the capacities of the first surrounding environment information and the second surrounding environment information is lower than a threshold. The threshold may be determined in various ways, for example, based on the amount of data that a learning model included in the electronic device 101 and used to generate analysis results for an accident event can process within 1 minute. This can be decided.

According to another embodiment, the processor 120 determines the complexity based on the number of sensors involved in the process of acquiring the first surrounding environment information and the second surrounding environment information, and configures the electronic device 101 according to a predetermined threshold number. ), or whether to perform information processing in the analysis server 250. For example, the first surrounding environment information is acquired by a camera and an impact detection sensor included in the sensor unit 140 of the electronic device 101, and the second surrounding environment information is obtained by a vehicle sensor that communicates with the vehicle control device 230. Among these, it can be acquired with an around view monitor (AVM) camera installed on the front and left side mirrors (for example, cameras installed at 320 and 330 in FIG. 3). At this time, the processor 120 may determine that the number of sensors involved in the process of acquiring the first surrounding environment information and the second surrounding environment information is 4, and since it is below the threshold (for example, 5), the electronic device 101 ) may decide to perform information processing.

According to another embodiment, the processor 120 may determine the complexity according to the difficulty of object detection in images included in the first surrounding environment information and the second surrounding environment information. For example, the first surrounding environment information may include a front image of the vehicle captured by a camera among the sensor units 140 of the electronic device 101, and the second surrounding environment information may include an image captured by a camera for an AVM among vehicle sensors. Video surrounding the vehicle may be included. When it is difficult to detect an object in an image through image analysis, the processor 120 may determine that the complexity of information processing for analyzing an accident event is high. For example, if the illuminance of the captured image is below the threshold, the processor 120 may determine that object detection is difficult and the complexity of information processing may be high. Based on the illuminance threshold, the processor 120 may determine that the electronic device 101 performs information processing when the illuminance of the captured image is greater than or equal to the threshold.

FIG. 6 is a diagram for explaining the operation of an electronic device when the electronic device performs information processing for accident event analysis, according to an embodiment.

Steps 610 to 640 may be performed by the processor 120 of the electronic device 101 described above with reference to FIG. 1, and for concise and clear explanation, content overlaps with the content described with reference to FIGS. 1 to 5. can be omitted.

According to one embodiment, in steps 610 to 630, when information processing is performed in the electronic device 101, which is described with reference to FIG. 4, the processor 120 performs processing based on first surrounding environment information and second surrounding environment information. It may correspond to an operation of generating an analysis result for an accident event (eg, step 460 of FIG. 4).

In steps 610 to 620, the processor 120 may input first surrounding environment information and second surrounding environment information into a learning model included in the electronic device 101 and obtain analysis results output from the learning model. The learning model included in the electronic device 101 may be stored in, for example, the memory 130 and may be updated.

In step 630, the processor 120 may update the learning model included in the electronic device 101 according to the analysis result. For example, the processor 120 may retrain the analysis results into a learning model.

As described with reference to step 460 of FIG. 4, the results of analyzing the accident event include size information of the external object related to the accident event, type information of the external object, impact direction information of the external object, impact speed information of the external object, and vehicle At least one of the status information of the occupants inside may be included.

According to one embodiment, the processor 120 generates an accident event analysis result (e.g., steps 610 to 630), then stores the analysis result in step 640, and stores the analysis result in a device linked to the electronic device 101. It can be transmitted to the user terminal. For example, the analysis results can be stored in the memory 130 of the electronic device 101 and transmitted to a smartphone linked to the electronic device 101 or transmitted to the vehicle control device 230. The vehicle control device 230 may store the analysis results and display them to the driver. For example, the analysis results may be displayed through at least one of the vehicle's head up display (HUD), center information display (CID), rear seat entertainment (RSE), and passenger seat display.

According to another embodiment, in step 640, the processor 120 may transmit an emergency rescue request according to status information of occupants inside the vehicle included in the analysis result. For example, if no movement is detected in the video of the occupants inside the vehicle after an accident event occurs, the processor 120 may generate occupant status information indicating that the occupant is unconscious as a result of the analysis, and may generate occupant status information indicating that the occupant is unconscious, and may generate occupant status information indicating that the occupant is unconscious, and may generate occupant status information indicating that the occupant is unconscious. , an emergency rescue request can be transmitted to the fire department's server or the police station's server.

FIG. 7 is a diagram illustrating the operation of an electronic device when information processing for accident event analysis is performed in an accident event analysis server, according to an embodiment.

Steps 710 to 720 may be performed by the processor 120 of the electronic device 101 described above with reference to FIG. 1, and for concise and clear explanation, content overlaps with the content described with reference to FIGS. 1 to 6. can be omitted.

According to one embodiment, it is determined to perform information processing to analyze the accident event in the analysis server 250, which is predetermined based on complexity, and the first surrounding environment information and the second surrounding environment information are sent to the analysis server 250. After the operation of transmitting (e.g., operation 470 of FIG. 4), steps 710 to 720 may be performed.

In step 710, the processor 120 may receive an analysis result from the predetermined analysis server 250. The operation of generating analysis results in the analysis server 250 will be described in detail with reference to FIG. 9.

In step 720, the processor 120 may store the analysis results and transmit the analysis results to the user terminal linked to the electronic device 101. Since step 720 overlaps with step 640 of FIG. 6, description is omitted.

Figure 8 is a flowchart for explaining the operation of an accident event analysis system according to an embodiment.

Steps 810 to 880 may be operations of the accident event analysis system 200 described above with reference to FIG. 2 and may be performed by the electronic device 101 (or the processor 120 of the electronic device 101) described above with reference to FIG. 1 ) or may be performed by the accident event analysis server 250. For concise and clear description, content that overlaps with the content described with reference to FIGS. 1 to 7 may be omitted.

In steps 810 to 850, as the accident event is detected, the electronic device 101 acquires information about the time of occurrence of the accident event, generates first surrounding environment information, receives second surrounding environment information, and generates first surrounding environment information. Based on the environmental information and the second environmental information, the complexity of information processing for analyzing the accident event is determined, and depending on the threshold level, whether to perform information processing for accident event analysis in the electronic device 101, the accident event analysis server ( 250), you can decide whether to perform information processing.

When information processing is performed in the electronic device 101, in step 860, the electronic device 101 may generate an analysis result for the accident event based on the first surrounding environment information and the second surrounding environment information.

When information processing is performed in the accident event analysis server 250, the electronic device 101 may transmit first surrounding environment information and second surrounding environment information to the predetermined analysis server 250 in step 870.

Since steps 810 to 870 overlap with the operations of the electronic device 101 described above with reference to FIG. 4 (e.g., steps 410 to 470 of FIG. 4 ), detailed description thereof will be omitted.

When the first surrounding environment information and the second surrounding environment information are transmitted to the analysis server 250, the analysis server 250 may generate an analysis result for the accident event in step 880. The operation of the accident event analysis server 250 to generate analysis results for an accident event will be described in detail with reference to FIG. 9 .

Figure 9 is a flowchart for explaining the operation of the accident event analysis server of the accident event analysis system, according to one embodiment.

Steps 910 to 940 may be performed by the accident event analysis server 250 of the accident event analysis system 200 described above with reference to FIG. 2, and are described with reference to FIGS. 1 to 8 for concise and clear explanation. Content that overlaps with the content may be omitted.

According to one embodiment, steps 910 to 930 are performed based on the first surrounding environment information and the second surrounding environment information when the analysis server 250 performs information processing, which is explained with reference to FIG. 8. This may correspond to an operation of generating an analysis result for an accident event (eg, step 880 of FIG. 8).

In steps 910 to 920, the analysis server 250 may input first surrounding environment information and second surrounding environment information into a learning model included in the analysis server 250 and obtain analysis results output from the learning model. . As described above with reference to FIG. 6, the analysis results include information on the size of the external object related to the accident event, information on the type of the external object, information on the direction of impact of the external object, information on the impact speed of the external object, and information on the status of the occupants inside the vehicle. At least one may be included.

The learning model included in the analysis server 250 and used to generate analysis results for the accident event is the learning model included in the electronic device 101 (e.g., the electronic device 101 described above with reference to FIG. 6 ) may perform better than the learning model included in ). For example, the learning model included in the analysis server 250 may have a faster computational processing speed and may be composed of more nodes than the learning model included in the electronic device 101. According to one embodiment, the learning model included in the analysis server 250 may be periodically updated.

In step 930, the analysis server 250 may update the learning model included in the analysis server 250 according to the analysis results. For example, the analysis server 250 may retrain the analysis results into a learning model.

As described above with reference to FIG. 4, the accident event analysis results generated by the analysis server 250 include size information of the external object related to the accident event, type information of the external object, impact direction information of the external object, and impact of the external object. At least one of speed information and status information of occupants inside the vehicle may be included.

According to one embodiment, the analysis server 250 may generate an accident event analysis result (eg, steps 910 to 930) and then transmit the analysis result to an entity related to the accident event in step 940. For example, the analysis results generated by the analysis server 250 may be transmitted to at least one of a fire department server located around the vehicle, a police station server, and an insurance company server linked to the vehicle.

According to another embodiment, the analysis results may be transmitted back to the electronic device 101. As described above with reference to operation 640 of FIG. 6, the electronic device 101 stores the analysis results received from the analysis server 250 in the memory 130 and analyzes them with a smartphone linked to the electronic device 101. The results can be transmitted or transmitted to the vehicle control device 230. The vehicle control device 230 may store the analysis results and display the analysis results through at least one of the vehicle's head up display (HUD), centerer information display (CID), rear seat entertainment (RSE), and passenger seat display. there is.

The embodiments described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, and a field programmable gate (FPGA). It may be implemented using a general-purpose computer or a special-purpose computer, such as an array, programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and software applications running on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include multiple processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used on any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on a computer-readable recording medium.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. A computer-readable medium may store program instructions, data files, data structures, etc., singly or in combination, and the program instructions recorded on the medium may be specially designed and constructed for the embodiment or may be known and available to those skilled in the art of computer software. there is. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

The hardware devices described above may be configured to operate as one or multiple software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on this. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent.

Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the claims described below.

Claims (20)

In an accident event analysis method performed by an electronic device,
As an accident event for a vehicle equipped with the electronic device is detected, obtaining information on the time when the accident event occurred;
Based on the visual information, generating first surrounding environment information for the accident event;
Based on the visual information, obtaining second surrounding environment information about the accident event from a vehicle control device that controls the vehicle;
determining a complexity indicating an information processing amount for analyzing the accident event as a first level or a second level based on the first surrounding environment information and the second surrounding environment information;
As the complexity is determined to be the first level, when the electronic device performs the information processing, an analysis result for the accident event is generated by analyzing the first surrounding environment information and the second surrounding environment information. step; and
As the complexity is determined to be the second level, when a predetermined analysis server performs the information processing, transmitting the first surrounding environment information and the second surrounding environment information to the analysis server - the analysis result is generated by the above analysis server -
Including,
How to analyze accident events.
According to paragraph 1,
The accident event is,
Sensed by the electronic device or by one or more vehicle sensors mounted on the vehicle in communication with the vehicle control device,
How to analyze accident events.
According to paragraph 1,
The first surrounding environment information is,
Generated based on at least one of a photographing device, a GPS sensor, an impact detection sensor, an acceleration sensor, and a sound detection sensor connected to the electronic device,
How to analyze accident events.
According to paragraph 1,
The second surrounding environment information is,
Ultrasonic sensor, RADAR sensor, LiDAR sensor, steering angle sensor, acceleration sensor, camera for AVM (around view monitor), shock detection sensor, and sound detection sensor mounted on the vehicle. Obtained through at least one of the
How to analyze accident events.
According to paragraph 1,
The step of determining the complexity as the first level or the second level is,
determining whether the vehicle is parked based on the first surrounding environment information and the second surrounding environment information;
If the vehicle is parked, determining the complexity to the first level; and
If the vehicle is not parked, determining the complexity to be the second level higher than the first level.
Including,
How to analyze accident events.
According to paragraph 1,
The step of calculating the complexity as the first level or the second level is,
The number of sensors involved in acquiring the first surrounding environment information or the second surrounding environment information, the capacity of the first surrounding environment information and the second surrounding environment information, and the first surrounding environment information and the second surrounding environment information. Determining the complexity of the information processing as the first level or the second level based on at least one of the difficulty of object detection in the image included in the environment information
Including,
How to analyze accident events.
According to paragraph 1,
The results of the above analysis are,
Containing at least one of size information of the external object related to the accident event, type information of the external object, impact direction information of the external object, impact speed information of the external object, and status information of the occupants inside the vehicle,
How to analyze accident events.
According to paragraph 1,
The analysis result is generated based on a learning model used by the electronic device to generate an analysis result for the accident event or a learning model used by the analysis server to generate an analysis result for the accident event,
The analysis results are used to retrain the learning model of the electronic device or the learning model of the analysis server,
How to analyze accident events.
According to paragraph 1,
When the analysis server performs the information processing, receiving the analysis result from the analysis server; and
Storing the analysis results and transmitting the analysis results to a user terminal linked to the electronic device.
Containing more,
How to analyze accident events.
According to paragraph 1,
When the analysis server performs the information processing,
The analysis result is an entity associated with the accident event - the entity includes at least one of a fire department server located around the vehicle, a police station server, and an insurance company server linked to the vehicle. transmitted,
How to analyze accident events.
According to paragraph 1,
The vehicle is an autonomous vehicle or a vehicle that supports an advanced driver assistance system,
How to analyze accident events.
According to paragraph 1,
The electronic device is either a black box or a vehicle video recording system (DVRS: Drive Video Record System),
The vehicle control device is a vehicle electronic control unit (ECU: Electronic Control Unit),
How to analyze accident events.
A computer-readable recording medium containing a program for performing the method of any one of claims 1 to 12.
In an electronic device performing an accident event analysis method,
a sensor unit configured to acquire first surrounding environment information about an accident event of a vehicle equipped with the electronic device;
a communication unit for communicating with a vehicle control device that controls the vehicle and a predetermined accident event analysis server;
Memory storing computer-executable instructions; and
A processor that accesses the memory and executes the instructions
Including,
The above commands are:
As the accident event is detected, information on the time when the accident event occurred is obtained, and based on the time information, the first surrounding environment information is generated through the sensor unit, and based on the time information, A first complexity indicating the information processing amount for obtaining second surrounding environment information about the accident event from the vehicle control device and analyzing the accident event based on the first surrounding environment information and the second surrounding environment information level or a second level, and as the complexity is determined to be the first level, when the electronic device performs the information processing, the first surrounding environment information and the second surrounding environment information are analyzed to determine the first level. Generating an analysis result for an accident event, and as the complexity is determined to be the second level, when the analysis server performs the information processing, the first surrounding environment information and the second surrounding environment information to the analysis server Transmit environmental information - the analysis results are generated by the analysis server -
configured to,
Electronic devices.
According to clause 14,
The accident event is,
Sensed by the sensor unit, or detected by one or more vehicle sensors mounted on the vehicle in communication with the vehicle control device,
Electronic devices.
According to clause 14,
The sensor unit,
Comprising at least one of a photographing device connected to the electronic device, a GPS sensor, an impact detection sensor, an acceleration sensor, and a sound detection sensor,
Electronic devices.
According to clause 14,
The second surrounding environment information is,
Ultrasonic sensor, RADAR sensor, LiDAR sensor, steering angle sensor, acceleration sensor, camera for AVM (around view monitor), shock detection sensor, and sound detection sensor mounted on the vehicle. Obtained through at least one of the
Electronic devices.
According to clause 14,
The step of determining the complexity as the first level or the second level is,
determining whether the vehicle is parked based on the first surrounding environment information and the second surrounding environment information;
If the vehicle is parked, determining the complexity to the first level; and
If the vehicle is not parked, determining the complexity to be at the second level higher than the first level.
Including,
Electronic devices.
According to clause 14,
The results of the above analysis are,
Containing at least one of size information of the external object related to the accident event, type information of the external object, impact direction information of the external object, impact speed information of the external object, and status information of the occupants inside the vehicle,
Electronic devices.
In an accident event analysis system for a vehicle,
As it is mounted on the vehicle and the accident event is detected, information on the time when the accident event occurred is obtained, and first surrounding environment information about the accident event is generated based on the time information, and the time information Information for obtaining second surrounding environment information about the accident event from a vehicle control device that controls the vehicle based on and analyzing the accident event based on the first surrounding environment information and the second surrounding environment information An electronic device that determines a complexity indicating throughput as a first level or a second level, and generates an analysis result for the accident event when the complexity is the first level; and
When the complexity is the second level, an accident event analysis server that receives the first surrounding environment information and the second surrounding environment information from the electronic device and generates an analysis result for the accident event.
Including,
Accident event analysis system.

Images