KR102067547B1 - Method and detection for peripheral vehicle risk based on image analysis and communication connection between portable communication terminals and building cctv camera - Google Patents

Abstract

According to the present invention, even if a vehicle does not have a separate device or facility, in order to enable emergency communication between vehicles, a situation is shared based on a cellular network between smartphones, emergency situations are transmitted to nearby vehicles. As vehicle owners drive vehicles while holding their smartphones in the vehicle, a situation notification service may be performed through such network communication between smartphones. In particular, by acquiring the CCTV image of the building, the dangerous situation is detected through image analysis, and is transmitted to the smartphone of the nearby vehicle, so as to perform compensation for the emergency situation awareness.

Description

Hazard detection and method of surrounding vehicle based on communication connection and image analysis between mobile communication terminal and building CCTV camera

The present invention relates to a communication technology for providing a dangerous situation to a mobile communication terminal located in a surrounding vehicle by utilizing a network transmission of a mobile communication terminal and a building CCTV camera. More specifically, the driver of a vehicle is always in a vehicle. While driving with a smart phone, the vehicle can be detected and propagated by sharing information through the mobile communication network between smart phones, but the communication between the smart phone and the building CCTV camera is used to recognize the danger situation. It is about a technology to complement.

Conventionally, there is an automatic accident notification function that can receive emergency and emergency rescue services by sending an accident-related message to the control center only when a vehicle accident occurs.

This automatic accident notification function can function normally only when the communication environment is guaranteed or the communication system related to the accident notification is not broken.

On the other hand, when an accident vehicle exists in a shaded area such as a tunnel or a mountainous area, and a communication connection with the control center is not smooth, or a communication antenna is damaged and cannot be connected to a general network, the unit of the accident vehicle (including a communication module) ) May not be able to connect to the network even if the network tries to connect to the control center continuously.

Meanwhile, the black box installed in the vehicle not only records the front state of the vehicle as a video using a camera but also stores still images when an event occurs using various sensors installed in the vehicle. In addition, a user interface such as a display or an input unit may be provided to receive various control commands from the user.

In addition, CCTVs in buildings have been mainly used for crime prevention, but are not used for the purpose of preventing accidents.

On the other hand, the related art related to the Korean Patent Publication No. 2017-0100422 (Invention: Apparatus and method for notifying the emergency situation of the vehicle using the external terminal ', Publication Date: 2017.09.04, IPC Classification: H04W -004/22), Korean Registered Patent Publication No. 1874123 (name of the invention: a vehicle and a mobile terminal communicating with it, publication date: June 27, 2018, IPC classification: H04W-004 / 00) is disclosed.

The present invention is to solve the above-mentioned problems of the prior art, characterized in that it is executed through the cellular network-based communication connection between the smartphone. In addition, the present invention can provide notification to subsequent vehicles of the emergency situation of the front vehicle, even if the accident situation due to a direct collision, by utilizing a black box installed in the vehicle and the user terminal possessed by the user, a special device for the vehicle In addition, it is possible to make an emergency communication connection with surrounding vehicles without installing or installing additional equipment.

As a technical means for achieving the above technical problem, according to an embodiment of the present invention, a method for detecting and notifying a dangerous situation of a surrounding vehicle based on a communication connection and an image analysis between a terminal and a building CCTV camera performed by a terminal includes: a) machine learning a pattern of a measurement value of a gyroscope sensor and an acceleration sensor of a user terminal in which an emergency notification application is installed; (b) receiving a signal for a specific change of the user's vehicle from the gyroscope sensor and the acceleration sensor; (c) comparing the specific change with a preset reference value to determine whether an emergency situation exists; (d) When it is determined that the emergency situation, the real-time video recorded at the time when the signal for the specific change is received from the black box in the user vehicle, and the CCTV of the building that is located near the user vehicle Receiving a real-time image photographed from a camera; And (e) if it is determined that the dangerous situation is based on the real-time video, the server transmits a signal about the dangerous situation to the server, and the server provides a live notification of the emergency notification application among all other user terminals existing within a preset range. Providing a warning notification to another user terminal periodically transmitting and receiving a signal; Including, wherein the pattern is a pattern for the change amount of the gyroscope sensor and the change amount of the acceleration sensor collected while the user vehicle is driving, the step (a), the measurement of the gyroscope sensor in an emergency situation Principal component analysis (PCA) collects values, patterns of the acceleration sensor, and patterns of the gyroscope sensor and the acceleration sensor when the vehicle is not in an emergency. The measurement method of the gyroscope sensor and the dimensions of the elements of each direction constituting the measured value of the accelerometer sensor are reduced, and the measured value of the reduced gyroscope sensor and the measured value input value of the acceleration sensor are reduced. And set the result value of whether it is an emergency or not as the output information to perform machine learning based on the supervised learning model. Wherein the black box includes two cameras arranged to photograph the front of the user's vehicle and configured to be spaced apart by a predetermined interval, wherein step (e) is based on a stereo vision algorithm and is collected from the two cameras. Determining a distance between the user vehicle and an obstacle in front of the vehicle based on a real time image, and determining whether there is a collision according to the distance; And determining the type of the obstacle in front of the user's vehicle and the distance to the obstacle based on the real time image of the black box, and determining the distance between the vehicle and the adjacent obstacle in the image based on the real time image of the CCTV camera. If the speed of the user vehicle measured based on the gyroscope sensor, the acceleration sensor, or the GPS sensor is greater than or equal to a preset speed in comparison with the determined distances, determining a dangerous situation; Including, but the value for the distance and the predetermined speed pair for determining as the dangerous situation may vary depending on whether the type of the obstacle is human or animal.

In the step (c), when the measured value of the acceleration sensor and the measured value of the gyroscope sensor indicate a change amount of more than a predetermined change amount in a specific direction for a time within 1 second, it may be determined as an emergency.

In the step (b), the specific change is a change in any one or more of rapid deceleration, rapid acceleration, and rapid driving direction change of the user vehicle, and may be a change in the amount of change measured by the gyroscope sensor and the acceleration sensor. have.

In the step (e), if it is determined that the risk is not based on the real-time image, the pattern for the specific change is stored, and if a signal of the same pattern is received later, the black box image is not checked and the non-emergency It can be judged as a situation.

In the step (c), the specific change is compared with a preset reference value, and the pattern of the measured value of the gyroscope sensor and the measured value of the acceleration sensor determined as an emergency situation, the position information of which the pattern is measured, and the Transmitting a pattern of the measured value of the gyroscope sensor, the measured value of the acceleration sensor, and the positional information on which the pattern is measured, which is not determined to be a dangerous situation; It may further include.

In addition, the server matches and stores the location information and the pattern to form a map for a pattern for each location, extract an accident risk zone based on the location of the pattern on the map, and the accident on the map Danger zones can be marked and provided to other user terminals.

In addition, the other user terminal, if the emergency notification application is running in the background and is not displayed on the screen of the other user terminal, if it is determined that the driving based on the measured values of the acceleration sensor and the gyroscope sensor, live A signal can be continuously transmitted to the server, and an ack signal for the live signal of the server can be received.

In the step (e), the feature point of the real-time image of the black box and the image received through the camera of the user terminal is extracted, the focal length is compensated and converted into the corrected real-time image, the camera of the user terminal is The vehicle may be mounted at a specific position in the center fascia of the vehicle to photograph the front of the user vehicle.

In this case, the feature point may be a feature point about any point on the bonnet edge of the user vehicle and any object on the bonnet in the real time image and the image received through the camera of the user terminal.

In the step (e), the horizontal extension line of the feature point about the edge of the bonnet of the vehicle existing in the image received through the camera and the real-time image of the black box, respectively, and the horizontal extension line of the feature point about any object on the bonnet. In order to coincide with each other, the image received through the camera and the real time image of the black box may be aligned and synchronized, and then the real time image of the black box may be corrected with reference to the image received through the camera.

The server may further include a table that defines, for each animal, the speed of the user vehicle that the animal can avoid about the relationship between the speed of the user vehicle based on the reflection speed and the distance to the plurality of animals. Can be created and saved

The present invention utilizes a gyroscope sensor and an acceleration sensor of a smart phone, obtains an image of a black box, and analyzes the image through an image analysis in order to enable emergency communication between vehicles even without a separate device or facility in the vehicle. It can detect dangerous situations and provide notifications to nearby vehicles.

It is possible to collect not only the notification of accidents, but also information about sudden start and stop of vehicles, and by acquiring and analyzing black box images, it provides notifications to nearby vehicles about various emergency or dangerous situations on the road. It may be possible to.

1 is an exemplary diagram of a system for performing a method of detecting and notifying a dangerous situation of a surrounding vehicle based on a communication connection and an image analysis between a terminal and a building CCTV camera according to an embodiment of the present invention.
2 is a flowchart illustrating a method for detecting and notifying a dangerous situation of a surrounding vehicle based on a communication connection and an image analysis between a terminal and a building CCTV camera according to an embodiment of the present invention.
3 is an exemplary view illustrating a dangerous situation for explaining a method for detecting and notifying a dangerous situation of a surrounding vehicle based on a communication connection and an image analysis between a terminal and a building CCTV camera according to an embodiment of the present invention.
4 is a flowchart illustrating a method for detecting and notifying a dangerous situation of a surrounding vehicle based on a communication connection and an image analysis between a terminal and a building CCTV camera according to an embodiment of the present invention.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and like reference numerals designate like parts throughout the specification.

Throughout the specification, when a part is "connected" to another part, it includes not only "directly connected" but also "electrically connected" with another element in between. . In addition, when a part is said to "include" a certain component, this means that it may further include other components, except to exclude other components unless otherwise stated.

In the present specification, the term 'unit' includes a unit realized by hardware, a unit realized by software, and a unit realized by both. In addition, one unit may be realized using two or more pieces of hardware, and two or more units may be realized by one piece of hardware. Meanwhile, '~' is not limited to software or hardware, and '~' may be configured to be in an addressable storage medium or may be configured to reproduce one or more processors. Thus, as an example, '~' means components such as software components, object-oriented software components, class components, and task components, and processes, functions, properties, procedures, and the like. Subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided within the components and the '~' may be combined into a smaller number of components and the '~' or further separated into additional components and the '~'. In addition, the components and '~' may be implemented to play one or more CPUs in the device or secure multimedia card.

The "terminal" referred to below may be implemented as a computer or a portable terminal capable of connecting to a server or another terminal via a network. Here, the computer is, for example, a laptop equipped with a web browser (desktop), desktop (desktop), laptop (laptop), VR HMD (for example, HTC VIVE, Oculus Rift, GearVR, DayDream, PSVR, etc.) It may include. Here, VR HMD can be used for PC (e.g. HTC VIVE, Oculus Rift, FOVE, Deepon, etc.) and mobile (e.g. GearVR, DayDream, Stormscape, Google Cardboard, etc.) and console (PSVR) This includes all stand-alone models (eg Deepon, PICO, etc.) that are implemented independently. The portable terminal is, for example, a wireless communication device that ensures portability and mobility, and is not only a smart phone, a tablet PC, and a wearable device, but also Bluetooth (BLE, Bluetooth Low Energy), NFC, RFID, and Ultrasonic (Ultrasonic). It may include a variety of devices equipped with a communication module, such as infrared, Wi-Fi (WiFi), LiFi (LiFi). In addition, "network" refers to a connection structure capable of exchanging information between respective nodes such as terminals and servers, and includes a local area network (LAN), a wide area network (WAN), and the Internet. (WWW: World Wide Web), wired and wireless data communication networks, telephone networks, wired and wireless television networks, and the like. Examples of wireless data networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi, Bluetooth communications, Infrared communications, Ultrasound Communication, visible light communication (VLC), liFi (LiFi) and the like, but are not limited thereto.

A program (or application) for performing a communication connection between the terminal and the building CCTV camera and a method for detecting and alerting a surrounding vehicle danger situation based on image analysis may include a memory for storing the processor and a processor for executing the above program. The processor may perform various functions according to the execution of the program stored in the memory.

Hereinafter, a program for performing a method for detecting and notifying a dangerous situation of a surrounding vehicle based on a communication connection and an image analysis between a terminal and a building CCTV camera is referred to as an emergency notification application.

Hereinafter, referring to FIG. 1, a system 10 for performing a method for detecting and notifying a dangerous situation of a surrounding vehicle based on a communication connection and an image analysis between a terminal and a building CCTV camera according to an embodiment of the present invention will be described in detail. Explain.

According to an embodiment of the present invention, the system 10 for performing a method for detecting and notifying a dangerous situation of a surrounding vehicle based on a communication connection and an image analysis between a terminal and a building CCTV camera includes a server 100 and a user terminal 200. , A black box 300 installed in the user vehicle and another user terminal 400 (only one is shown in the drawing, but a plurality of other user terminals may exist), and the user terminal 200 and the user vehicle The black box 300 installed in the may be connected through a network, and the server 100 and the user terminal 200 and the other user terminal 400 may be connected through a network.

According to an embodiment of the present invention, the server 100 for performing the surrounding vehicle dangerous situation detection and notification method may receive a signal for a dangerous situation from the user terminal 200, and from the user terminal 200. When receiving a signal for a dangerous situation, it is possible to provide an alert notification to other user terminals periodically transmitting and receiving live signals for emergency notification applications among all other user terminals existing within a preset range.

In addition, the server 100 for performing the surrounding vehicle dangerous situation detection and notification method according to an embodiment of the present invention, by matching and storing the location information and the pattern, to form a map for the pattern for each location, on the map The accident risk zone may be extracted based on the location where the pattern is generated, and the accident risk zone may be marked on the map and provided to the user terminal 200 and the other user terminal 400.

When a user having a specific terminal installed with an emergency notification application drives a vehicle, the user may drive various roads, and a specific road may include a road where an emergency occurs frequently. In this case, according to an embodiment of the present invention, the determination of the emergency may be performed, and when the determination of the emergency is performed from the terminal more than a preset number of times, the server 100 may determine the emergency situation in the corresponding area or branch. Determining the frequent occurrence point, the terminal or the emergency situation notification application is installed to the danger zone for the corresponding area or point can be provided on the map and alert notification, such as accident risk zone.

According to an embodiment of the present invention, the user terminal 200 for performing the surrounding vehicle dangerous situation detection and notification method, from a sensor provided in the user terminal 200 in which the emergency notification application is installed, the specific change of the user vehicle Receives a signal for, and compares a specific change with a predetermined reference value, to determine whether or not an emergency situation, if it is determined that the emergency situation, a signal for a specific change from the black box 300 in the user's vehicle Received real-time video recorded at the time of receiving, and based on the real-time video, if it is determined that the risk situation, the server 100 may transmit a signal for the dangerous situation.

In this case, the user terminal 200 may include a gyroscope sensor, an acceleration sensor, and a GPS.

According to an embodiment of the present invention, a black box 300 in a user vehicle for performing a method for detecting and notifying a dangerous situation of a surrounding vehicle, may be configured to photograph two fronts of the user vehicle and may be spaced apart from each other by a predetermined interval. It may include.

Also, the image received from the black box 300 in the user vehicle is a real time image collected from two cameras, and the distance between the user vehicle and the obstacle in front of the user terminal 200 is determined based on a stereo vision algorithm. This may be possible.

According to an embodiment of the present invention, the other user terminal 400 is based on the measured values of the acceleration sensor and the gyroscope sensor even when the emergency notification application is executed in the background and is not displayed on the screen of the other user terminal. If it is determined that the vehicle is driving, the live signal may be continuously transmitted to the server 100 and the ack signal for the live signal of the server 100 may be received.

Hereinafter, with reference to FIG. 2, a method for detecting and notifying a dangerous situation of a surrounding vehicle based on a communication connection and an image analysis between a terminal and a building CCTV camera according to an embodiment of the present invention will be described in detail.

When a sensor provided in the user terminal 200 in which an emergency notification application is installed detects a change amount of more than a preset change amount, a specific change of a user vehicle may be detected (510).

According to an embodiment of the present invention, the user terminal 200 receives a signal indicating that a specific change is detected from the sensor, and compares the pattern with respect to the change amount of the previously stored gyroscope sensor and the change amount of the acceleration sensor, and in an emergency situation. It may be determined whether or not (520).

When it is determined that the pattern is not an emergency as a result of the pattern comparison, the pattern may be stored as a non-emergency pattern so that the pattern may be used as machine learning data (570).

When it is determined that the emergency situation is determined as a result of the pattern comparison, the black box 300 may request the recorded real-time image at the time when the signal for the specific change is received, and receive the real-time image (530).

According to an embodiment of the present invention, the user terminal 200 may analyze the received real-time image to determine whether a dangerous situation (540).

At this time, the image analysis, based on the stereo vision algorithm, based on the images collected from the two cameras provided in the black box 300, to determine the distance to the obstacle, based on the distance, whether the call or collision risk You can make a judgment about the situation.

In addition, the user terminal 200 may receive a real-time image photographed from a CCTV camera of a building in a location adjacent to the user vehicle, and may consider the risk situation based on the image. In detail, the user terminal 200 may receive an image of a CCTV camera of a structure such as a building CCTV camera or other street lamps located nearby. For example, some of the CCTV cameras may be equipped with a communication module that can communicate with the user terminal 200. The user terminal may transmit a live signal to the surroundings while a preset application is executed, and when the CCTV camera receives it, may transmit an ACK signal (including camera identification information and time) to the user terminal. Thereafter, the user terminal 200 may transmit a query to a predetermined server based on the ACK signal or transmit the query to the CCTV camera to request and receive an image captured by a camera having the corresponding identification information at a corresponding time.

In this case, an image including a user vehicle may be presented in the CCTV camera image unlike the black box image. Therefore, it may be a basic data for judging whether a dangerous situation is seen from an external objective point of view than a black box image. For example, by determining the distance between a vehicle (a user's vehicle) and an obstacle close to the real-time image of the CCTV camera, a distance value that can compensate for the distance measurement by the stereo vision of the black box can be obtained.

In this case, when it is determined that the image analysis is not a non-collision or a dangerous situation, the corresponding pattern may be stored as a non-emergency pattern to be used as machine learning data (570).

When it is determined that the image analysis result is a collision or a dangerous situation, the user terminal 200 may transmit a danger signal to the server 100 (550). In this case, when the speed of the user vehicle measured based on the gyroscope sensor, the acceleration sensor, and the GPS sensor is more than a preset speed in preparation for the distances measured in the black box image and the CCTV camera image, it may be determined as a dangerous situation.

In this case, the signal may be provided in a form including any one or more of text, an image, a video, and a sound.

The server 100 receiving the dangerous signal from the user terminal 200 may provide the received signal to another user terminal 400, including a warning notification, and all other existing within a preset range. The user terminal may periodically provide a live signal for an emergency notification application among other user terminals to another user terminal 560.

Hereinafter, a method for detecting and notifying a dangerous situation of a surrounding vehicle based on a communication connection and an image analysis between a terminal and a building CCTV camera will be described in detail with reference to FIG. 3.

A, B, C and D vehicles in Figure 3 are all users having a terminal with an emergency notification application is installed.

For example, it is assumed that vehicle A has an accident with the preceding vehicle, and that vehicles B, C, and D are driving in a preset range in the rear according to an embodiment of the present invention.

The user terminal 200 in which the emergency notification application is installed in the vehicle A detects a change in rapid deceleration and is determined to be a change amount greater than or equal to a preset reference value, and receives a signal for a change in the black box 300. Recorded real-time video can be received. In addition, it is also possible to receive the image of the CCTV from the adjacent CCTV camera or from the server connected to the CCTV camera.

Based on the stereo vision algorithm based on the video analysis results and CCTV video analysis results, it is determined that a crash has occurred, the user terminal 200 in the vehicle A, the server 100, the information about the accident and the real-time video The server 100 may receive the corresponding signal and provide the signal to another user terminal 400 in which emergency notification applications of B, C, and D vehicles are installed.

In a further embodiment of the present invention, by analyzing the image of the black box 300, it may be possible to confirm the accident within the visible distance of the camera of the black box 300, not the user vehicle.

For example, vehicle A does not have a terminal with an emergency notification application, there is a terminal with an emergency notification application installed only in vehicles B, C, and D, and vehicle B has an accident site within a line of sight. , C and D vehicles are assumed to be out of sight.

When the accident site is within the visible distance, the vehicle B may suddenly change the driving direction to prevent a collision, and at this time, the sensor of the user terminal 200 may detect the change in the driving direction.

Subsequently, it is determined that the amount of change is greater than or equal to a predetermined reference value, and the real time image recorded at the time when the signal for the change is received by the black box 300 may be received, and the image analysis result B based on the stereo vision algorithm for the real time image. Although the vehicle did not collide, the collision of the vehicle A is confirmed, and information and a real-time image of the accident are provided to the server 100, and the server 100 receives the corresponding signal, and the emergency situation of the C and D vehicles. The notification application may be provided to the other user terminal 400 installed.

By the above method, even when it is difficult to secure the visibility due to heavy fog or bad weather, it may be possible to prepare for an emergency in the driving road.

Hereinafter, with reference to FIG. 4, the step-by-step flow of the method for detecting and alerting the dangerous situation of the surrounding vehicle based on the communication connection and image analysis between the terminal and the building CCTV camera according to an embodiment of the present invention will be described in detail.

According to an embodiment of the present invention, a method for detecting and notifying a dangerous vehicle situation based on a communication connection and an image analysis between a terminal and a building CCTV camera may include identifying a user's vehicle from a sensor provided in a user terminal having an emergency notification application. A signal for change may be received (S610).

Here, the specific change is a change in any one or more of rapid deceleration, rapid acceleration, and rapid driving direction change of the user vehicle, and may be a change to the amount of change measured by the gyroscope sensor and the acceleration sensor.

In addition, before the step (S610), it is possible to perform a machine learning the pattern of the measurement value of the gyroscope sensor and the acceleration sensor in the user terminal 200, the measurement value of the gyroscope sensor and the acceleration sensor The pattern for the measured value may be a pattern for the amount of change in the gyroscope sensor and the amount of change in the acceleration sensor collected while the user's vehicle is traveling.

Specifically, the pattern of the measured value of the gyroscope sensor and the acceleration sensor in the emergency situation, the pattern of the measured value of the gyroscope sensor and the acceleration sensor in the non-emergency situation is collected, Principal Component Analysis (PCA) technique can reduce the dimensions of each direction element constituting the measured value of the gyroscope sensor and the acceleration sensor.

The PCA method sets the measured value of the reduced-size gyroscope sensor and the measured value of the acceleration sensor as input information, and sets the result value for the emergency situation as output information to perform machine learning based on the supervised learning model. can do.

Thereafter, by comparing the received signal for a specific change with a preset reference value, it may be determined whether the emergency situation.

The determination of the emergency situation may be determined to be an emergency when the measured value of the acceleration sensor and the measured value of the gyroscope sensor show a change amount of more than a predetermined change amount in a specific direction for a time within 1 second.

If it is determined that the emergency situation, the user terminal 200 may request the real-time video recorded at the time of receiving a signal for a specific change to the black box 300 in the user vehicle, and receive the corresponding real-time video ( S630).

The black box 300 existing in the user's vehicle may be provided to include two cameras arranged to photograph the front of the user's vehicle and separated by a predetermined interval.

When it is determined that the risk situation is based on the real-time image received in operation S630, a signal about the risk situation may be transmitted to the server 100 (S640).

At this time, based on the stereo vision algorithm, the distance between the user vehicle and the obstacle in front of the vehicle based on the real-time images collected from the two cameras, it is possible to determine whether there is a collision, according to the distance.

As a further embodiment of the present invention, the user terminal 200 may include an image processing apparatus, and it may be possible to correct an image based on a stereo vision algorithm to accurately determine a distance from an obstacle.

When the camera of the user terminal 200 is fixed to a specific position in the center fascia of the vehicle by using a cradle or the like, a user terminal camera and two black box cameras (total three) may be provided. Can be. Each terminal and the black box may extract feature points by performing a scale-invariant feature transform (SIFT) on the acquired camera image.

At this time, the image obtained from the two cameras of the black box 300, the merged one image 'A', the image obtained from the camera of the fixed user terminal 200 through a cradle to shoot the front of the vehicle It will be described in detail below assuming 'B'. Meanwhile, the 'A' image may be a merged image composed of only common pixels of the image photographed by each camera.

The feature point set of the image A extracted through image processing is

The feature point set of image B extracted through image processing is

라고, 가정하면,

과

은 순서에 따라, 서로 대응하는 특징점을 포함할 수 있다.

Assuming,

and

According to the order, may include feature points corresponding to each other.

It may be possible to perform the compensation by identifying the value of the feature point based on the vertical direction, and in this case, the feature point set may be changed to an identifier according to the vertical direction.

The image processing apparatus may determine the distance between the corresponding specific feature points in the image A and the image B. The distance between the specific feature points in the image A is called a first distance, and the distance between the specific feature points equal to the image a in the image B is determined. Assuming that the distance is the second distance, the distance between the specific feature points corresponding to the image A and the image B may correspond to the distance between the first distance and the second distance (the location of the same object) corresponding to each other in the image. .

The image processing apparatus may compensate for the focal length of the image based on the difference between the first distance and the second distance.

For example, the user terminal 200 finds the feature points of the images A and B, and the two points where the feature points indicating the edges of the bonnet of the vehicle and the horizontal extension lines of the feature points about any object photographed above the bonnet of the vehicle are connected to each other. Two images may be corrected to correspond to each other in the two images. That is, the image processing apparatus may be corrected such that the image A and the image B are parallel to each other based on the information on the specific object displayed on the screen.

Since the first camera and the second camera are disposed in different positions, the image A and the image B basically have different viewpoints. The image processing apparatus may perform region removal.

The user terminal 200 may perform calibration on the black box 300 image having the focal length compensated, and the calibration may be performed by converting an image to parallel the epipolar line in the image A and the image B. it means.

Then, the object may be extracted from the image of the black box 300, and the distance between the object and the vehicle may be calculated.

In addition, the type of the obstacle in front of the user vehicle and the distance from the obstacle is determined based on the real-time image, and the speed of the user vehicle measured based on the gyroscope sensor, the acceleration sensor, and the GPS sensor is greater than or equal to the preset speed against the distance. In this case, it can be judged as a dangerous situation.

At this time, the value for the distance and the preset speed pair for judging a dangerous situation may vary depending on whether the type of obstacle is a human or an animal, and the type of obstacle may be present in various forms, as well as rocks, rocks, and trees. When the vehicle is located on the road on which the vehicle is traveling, such as a structure, the driving direction may be changed or all obstacles may be impaired.

For example, the reflection speed of an animal (Cheeta: 32m / s, Rabbit: 21m / s, etc.) is stored in advance, and the relationship between the reflection speed of each animal, the visible distance of the vehicle light that the animal is visible while driving, and the speed of the vehicle are shown. Learning about the vehicle, a table defining each vehicle's speed that the animal can avoid can be generated and stored. For example, when the vehicle's light is 40m (the distance between the vehicle and the cat is 40m), the minimum vehicle speed that the cat can avoid is 60km / h, and the minimum vehicle speed that the cheetah can avoid It can be 100km / h.

By referring to this, it is possible to determine whether a dangerous situation is based on the speed according to the animal, and which kind of the animal belongs to the image in the image based on the supervised or non supervised learning algorithm. Learning may be performed to enable judgment.

Thereafter, the server 100 may provide a warning notification to other user terminals periodically transmitting and receiving live signals for emergency notification applications among all other user terminals existing within a preset range (S650).

In addition, when it is not determined as a dangerous situation based on a real time image, a pattern for a specific change may be stored, and when a signal of the same pattern is received later, the black box image may not be checked and may be determined as a non-emergency situation. .

That is, a pattern for a specific change that is not determined as a dangerous situation based on the real time image may be data for machine learning.

In operation S520, a specific change is compared with a preset reference value, and the pattern and pattern of the measured value of the gyroscope sensor and the acceleration sensor determined as the emergency situation are determined as the measured position information and the dangerous situation. It is possible to transmit the measured value of the gyroscope sensor and the pattern of the measured value of the acceleration sensor and the position information of the measured pattern to the server.

As a further embodiment of the present invention, the user terminal 200 and the other user terminal 400 may be directly connected to the network.

For example, even when the emergency notification application is executed in the background and is not displayed on the screen of the user terminal 200, if it is determined that the vehicle is driving based on the measured values of the acceleration sensor and the gyroscope sensor, the live signal may be sent to the server ( 100 and continuously receive the ack signal for the live signal of the server 100, in this process, if the ack signal for the live signal of the server 100 is not received, the user terminal 200 And the other user terminal 400 may be directly connected to the network.

That is, in an environment where the network with the server 100 is not possible, a network group is formed with other user terminals existing within a predetermined radius around each terminal, and when a dangerous situation occurs, a signal for a dangerous situation is generated within the group. It may be possible to provide.

One embodiment of the present invention may also be implemented in the form of a recording medium containing instructions executable by a computer, such as a program module executed by the computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, computer readable media may include all computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

Although the methods and systems of the present invention have been described in connection with specific embodiments, some or all of their components or operations may be implemented using a computer system having a general purpose hardware architecture.

The foregoing description of the present invention is intended for illustration, and it will be understood by those skilled in the art that the present invention may be easily modified in other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is shown by the following claims rather than the detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included in the scope of the present invention. do.

10: system
100: server 200: user terminal
300: black box in the user's vehicle 400: other user terminal

Claims (11)

In the communication between the terminal and the building CCTV camera performed by the terminal and the surrounding vehicle risk situation detection and notification method based on image analysis,
(a) machine learning a pattern for the measured value of the gyroscope sensor and the acceleration sensor in the user terminal in which the emergency notification application is installed;
(b) receiving a signal for a specific change of the user's vehicle from the gyroscope sensor and the acceleration sensor;
(c) comparing the specific change with a preset reference value to determine whether an emergency situation occurs;
(d) If it is determined that the emergency situation, the real-time video recorded at the time of receiving the signal for the specific change from the black box in the user vehicle, and receives a CCTV of the building located near the user vehicle Receiving a real-time image photographed from a camera; And
(e) When it is determined that the risk situation is based on the real-time images, the server transmits a signal about the risk situation, and the server transmits a live signal for the emergency notification application among all other user terminals existing within a preset range. Providing a warning notification to another user terminal periodically transmitting and receiving a message; Including but not limited to:
The pattern is,
It is a pattern for the change amount of the gyroscope sensor and the change amount of the acceleration sensor collected while the user's vehicle is driving,
In step (a),
Collect the pattern of the measurement value of the gyroscope sensor and the acceleration sensor of the emergency situation, the pattern of the gyroscope sensor and the measurement value of the acceleration sensor in the case of non-emergency situation while driving ,
Principal Component Analysis (PCA) is used to reduce the dimension of each direction element constituting the measured value of the gyroscope sensor and the measured value of the acceleration sensor,
Set the measured value of the reduced gyroscope sensor and the measured value input information of the acceleration sensor, and set the result value of whether or not the emergency situation as output information to perform machine learning based on the supervised learning model. ,
The black box,
Includes two cameras arranged to shoot the front of the user's vehicle and configured at a predetermined interval apart,
In step (e),
Based on a stereo vision algorithm, determining a distance between the user vehicle and an obstacle in front of the vehicle based on real-time images collected from the two cameras, and determining whether there is a collision according to the distance; And
The type of the obstacle in front of the user's vehicle and the distance to the obstacle based on the real-time image of the black box, and the distance between the vehicle and the adjacent obstacle in the image based on the real-time image of the CCTV camera, the gyro When the speed of the user vehicle measured based on the scope sensor, the acceleration sensor, and the GPS sensor is greater than or equal to a preset speed in comparison with the determined distances, determining a dangerous situation; Including but not limited to:
The value for the distance and the value for the predetermined speed for determining the dangerous situation is to vary depending on whether the type of the obstacle is human or animal,
Method of detecting and notifying the dangerous situation of the surrounding vehicle based on the communication connection and image analysis between the terminal and the building CCTV camera.
The method of claim 1,
In step (c),
If the measured value of the acceleration sensor and the measured value of the gyroscope sensor indicates a change amount more than a predetermined change amount in a specific direction for a time within 1 second, it is determined that the emergency situation,
Method for detecting and notifying the dangerous situation of the surrounding vehicle based on the communication connection and image analysis between the terminal and the building CCTV camera.
The method of claim 1,
In step (b),
The specific change is,
Change of the change amount measured by the gyroscope sensor and the acceleration sensor by at least one change of the rapid deceleration, rapid acceleration and rapid running direction of the user vehicle,
Method for detecting and notifying the dangerous situation of the surrounding vehicle based on the communication connection and image analysis between the terminal and the building CCTV camera.
The method of claim 1,
In step (e),
If it is not determined as a risk based on the real-time video,
Storing the pattern for the specific change, and if a signal of the same pattern is received later, without determining the black box image, it is determined to be a non-emergency situation,
Method for detecting and notifying the dangerous situation of the surrounding vehicle based on the communication connection and image analysis between the terminal and the building CCTV camera.
The method of claim 4, wherein
Step (c) is,
The specific change is compared with a preset reference value, and the pattern of the measured value of the gyroscope sensor and the acceleration sensor determined as an emergency situation, the positional information where the pattern is measured, and the risk not determined as the dangerous condition. Transmitting a pattern of a gyroscope sensor, a pattern of the measured value of the acceleration sensor, and position information of the measured pattern to the server;
To further include,
Method for detecting and notifying the dangerous situation of the surrounding vehicle based on the communication connection and image analysis between the terminal and the building CCTV camera.
The method of claim 5, wherein
The server matches and stores the location information and the pattern to form a map for a location-specific pattern,
Extracting an accident risk zone based on the location of the pattern on the map,
To indicate to the accident risk zone on the map to provide to other user terminal,
Method for detecting and notifying the dangerous situation of the surrounding vehicle based on the communication connection and image analysis between the terminal and the building CCTV camera.
The method of claim 1,
If the other user terminal is determined to be driving based on the measured values of the acceleration sensor and the gyroscope sensor even when the emergency notification application is executed in the background and is not displayed on the screen of the other user terminal, Continuously transmitting to the server and receiving an ack signal for the live signal of the server,
Method for detecting and notifying the dangerous situation of the surrounding vehicle based on the communication connection and image analysis between the terminal and the building CCTV camera.
The method of claim 1,
In step (e),
Extract the feature points of the real-time image of the black box and the image received through the camera of the user terminal, compensate for the focal length and convert to a corrected real-time image,
The camera of the user terminal is mounted on a specific position in the center fascia of the vehicle to photograph the front of the user vehicle,
Method for detecting and notifying the dangerous situation of the surrounding vehicle based on the communication connection and image analysis between the terminal and the building CCTV camera.
The method of claim 8,
The feature point,
It is a feature point for any point on the bonnet edge of the user vehicle and any object on the bonnet in the real-time image and the image received through the camera of the user terminal,
Method for detecting and notifying the dangerous situation of the surrounding vehicle based on the communication connection and image analysis between the terminal and the building CCTV camera.
The method of claim 9,
In step (e),
The horizontal extension line of the feature point about the edge of the bonnet of the vehicle existing in the image received through the camera and the real time image of the black box and the horizontal extension line of the feature point about any object on the bonnet coincide with each other; After aligning and synchronizing the image received through the real-time image of the black box, and correcting the real-time image of the black box with reference to the image received through the camera,
Method for detecting and notifying the dangerous situation of the surrounding vehicle based on the communication connection and image analysis between the terminal and the building CCTV camera.
The method of claim 1,
The server,
Generating and storing a table for each animal that defines the speed of the user vehicle that the animal can avoid about the relationship of the speed of the user vehicle based on the reflection speed and the distance to a plurality of animals ,
Method for detecting and notifying the dangerous situation of the surrounding vehicle based on the communication connection and image analysis between the terminal and the building CCTV camera.

Images