KR20210141043A - Crosswalk traffic accident notification system based on image analysis - Google Patents

Abstract

The present invention relates to a crosswalk traffic accident notification system based on image analysis, which provides a caution signal to a pedestrian crossing or waiting to cross a crosswalk, when the speed of a vehicle traveling on a road exceeds a predetermined speed, if a vehicle traffic light is a red or yellow light. The crosswalk traffic accident notification system based on image analysis, according to the present invention, comprises: a camera unit including a plurality of cameras for generating image data by imaging the pedestrian or the vehicle in the crosswalk, the road, or a safety zone; a control unit for detecting the risk of the pedestrian by calculating the image data generated by the camera unit; and a speaker unit for generating an alarm by the control unit, when the control unit detects the risk of the pedestrian.

Description

Crosswalk risk notification system based on image analysis {CROSSWALK TRAFFIC ACCIDENT NOTIFICATION SYSTEM BASED ON IMAGE ANALYSIS}

The present invention relates to a crosswalk hazard notification system, and more particularly, to a crosswalk hazard notification system based on image analysis.

In general, a crosswalk is installed for pedestrians to cross on a road where vehicles pass, and it is installed on the road and along with a traffic light for vehicles to recognize whether a vehicle can drive or not, on both sides of the crosswalk, it is necessary to check whether pedestrians can cross. Pedestrian traffic lights to recognize the situation are added.

Since these crosswalks and traffic lights simply display control signals for pedestrians crossing or driving vehicles, the driver of a vehicle traveling on the roadway may cause an accident if the identification of pedestrians crossing the road is delayed, which may put the pedestrian in a dangerous state. problems may arise.

In addition, pedestrians waiting for a crossing tend to be psychologically adjacent to the road in a hurry, and as they get out of the safe zone within the safety line and approach the road, they act as a major cause of traffic accidents. In particular, the elderly or children who lack the ability to judge the situation often cross the safety line and wait, and in the case of the visually impaired, it is difficult to recognize traffic signals, which creates a more dangerous situation.

On the other hand, many people are using mobile communication devices, including smartphones, while walking, and despite the fact that special attention is required while walking at a crosswalk, they are exposed to a lot of risks in a situation where there are no sanctions or guidance for this. is agreed upon by everyone.

In particular, when a mobile communication device is used while walking on the street, the distance from which the sound can be heard and the field of view are reduced, increasing the risk of an accident. In fact, if a pedestrian walks while using a mobile communication device, they do not even know that a vehicle is running, and they are only looking at the mobile communication device in their hand and have an accident while walking while looking at the mobile communication device. Cases of this are common.

As described above, walking on the street while looking at a mobile communication device is exposed to the risk of an accident, but the number of traffic accidents occurring while walking while looking at a mobile communication device is increasing, so urgent measures are required.

Republic of Korea Patent Publication No. 10-1672685 (2016. 11. 03. Announcement)

An object of the present invention for solving the above problems is to analyze in real time an image captured by a vehicle traveling on a road by a camera, and if the vehicle traffic light is a red or yellow light, the speed of the vehicle traveling on the road is greater than or equal to a certain speed It is to provide an image analysis-based crosswalk hazard notification system that can provide a caution signal to pedestrians waiting to cross or cross the crosswalk when they cross.

Another object of the present invention is to analyze an image captured by a pedestrian in a crosswalk or a safety zone by a camera in real time, and a safety line formed on the sidewalk in contact with the crosswalk and a pedestrian waiting in a safety zone formed within the safety line To provide an image analysis-based crosswalk hazard notification system that recognizes abnormal behavior of pedestrians and provides an alarm when abnormal behavior of pedestrians is recognized.

Another object of the present invention is to analyze the image captured by a pedestrian in a crosswalk or a safety zone by a camera in real time, and when it is a crosswalk walking impossible signal, based on a plurality of virtual lines on the sidewalk in contact with the crosswalk, crossing The goal is to provide an image analysis-based crosswalk hazard notification system that can provide an alarm when a pedestrian approaches a virtual line close to the sidewalk.

In order to achieve the above object, an image analysis-based crosswalk risk notification system according to an embodiment of the present invention is a plurality of image data generated by imaging a pedestrian or a vehicle in a crosswalk, a road, or a safety zone. A camera unit including a camera of, a control unit for detecting the risk of the pedestrian by calculating the image data generated by the camera unit, and a speaker unit for generating an alarm by the control unit when the control unit detects the danger of the pedestrian do.

The control unit, the communication unit receiving the signal of the pedestrian traffic light or the vehicle traffic light operated on the crosswalk or the road, respectively, and the camera unit analyze the image data captured by the vehicle on the road Vehicle speed to calculate the speed of the vehicle Further comprising a calculation unit, wherein the vehicle speed calculation unit, when the signal of the vehicle traffic light is a red signal or a yellow signal, when the speed of the vehicle does not decrease by a preset value, it is characterized in that it transmits an alarm signal toward the speaker unit do.

The camera unit includes a plurality of road cameras for imaging the vehicle on the road, wherein the road cameras include a first road camera for imaging a first frame and the second camera to have a time difference with the first frame and a second road camera that captures a second frame closer to the crosswalk than one frame, wherein the vehicle speed calculator includes a virtual virtual camera that is respectively formed in the image data captured in the first frame and the second frame, respectively. It is characterized in that the speed of the vehicle is calculated based on a difference in data on the number of pixels occupied by the vehicle, which varies according to a predetermined time in the box.

The camera unit includes a sidewalk camera unit for imaging the crosswalk and the safety area in the sidewalk, and the control unit is close to the crosswalk among the virtual lines based on a plurality of virtual lines of the safe area adjacent to the crosswalk. It is characterized in that it further comprises a pedestrian calculation unit for detecting the danger of the pedestrian as the pedestrian is positioned on the virtual line, and when the danger of the pedestrian is detected according to the pedestrian calculation unit, the size of the alarm or the frequency of the alarm is changed.

The control unit, when the pedestrian traffic light is a red signal, determines the presence of a pedestrian in the crosswalk and the safety zone in the sidewalk, detects an abnormal behavior of the pedestrian, and generates an alarm through the speaker unit do.

According to the present invention's image analysis-based crosswalk danger notification system, when the vehicle traffic light is a red or yellow light, there is an effect that the pedestrian can be notified of the presence of a vehicle entering the crosswalk by ignoring the signal.

According to the present inventors' image analysis-based crosswalk danger notification system, there is an effect that a danger signal can be provided by recognizing abnormal behavior of pedestrians in a safe area.

According to the present inventor's image analysis-based crosswalk hazard notification system, a danger signal can be notified to a pedestrian approaching the road outside the safety zone within the safety line, thereby preventing major causes of traffic accidents.

1 is a block diagram schematically showing the configuration included in the image analysis-based crosswalk danger notification system according to the present invention.
2 is a view showing an embodiment in which a camera unit and a speaker unit are installed on a pillar member according to the present invention.
3 is a view showing a state in which the image analysis-based crosswalk danger notification system according to the present invention is installed near an intersection.
4 is a view exemplarily illustrating a process in which the vehicle speed calculator according to the present invention measures the speed of a vehicle traveling on a road.
5 is a view showing a state in which a plurality of road camera units according to the present invention capture images of different areas, respectively.
6 is a view showing a state in which the sidewalk camera unit according to the present invention captures a pedestrian in a crosswalk and a safety area.

Hereinafter, some embodiments of the present invention will be described in detail with reference to exemplary drawings. In adding reference numerals to components in each drawing, it should be noted that the same components are given the same reference numerals as much as possible even though they are indicated in different drawings.

In the description of the embodiment of the present invention, if it is determined that a detailed description of a related known configuration or function interferes with the understanding of the embodiment of the present invention, the detailed description thereof will be omitted.

In addition, in describing the components of the embodiment of the present invention, terms such as first, second, A, B, (a), (b), etc. may be used. These terms are only for distinguishing the components from other components, and the essence, order, or order of the components are not limited by the terms.

Hereinafter, the present invention will be described in more detail with reference to the accompanying drawings.

1 is a block diagram schematically showing the configuration included in the image analysis-based crosswalk danger notification system 1000 according to the present invention.

1, the image analysis-based crosswalk danger notification system 1000 according to the present invention is a camera including a plurality of cameras that generate image data by imaging a pedestrian or vehicle in a crosswalk, a road, or a safety zone When the unit 100, the control unit 200 that detects the danger of the pedestrian by calculating the image data generated by the camera unit 100, and the control unit 200 detects the danger of the pedestrian, the control unit 200 ) includes a speaker unit 300 for generating an alarm.

Each component included in the image analysis-based crosswalk danger notification system 1000 according to the present invention may be provided integrally in a single module, but is not limited thereto, for example, the camera unit 100 and the control unit ( 200 is provided integrally so as to be included in a single module and is installed on a pillar member 10 , which will be described later, extended on the road, and the speaker unit 300 is spaced apart from the camera unit 100 and the control unit 200 . Thus, it may be separately installed on the pillar member 10 installed on the sidewalk, which will be described later. Of course, only the camera unit 100 is separately installed on the pillar member 10 extending on the road, and the camera unit 100 and the control unit 200 are spaced apart from the camera unit 100 and installed on the sidewalk pillar member 10 ) that may be installed separately will be understood by those skilled in the art to which the present invention pertains. The camera unit 100 , the control unit 200 , and the speaker unit 300 may be connected to each other to transmit and receive electrical signals. The control unit 200 may be configured integrally with the camera unit 100 or the speaker unit 300 .

2 is a view showing an embodiment in which the camera unit 100 and the speaker unit 300 according to the present invention are installed on the pillar member 10 .

Referring to FIG. 2 , as an embodiment of the image analysis-based crosswalk danger notification system 1000 according to the present invention, the image analysis-based crosswalk danger notification system 1000 according to the present invention has an upper end It is formed to have a certain height toward the direction, and may further include a pillar member 10 having the other end installed on the ground. The pillar member 10, as described above, the other end is installed on the ground, the installation method can use a known method. For example, the other end of the pillar member 10 may be fixed by being inserted into the interior of the ground by a predetermined depth. One end of the pillar member 10, as described above, may be formed to be spaced apart from the ground so as to have a certain height upward. Preferably, one end of the pillar member 10 may be formed to extend to a point higher than the total height of all vehicles capable of running on the road.

The pillar member 10 according to the present invention may be installed on a sidewalk, and when the pillar member 10 is installed on the sidewalk, one end of the pillar member 10 is the camera unit 100 . For convenience of operation, it may be bent to have a predetermined angle (preferably, 90 degrees) on the road.

Alternatively, the pillar member 10 according to the present invention is a first pillar member 11 and a first pillar member formed so that the other end is fixed to a predetermined depth into the interior of the ground, and one end is formed to have a predetermined height toward the upper side. The camera unit 100 according to the present invention is coupled to a specific position of (11) (preferably a position higher than the total height of all vehicles capable of running on the road) and is formed to extend a predetermined length on the road and is located on the road. It may include a second pillar member 12 is installed. In this case, the speaker unit 300 may be installed on the first pillar member 11 . A wire member 13 connecting one end of the second pillar member 12 and one end of the first pillar member 11 to support the second pillar member 12 may be further included.

Alternatively, as an embodiment of the image analysis-based crosswalk hazard notification system 1000 according to the present invention, the camera unit 100, the control unit 200 and the speaker unit 300 according to the present invention are installed on the road. It may be installed together with the pillar member 10 of the traffic light.

3 is a view showing a state in which the image analysis-based crosswalk danger notification system 1000 according to the present invention is installed near an intersection.

Referring to FIG. 3 , the image analysis-based crosswalk hazard notification system 1000 according to the present invention may be installed near a road including an intersection.

The pillar member 10 according to the present invention, as described above, may be installed on the sidewalk, and may be formed to extend from the sidewalk onto the road. When the pillar member 10 extending from the sidewalk onto the road is referred to as the second pillar member 12 , the camera unit 100 is installed at a point where the second pillar member 12 is installed to extend on the road. When the second pillar member 12 extends onto the road, the extending direction may be extended to be parallel to the crosswalk (ie, perpendicular to the road), and as shown in FIG. 3 , the center of the intersection may be extended to When the second pillar member 12 extends toward the center of the intersection, there is an effect that an object such as a pedestrian or a vehicle can be imaged in all crosswalk directions installed at the intersection only with a single camera unit 100 .

The camera unit 100 according to the present invention includes a plurality of cameras, and each camera may be installed by adjusting the direction of the lens toward the crosswalk installed at the intersection. For example, as shown in FIG. 3 , in the case of a four-branch intersection, the camera unit 100 is installed in four directions, respectively, in the first to fourth directions toward the four crosswalks. It may further include a camera unit. Hereinafter, the camera unit 100 according to the present invention will be described with only the first camera unit, but all contents regarding the first camera unit are equally applied to the second to fourth camera units, and the first camera unit and the lens Only the direction they are facing can be different.

The camera unit 100 according to the present invention generates image data by imaging a pedestrian or vehicle in a crosswalk, a road, or a safety zone. The image data captured by the camera unit 100 can be calculated in real time, to be described later, a danger signal for a pedestrian or a vehicle entering the crosswalk by the control unit 200 . The camera unit 100 includes a plurality of cameras.

Each of the plurality of cameras included in the camera unit 100 according to the present invention may be an RGB camera that takes an image as an RGB image. In addition, the camera unit 100 is a motion detector camera having a function of detecting a moving object in a safe area and a crosswalk, an infrared camera having a heat detection function, a camera with a radar, and a pan and tilt camera IP communication for performing automatic tracking. It may further include at least one of network cameras capable of performing the operation.

A plurality of cameras included in the camera unit 100 according to the present invention is a road camera unit 120 and a safety zone (S) and a crosswalk in which the direction of the lens is directed toward the road in order to image a vehicle traveling on the road. The direction of the lens may include a plurality of camera units 110 for sidewalks facing the direction including the safety zone (S) and the crosswalk (C) in order to image the pedestrian in the zone (C). The camera unit 100 may convert each captured image into image data. The control unit 200 may further include a memory unit (not shown) for storing image data captured by the camera unit 100, and the image data captured by the camera unit 100 may be transmitted to and stored in the memory unit. The present invention is not limited thereto, and for the image data captured by the camera unit 100 , the risk of pedestrians may be calculated in real time by the calculation unit 220 , which will be described later.

First, the sidewalk camera unit 110 according to the present invention includes a sidewalk camera for imaging the area (C) including the crosswalk. The sidewalk camera captures not only the crosswalk, but also the crosswalk and the adjacent sidewalk (S). The sidewalk (S) adjacent to the crosswalk includes a safety area (S) in which pedestrians waiting for a signal to cross the sidewalk can be safely located. The sidewalk camera unit 110 may generate image data by capturing the area C including the crosswalk and the safety area S together.

The road camera unit 120 according to the present invention may include a plurality of road cameras. In this case, the road camera unit 120 is, as an embodiment, the first road camera unit 120 and the first frame P1 that take images of a vehicle traveling on the road in the first frame P1 as an object. ) and may include a second road camera unit 120 for capturing a second frame (P2) closer to the crosswalk than the first frame (P1) to have a time difference. The camera unit 100 according to the present invention includes a third road camera unit 120 that captures a third frame closer to the crosswalk than the second frame P2 to have a time difference with the second frame P2. Of course, more may be included.

The first frame P1 of the area imaged by the first road camera unit 120 according to the present invention and the second frame P2 of the area imaged by the second road camera unit 120 have a time difference . That is, a vehicle driving on the road and approaching a crosswalk, that is, driving to pass an intersection, first passes the first frame P1, passes the first frame P1, and then passes the second frame P2. do.

A crosswalk warning sign N may be included between the first frame P1 and the second frame P2 according to the present invention. That is, the positions of the first frame P1 and the second frame P2 may be determined with the crosswalk warning sign N interposed therebetween. The crosswalk warning sign (N) is generally located at a point 50m behind the crosswalk to inform the vehicle driver that there is a crosswalk ahead. It is common for the driver to decelerate the vehicle after recognizing the crosswalk warning sign (N). Therefore, the vehicle speed calculated in the first frame P1 before the crosswalk warning indication (N) and the second frame P2 after the crosswalk warning indication (N) is the first frame P1 in the second frame P2. ), it can be expected that it will be a safer situation for pedestrians crossing the crosswalk. On the other hand, if the speed of a vehicle or vehicle traveling with the same speed is accelerated more in the second frame P2 than in the first frame P1 even after the crosswalk warning sign (N), a dangerous situation for pedestrians in the near future may occur. can be expected to occur. The process of calculating the vehicle speed in each frame P1 and P2 will be described later with reference to FIGS. 4 and 5 .

The control unit 200 according to the present invention further includes a communication unit 210 that receives a signal of a pedestrian traffic light or a vehicle traffic light operated on a crosswalk or road, respectively. A traffic signal controller capable of controlling the signal system of the traffic light is installed near the traffic light. The traffic signal controller changes the signal system of the intersection according to the pre-programmed result. The communication unit 210 is connected to the traffic signal controller to receive changes in current signals and future signals. The communication unit 210 receives a signal of a pedestrian traffic light or a vehicle traffic light from the traffic signal controller.

The control unit 200 according to the present invention further includes an operation unit 220 for calculating the image data generated by the camera unit 100 capturing each image. The calculation unit 220 calculates the image data captured in each designated area by the sidewalk camera unit 110 and the vehicle camera unit 120 to detect the danger of pedestrians.

The calculation unit 220 according to the present invention includes a vehicle speed calculation unit 223 for calculating the speed of the vehicle entering the crosswalk by analyzing the image data captured by the vehicle on the road by the road camera unit 120 . do.

The vehicle speed calculation unit 223 according to the present invention receives signal information of a vehicle traffic light from the communication unit 210 . When the current signal of the vehicle traffic light is a red or yellow signal, the vehicle speed calculation unit 223 analyzes the image data generated by the road camera unit 120 to recognize the speed of the vehicle traveling on the road. As described above, each road camera unit 120 captures images of a vehicle in motion by using a specific point on the road as a specific frame, respectively.

4 is a view exemplarily illustrating a process in which the vehicle speed calculator 223 measures the speed of a vehicle traveling on a road according to the present invention.

A process in which the vehicle speed calculating unit 223 according to the present invention measures the speed of a vehicle traveling on a road will be described with reference to FIG. 4 .

As shown in FIG. 4 , the road camera unit 120 according to the present invention captures a vehicle as an object when a vehicle traveling on the road enters a specific frame P. The image data obtained by the road camera unit 120 capturing the vehicle may be calculated by the vehicle speed calculating unit 223 in real time.

The vehicle speed calculating unit 223 according to the present invention may draw a virtual box B in the frame P of the image data generated by the road camera unit 120 . When the vehicle enters the frame P, the vehicle speed calculation unit 223 prepares to measure the vehicle speed. Preparation for measuring the speed of the vehicle includes counting the time from the moment the vehicle in the frame P enters. The vehicle speed calculation unit 223 calculates the time from the vehicle entering the frame P until the vehicle entering the frame P enters the virtual box B drawn by the vehicle speed calculation unit 223. can be measured The vehicle speed calculation unit 223 determines the time from when the vehicle enters the frame P until it enters the virtual box B as a reference value.

The vehicle speed calculator 223 according to the present invention calculates the number of pixels that the current vehicle occupies in the virtual box B after a reference value elapses after the vehicle enters the virtual box B. . If the number of the first pixels calculated by the vehicle speed calculating unit 223 is the first pixel number A1, the vehicle speed calculating unit 223 calculates the first pixel number A1 and then the time by the reference value is further increased. After passing, the number of pixels occupied by the current vehicle in the virtual box B is calculated again. The number of pixels calculated again by the vehicle speed calculator 223 may be referred to as the second pixel number A2. The vehicle speed calculating unit 223 compares the value obtained by calculating the difference between the first pixel number A1 and the second pixel number A2 calculated in this way and the value of the first pixel number A1 . The difference between the first pixel number A1 and the second pixel number A2 is the speed change of the vehicle after the first pixel number A1 is calculated. If the difference between the first pixel number A1 and the second pixel number A2 is greater than the first pixel number A1, it means that the speed of the vehicle entering the current frame is increased. Conversely, if the difference between the first pixel number A1 and the second pixel number A2 is smaller than the first pixel number A1, it means that the speed of the vehicle entering the current frame P has decreased. When the vehicle speed calculated by the vehicle speed calculation unit 223 increases, the control unit 200 may generate an alarm toward the speaker unit 300 .

A vehicle that has entered the frame P of the road camera unit 120 according to the present invention is a vehicle that will soon enter a crosswalk, and the vehicle speed calculation unit 223 operates only when the vehicle traffic light is a red or yellow signal. . Therefore, if the speed of the vehicle entering the frame P does not decrease even though the vehicle traffic light is a red or yellow signal, it may mean that a dangerous situation will occur for pedestrians who will cross the crosswalk. The control unit 200 transmits an alarm signal to generate an alarm toward the speaker unit 300 when the vehicle speed calculation unit 223 detects a pedestrian danger.

The speaker unit 300 according to the present invention, as described above, is installed on the pillar member 10 installed in the sidewalk, particularly the first pillar member 11 . That is, the speaker unit 300 is installed to be close to the pedestrian in the crosswalk and the safety zone S in the sidewalk, and the pedestrian hears an alarm generated through the speaker unit 300 and enters the crosswalk quickly without reducing the speed. be aware that this exists. On the other hand, the area imaged by the road camera unit 120 according to the present invention may be set closer to or farther from the crosswalk. The reduction value of the vehicle speed may be preset according to the location of the area imaged by the road camera unit 120 . A smaller reduction value may be set if the imaging area is set further from the crosswalk, whereas a larger reduction value may be set if the imaging area is set closer to the crosswalk. Meanwhile, the alarm generated by the speaker unit 300 may be pre-recorded music, voice, or a specific signal.

5 is a view showing a state in which a plurality of road camera units 120 according to the present invention image different areas, respectively.

Meanwhile, referring to FIG. 5 , the road camera unit 120 according to the present invention may include a plurality of road camera units 120 as described above. The plurality of road camera units 120 are respectively installed in different positions of the regions where images are captured.

As described above with reference to FIG. 3, the area imaged further from the crosswalk is called the first frame P1, and the area imaged with a time difference from the first frame P1 is called the second frame P2. Then, the vehicle speed calculation unit 223 may calculate the speed change of the vehicle entering from the first frame P1 and then calculate the speed change of the same vehicle in the second frame P2 .

The method for the vehicle speed calculating unit 223 according to the present invention to calculate the speed change of the vehicle is as described above. That is, using the time from when the vehicle enters the first frame P1 until the vehicle enters the virtual box B in the first frame P1 as a reference value, the virtual box ( Calculate the speed of the vehicle by calculating the number of pixels occupied by the vehicle in B). In the second frame P2, the vehicle speed may be calculated in the same manner as in the first frame P1, and the reference value used in the second frame P2 may be the reference value calculated in the first frame P1. .

That is, the vehicle speed calculating unit 223 according to the present invention is the number of pixels occupied by the vehicle in the virtual box B during the time corresponding to the same reference value in the first frame P1 and the second frame P2. calculate The vehicle speed calculator 223 may compare the speed change of the vehicle calculated in the first frame P1 with the speed change of the vehicle calculated in the second frame P2, and if the vehicle speed change calculated in the second frame P2 is If the speed change of the vehicle is greater than the speed change of the vehicle calculated in the first frame P1, this may mean that a dangerous situation will occur for pedestrians walking in a crosswalk or waiting. As described above, when the control unit 200 recognizes that a dangerous situation will occur by the vehicle speed calculation unit 223 , it may transmit a signal to generate an alarm signal through the speaker unit 300 .

In the road camera unit 120 according to the present invention, only a single road camera unit 120 may be used, or a plurality of road camera units 120 may be used. When only a single road camera unit 120 is used to calculate the speed of the vehicle in a single frame to detect a pedestrian danger, and a plurality of road camera units 120 are used to measure the speed of the vehicle in a plurality of frames. When a pedestrian risk is detected by calculation, a difference may occur in the alarm method by the speaker unit 300 . For example, when only a single road camera unit 120 is used, an alarm having the same frequency and tone may be generated, and a plurality of road camera units 120 are used to generate a plurality of frames P1 and P2. ), when the danger of pedestrians is detected by calculating the speed of the vehicle in stages, an alarm having a changing frequency, an alarm sound level, and a changing tone may be generated.

6 is a view showing a state in which the sidewalk camera unit 110 according to the present invention captures images of pedestrians in a crosswalk and a safety area.

Referring to FIG. 6 , the camera unit 100 according to the present invention includes a sidewalk camera unit 110 for imaging a crosswalk and a safety area in the sidewalk. The safe zone (S, see FIG. 3 ), as described above, means a zone in which pedestrians wait to cross the sidewalk using the crosswalk.

The sidewalk camera unit 110 according to the present invention takes images of the entire crosswalk and pedestrians in safety zones located at both ends of the crosswalk as objects, and generates image data captured by the pedestrians.

The calculation unit 220 included in the control unit 200 according to the present invention analyzes the image data of the safe area scratched by the sidewalk camera unit 110 and the area (C) including the crosswalk to crosswalk and safety It further includes a pedestrian calculation unit 221 for recognizing the presence of pedestrians between the zones, and detecting the danger of pedestrians.

The pedestrian calculating unit 221 according to the present invention analyzes the image data generated by the sidewalk camera unit 110 . The pedestrian calculation unit 221 may draw a plurality of virtual lines D1 and D2 in the safety area adjacent to the crosswalk. The virtual lines D1 and D2 may be drawn in a direction perpendicular to the direction of the crosswalk formed to connect the two sidewalks, as an embodiment. That is, the virtual lines D1 and D2 may be drawn in a direction parallel to the road.

The pedestrian calculation unit 221 according to the present invention may draw a plurality of virtual lines D1 and D2, for example, if there are two virtual lines D1 and D2, as shown in FIG. 6 , any one (D2) can be drawn close to the crosswalk, and another (D1) can be drawn farther from the crosswalk. If an imaginary line drawn farther from the crosswalk is called a first imaginary line D1, and a imaginary line D2 drawn closer to the crosswalk is called a second imaginary line, the pedestrian calculation unit 221 performs crossing in the safety zone. Whether the position of the waiting pedestrian is waiting away from the 1st virtual line D1, or whether it is waiting to pass through the 1st virtual line D1 and to come into contact with the 2nd virtual line D2, or the 1st virtual line (D1) and the second imaginary line (D2) pass both to determine whether the waiting.

The pedestrian calculating unit 221 according to the present invention may receive the signal of the pedestrian traffic light from the communication unit 210 and may operate when the pedestrian traffic light is currently red. The pedestrian calculation unit 221 is, when the pedestrian traffic light is currently red, when the pedestrian passes the first virtual line D1 and waits to come into contact with the second virtual line D2, and the first virtual line D1 and When all of the second virtual line D2 is passed and waiting, the danger of pedestrians is sensed. When the pedestrian calculating unit 221 detects a risk of a pedestrian, the control unit 200 transmits a signal for generating an alarm toward the speaker unit 300 . When the control unit 200 generates an alarm toward the speaker unit 300, the pedestrian's position passes the first virtual line D1 and waits to come into contact with the second virtual line D2 and the first virtual line It can be said that it is preferable that the alarms in the case of waiting by passing both (D1) and the second virtual line (D2) are different from each other. That is, even if the detection of the pedestrian's danger is the same, the case where the pedestrian's position crosses both the first virtual line D1 and the second virtual line D2 and waits passes the first virtual line D1 to the second It can be determined that it is more dangerous than the case of waiting to come into contact with the virtual line D2.

The controller 200 according to the present invention is more dangerous than when it is determined that the current pedestrian's position is more dangerous, that is, when the pedestrian passes both the first virtual line D1 and the second virtual line D2 and waits. You can set the alarm to have a large alarm and a frequency. Also, there may be a change in tone, and may change to a higher tone.

The pedestrian calculation unit 221 according to the present invention can detect an abnormal behavior in the safety zone S while classifying the danger according to the position of the pedestrian waiting in the safety zone S. As described above, the pedestrian calculation unit 221 may operate when the pedestrian traffic light is a red signal, and may determine and recognize the presence of pedestrians in the zone C including the safety zone and the crosswalk. When the pedestrian traffic light is a red signal and there is a pedestrian in the crosswalk, the control unit 200 generates an alarm to give an auditory attention effect to the pedestrian and at the same time to give an auditory attention effect to a vehicle running normally on the road. can

The sidewalk camera unit 110 according to the present invention includes a motion detector camera having a function of detecting a moving object in a safe area and a crosswalk, an infrared camera having a heat detection function, a camera with a radar, and a fan for performing automatic tracking. It may further include at least one of the network cameras capable of performing tilt camera IP communication.

The pedestrian calculation unit 221 according to the present invention may further include an object detection unit and an abnormal behavior detection unit. The object detection unit may detect a change in the position of the pedestrian in the zone C including the safety zone and the crosswalk. As an embodiment, the object detecting unit detects the pedestrian by analyzing the movement trajectory of the pedestrian. In addition, the object detection unit patterns the image data to determine the presence of a pedestrian when a pedestrian is detected in the safety zone based on the safety zone. The presence or absence of pedestrians is performed based on shape information and feature information. Pedestrian location tracking performs a block-based matching algorithm based on the shape control points extracted according to the occurrence of object occlusion to track the pedestrian, or tracks the pedestrian by a block matching algorithm based on the center of gravity or a block matching algorithm based on periodic update. can do.

Specifically, the pedestrian calculator 221 generates a background image based on continuously input image frames, and updates the background image according to whether or not the pedestrian moves detected from the image frames, and the background image and input Steps of detecting a rectangle including all blocks with differences between image frames as a movement area, extracting a shape control point (SCP) from the detected movement area, and extracting a shape if object occlusion does not occur Based on the control points, the block matching algorithm (BMA) is performed to track the pedestrian, and when object occlusion occurs, the center-of-gravity-based Block Matching Algorithm (CBMA) and The method may include tracking the pedestrian by any one algorithm among Periodic-update-based Block Matching Algorithms (PBMA).

In the step of updating the background image, if the value obtained using Equation 1 below is less than a preset threshold, the background image is updated using Equation 2 below.

Here, (u, v) denotes horizontal and vertical coordinates, (mx, my) denotes a motion vector, and (p, q) denotes horizontal and vertical sizes of an image block.

Here, B(t) represents the background at time t, I(t) represents the input image at time t, and σ represents the mixing ratio experimentally set in the range from 0 to 1. Accordingly, tracking of pedestrians can be continued with only the verified shape control points, thereby being free from potential failure factors such as temporal spatial similarity between the pedestrian and the background, object deformation, and occlusion.

The abnormal behavior detecting unit according to the present invention performs pattern analysis by detecting irregular loitering when a specific pedestrian moves continuously in an unspecified moving trajectory for a certain period of time within the safety zone. In the case of loitering and abnormal behavior detection, by detecting abnormal situations such as loitering that may occur within the safe zone area based on the change in the position of the pedestrian detected by the difference between the image frame and the background image, it is possible to determine whether a specific abnormal situation occurs. effective unattended monitoring.

The abnormal behavior detection unit, as an embodiment, detects an optical flow detected between a current image frame and a previous image frame temporally preceding the current image frame among a plurality of image frames received from the reporting camera unit 110 . A background image generation step of generating a background image corresponding to the current image frame based on the background image generation step, a moving object detection step of detecting the movement of a pedestrian from the current image frame by the difference between the background image and the current image frame, and a temporal step until the current image frame Based on the movement trajectory, which is the change in the movement position of the pedestrian detected equally from consecutive image frames with It is divided into a plurality of blocks of the size set in It may include an abnormal behavior detection step.

Additionally, it is possible to detect a still pedestrian based on the difference between the background image and the current image frame and the difference between the past background image and the current image frame, which is a background image generated in response to the image frame preceding the current image frame by a preset reference time. , if the position of the still pedestrian is the same on a preset reference number of image frames that are temporally continuous up to the current image frame, it can be estimated as a still pedestrian. Such a stationary pedestrian is estimated to correspond to a person who faints based on the invariant moment value. In addition, it is possible to estimate the behavior of a person who rushes toward a crosswalk from a safety zone.

The crosswalk danger notification system 1000 based on image analysis according to the present invention may further include a lighting unit (not shown) capable of generating a visual attention effect together with the auditory attention effect of the speaker unit 300 . The lighting unit may illuminate a crosswalk, a safety zone, and a road. In addition, the lighting unit may illuminate the crosswalk warning display (N). When the risk calculated by the pedestrian calculating unit 221 and the vehicle speed calculating unit 223 occurs, the lighting unit may operate together with the speaker unit 300 . The lighting unit can give a greater attention effect, especially to the driver who is controlling the vehicle. The lighting unit may include a known light emitting device or the like.

On the other hand, the image analysis-based crosswalk danger notification system 1000 according to the present invention is connected to the camera unit 100 and the long-distance transmission system by an external server that receives image data captured by the camera unit 100 . may further include. The external server may further include an external calculation unit 220 that analyzes the image data. The external operation unit 220 may analyze the image data in the same manner as described above to detect the danger of the pedestrian, and when the danger of the pedestrian is detected, the auditory or visual attention effect through the speaker unit 300 or the lighting unit may cause

In the present specification, each component of the control unit 200 for analyzing image data may include a main processing unit and a video card, and image data may be analyzed by a video card, or image data may include a main processing unit and a video card. It can be analyzed by card.

In the present specification, the controller 200 may be processors that execute consecutive execution processes stored in a memory. Alternatively, it may operate as software modules driven and controlled by a processor. Furthermore, the processor may be a hardware device.

In this specification, a "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. In addition, one unit may be implemented using two or more hardware, and two or more units may be implemented by one hardware.

The protection scope of the present invention is not limited to the description and expression of the embodiments explicitly described above. In addition, it is added once again that the protection scope of the present invention cannot be limited due to obvious changes or substitutions in the technical field to which the present invention pertains.

10: column member 11: first column member
12: second pillar member 13: wire member
100: camera unit 110: side camera unit
120: road camera unit 200: control unit
210: communication unit 220: calculation unit
221: pedestrian calculation unit 223: vehicle speed calculation unit
300: speaker unit
1000: Image analysis-based crosswalk danger notification system

Claims (5)

a camera unit including a plurality of cameras for generating image data by imaging a pedestrian or a vehicle in a crosswalk, a road, or a safety zone;
a control unit for calculating the image data generated by the camera unit to detect the danger of the pedestrian; and
a speaker unit for generating an alarm by the control unit when the control unit detects the danger of the pedestrian;
A crosswalk hazard notification system based on image analysis, characterized in that it comprises a. The method according to claim 1,
The control unit is
a communication unit receiving a signal of a pedestrian traffic light or a vehicle traffic light operated on the crosswalk or the road, respectively; and
a vehicle speed calculation unit for calculating the vehicle speed by analyzing the image data captured by the camera unit on the road vehicle;
further comprising,
The vehicle speed calculation unit,
When the signal of the traffic light for the vehicle is a red signal or a yellow signal, when the speed of the vehicle does not decrease by a preset value, an alarm signal is transmitted toward the speaker unit, an image analysis-based pedestrian crossing danger notification system. 3. The method according to claim 2,
The camera unit,
Containing; a plurality of road cameras for imaging the vehicle on the road;
The road camera,
a first road camera for capturing a first frame; and
a second road camera for capturing a second frame closer to the crosswalk than the first frame to have a time difference with the first frame;
includes,
The vehicle speed calculation unit,
Calculating the speed of the vehicle based on a difference in data on the number of pixels occupied by the vehicle that changes over a certain time in a virtual box formed in image data captured in the first frame and the second frame, respectively A crosswalk hazard notification system based on image analysis. 4. The method according to claim 3,
The camera unit,
Including; a sidewalk camera unit for imaging the crosswalk and the safety area in the sidewalk;
The control unit is
a pedestrian calculation unit for detecting a risk of a pedestrian as the pedestrian is located on a virtual line close to the crosswalk among the virtual lines based on a plurality of virtual lines of the safety zone adjacent to the crosswalk;
further comprising,
When a risk of a pedestrian is detected according to the pedestrian calculation unit, an image analysis-based crosswalk danger notification system, characterized in that the alarm size or frequency of the alarm is changed. 5. The method according to claim 4,
The control unit is
When the pedestrian traffic light is a red signal, it is determined whether there is a pedestrian in the crosswalk and a safety zone in the sidewalk, detects an abnormal behavior of the pedestrian, and generates an alarm through the speaker unit. Pedestrian crossing hazard notification system.

Images