RU165235U1 - ROAD SCENE RECOGNITION AND ANALYSIS SYSTEM - Google Patents

Abstract

A scene recognition and analysis system containing a visual representation of a road scene, the first and second information outputs of which are connected to the first and second information inputs of the output signal integration module, the first information output of which is the first information output of the system designed to output the road scene data to the screen the driver’s display, the second information output of the output signal integration module is the second information output of the systems intended for issuing the image of the road marking lines on the display screen of the vehicle driver, the first and second control outputs of the output signal integration module are the control outputs of the system designed to provide signals to the input of the steering gear control system, a module for representing the current position of the vehicle, the first and second information inputs of which are the first and second information inputs of the system, designed to receive codes with of the corresponding sensors of the angular position of the left and right front wheels of the vehicle, the third and fourth information inputs of the module for representing the current position of the vehicle are the third and fourth information inputs of the system, respectively, designed to receive the coordinates of the vehicle positioning point on the road scene, and the control input of the module for presenting the current vehicle position is the control input of the system, designed to

Description

The utility model relates to the field of computer vision, in particular, to a system for recognizing and analyzing a road scene.

The modern world of computer systems makes extensive use of machine vision, or computer vision technology. In particular, solutions designed for cars are gaining popularity, reducing the likelihood of accidents and simplifying the process of driving a vehicle.

The main operation in the operation of these systems is the recognition of road markings, since there is a tendency to increase the number of cases when drivers leave for markings, which leads to traffic accidents.

The lane assist system (other names: lane assist, lane keeping assist) helps the driver to keep to the selected lane and thereby prevent emergency situations.

The system is effective when driving on highways and equipped federal roads, that is, where there is high-quality road marking. There are two types of lane assistance systems: passive and active.

The passive system warns the driver about deviation from the selected lane, and the active system, along with a warning, makes corrective intervention in the steering operation.

In the process of functioning of the active lane assistance system, the following main functions are implemented:

- recognition of the trajectory (strip) of movement;

- visual information about the system;

- adjustment of the steering;

- driver warning.

Known technical solutions that could be used to build a system for analyzing the road situation and generating vehicle control signals (1, 2, 3).

The first known technical solution comprises one or more video cameras mounted on a vehicle with the ability to review the location of signs on the road, an automatic traffic sign recognition unit associated with a system for detecting traffic signs and traffic signals, control and memory units, and a video projector, and / or a display and / or audio device designed to display information about a traffic sign or traffic signal from the moment of detection to the moment the traffic sign or signal comes out etofora zone of action of the device [1].

When the vehicle is moving, the camcorder or camcorders scan the top and side of the road, where traffic signs and traffic lights are usually located. The image obtained using the video camera is processed in the unit for automatic recognition of traffic signs and traffic signals.

When traffic signs or traffic lights enter the field of view of video cameras, the automatic recognition unit for traffic signs and traffic signals provides a corresponding code signal about it to the driver informing unit, which issues a corresponding message about a traffic sign or traffic signal to a video device and / or audio device.

The disadvantage of this technical solution is the low speed of the system, due to the fact that the reaction of the driver of the vehicle to received messages about the state of the road situation requires additional time to make a decision.

Another system is known, comprising a unit for automatically recognizing traffic signs and traffic signals located on a vehicle and connected to a system for detecting traffic signs and traffic signals, control and memory units, as well as a video projector and / or display and / or audio device for display information about a traffic sign or traffic signal from the moment of detection until the moment the traffic sign or traffic signal leaves the coverage area [2].

According to the invention, the system for detecting traffic signs and traffic signals for automatic transmission to the vehicle interior is made in the form of a radio or sensor transmitter designed to be installed at traffic lights and traffic signs and broadcasting traffic flow limited in front of a code or traffic signal in the area of the code signals and a receiving device mounted on the vehicle associated with the automatic racer unit connected to the control unit cognition of traffic signs and traffic signals and intended for receiving signals from the transmitter.

Its disadvantage is the low speed of the system, due to the fact that the analysis of the situation in the area of observation of road signs is carried out through the search for signs of road signs and their subsequent recognition, which inevitably leads to unjustified costs and the inability to make decisions in real time.

Another technical solution of the problem is known, comprising means for detecting a lateral object, control means for activating the side control control, means for determining the start of entry into an adjacent lane, driving suppression means for suppressing the side control control . Moving to the side is accompanied by a change in the lateral position of the vehicle relative to the road lane. [3].

The last of the above technical solutions is closest to the described.

The disadvantage of this technical solution is its low reliability, due to the fact that in difficult weather conditions, the first and second digital cameras are also physically unable to solve the task set for him to identify the state of the road environment.

The purpose of the utility model is to increase the reliability of the system by identifying the position of the vehicle relative to the marking lines of the road scene and warning the driver about crisis situations.

This goal is achieved by the fact that in a known system containing a module for visual representation of the road scene, the first and second information outputs of which are connected to the first second information inputs of the output signal integration module, the first information output of which is the first information output of the system intended for the output of road scene data to the display screen of the vehicle driver, the second information output of the output signal integration module is the second information the system output intended for outputting the image of the road marking lines on the display screen of the vehicle driver, the first and second control outputs of the output signal integration module are the control outputs of the system intended for issuing signals to the input of the steering gear control system, a module for representing the current position of the vehicle , the first and second information inputs of which are both the first and second information inputs of the system, intended for codes from the respective sensors of the angular position of the left and right front wheels of the vehicle, the third and fourth information inputs of the module for representing the current position of the vehicle are the third and fourth information inputs of the system, respectively, designed to receive the coordinates of the vehicle positioning point on the road scene, and the control input module representing the current position of the vehicle is the control input of the system, designed To receive driver request signals, the first module for selecting a signal for changing the direction of movement of the vehicle, the clock input of which is connected to the first clock output of the module for visual representation of the road scene, the first signal input of the first module for selecting signals for changing the direction of movement of the vehicle is the first signal input of the system receiving data on the position of the rotation handle when it is installed in the left position, the second signal input of the first module The section of signals for changing the direction of movement of the car is the second signal input of the system, intended for receiving data on the position of the turn handle when it is set to the right position, the second module for selecting signals for changing the direction of movement of the vehicle, the clock input of which is connected to the second clock output of the module for visual representation of the road scene , the first signal input of the second module of the selection of signals for changing the direction of movement of the vehicle is connected to the second the system’s main input, and the second signal input of the second vehicle direction change signal selection module is connected to the first system signal input, while the control outputs of the first and second vehicle direction change signal selection modules are connected to the first and second control inputs of the output signal integration module and to the first and second control inputs of the module representing the current position of the vehicle, respectively, synchronizing in the course of the first module for selecting signals for changing the direction of movement of the vehicle is connected to the installation input of the second module for selecting signals for changing the direction of movement of the vehicle and with one clock input of the module for representing the current position of the vehicle, and the synchronizing output of the second module for selecting signals for changing the direction of the vehicle is connected to the installation the input of the first module of the selection of signals for changing the direction of movement of the transport means to another synchronizing input of the module for representing the current position of the vehicle, a module for determining the value of the signal for correcting the position of the vehicle is introduced, the information input of which is connected to the first information output of the module for representing the current position of the vehicle, the first and second synchronizing inputs of the module for determining the value of the signal for correcting the position of the vehicle means connected to the synchronizing outputs of the first and second signal selection modules changes in the direction of movement of the vehicle, respectively, and the information output of the module for determining the value of the signal for correcting the position of the vehicle is connected to the third information input of the integration module of the output signals, the identification module of the current sections of the marking of the road scene, the information inputs of which are connected to the second and third information outputs of the presentation module of the current vehicle positions, respectively, and the synchronizing input of the identification module is current of their sections of the marking of the road scene is connected to the synchronizing output of the module representing the current position of the vehicle, a memory module whose address input is connected to the address output of the identification module of the current sections of the marking of the road scene, the first synchronizing output of which is connected to the read input of the memory module, and the information output of the memory module connected to the fourth information input of the output signal integration module, a module for fixing the boundaries of the current sections of the road scene, information whose input is connected by the information output of the identification module of the current sections of the marking of the road scene, the synchronizing input is connected to the second synchronizing output of the identification module of the current sections of the marking of the road scene, the timing input is the clocking input of the system designed to receive signals from the measurement sensor of the path traveled by the vehicle, and the output the module for fixing the boundaries of the current sections of the road scene is the signal output of the system, designed to issue a signal warning the driver of the vehicle, the second timing output current sections of the road scene module boundaries fixing is connected to the clock input integration module outputs.

The invention is illustrated by drawings, where in FIG. 1 is a structural diagram of a system; FIG. 2 is a block diagram of a road scene visualization module 2, FIG. 3 is a structural diagram of a first module for selecting signals for changing a direction of movement of a vehicle, FIG. 4 is a structural diagram of a second module for selecting signals of changing a direction of movement of a vehicle, FIG. 5 is a block diagram of a module for presenting a current position of a vehicle; FIG. 6 is a structural diagram of a control signal magnitude selecting module; FIG. 7 is a block diagram of an identification module of a current marking section of a road scene; FIG. 8 is a block diagram of a module for fixing the boundaries of the current sections of the road scene; FIG. 9 is a block diagram of an output signal integration module.

The system (Fig. 1) contains a module 1 for visual representation of the road scene, the first 2 and second 3 modules for selecting signals for changing the direction of the vehicle, module 4 for presenting the current position of the vehicle, module 5 for determining the magnitude of the control signal, module 6 for identifying the current section of the road marking scenes, memory module 7, module 8 for fixing the boundaries of the current sections of the road scene, and module 9 for the integration of output signals.

The drawing shows the first 11, second 12, third 13, and fourth 14 information inputs of the system, the first 15 and second 16 signal inputs of the system, which receive signals from the sensors of the position of the handle of the rotation switch, control 17 and clock 18 inputs, as well as the first 19 and the second 20 information outputs of the system, the first 21 and second 22 control and signal 23 outputs of the system.

Module 1 (Fig. 2) of the visual representation of the road scene contains the first 25 and second 26 of the video camera, the first 27 and second 28 road marking image detectors. The drawing shows information 29, 30 and synchronizing 31, 32 outputs of the module. The video cameras 25, 26 are mounted on the vehicle so that the line marking the road scene from the left side of the traffic enters the field of view of the video camera 25, and the line marking the lane on the right side of the road enters the field of view of the video camera 26.

Module 2 (Fig. 3) of the selection of signals for changing the direction of movement of the vehicle contains triggers 35, 36, element 37 OR, elements 38, 39 I. The drawing shows timing 40, first 41 and second 42 signal and installation 43 inputs, as well as a control 44 and synchronizing 45 outputs.

Module 3 (Fig. 4) of the selection of signals for changing the direction of movement of the vehicle contains triggers 46, 47, element 48 OR, elements 49, 50 I. The drawing shows the timing 51, first 52 and second 53 signal and installation 54 system inputs, as well as control 55 and synchronizing 56 outputs.

Module 4 (Fig. 5) representing the current position of the vehicle contains registers 60-63, trigger 64, groups 65 to 68 of AND elements, group 69 of OR elements, and delay element 70. The drawing shows the first 11, second 12, third 13 and fourth 14 information inputs, control 17 input, first 71 and second 72 enable inputs, the first 73 and second 74 signal inputs, as well as the first 75, second 76, third 77 information and timing 78 exits.

The information inputs 11, 12 receive data on the angular position of the left front and right front wheels of the vehicle, and the information inputs 13, 14 receive the coordinates X, U of the vehicle positioning point obtained from the output of the navigator (not shown in the drawing).

Module 5 (FIG. 6) for determining the value of the vehicle position correction signal contains a decoder 80, read-only memory 81, register 82, AND elements 83-85, OR element 86, first 87 and second 88 delay elements. The drawing also shows information 89, first 90 and second 91 clock inputs, as well as information output 92.

Module 6 (Fig. 7) identifying the current sections of the marking of the road scene contains a decoder 93, read-only memory 94, register 95, elements 96-98 AND, elements 99-101. The drawing shows the first 102 and second 103 information, and clock inputs 104, as well as the first 105 and second 106 information, first 107 and second 108 clock outputs.

The module 7 (Fig. 1) memory is made in the form of random access memory having an address 133 and read 134 inputs, as well as information 135 output.

Module 8 (Fig. 8) for fixing the boundaries of the current sections of the road scene contains a register 110, a comparator 111, a counter 112, a trigger 113, an element 114 AND, an element 115 delay. The drawing shows information 116, synchronizing 117 and controlling 118 inputs, as well as signal 23 output.

Module 9 (Fig. 9) of the integration of the output signals contains a register 120, digital-to-analog converters 121-122, and groups 123 to 125 elements I. The drawing also shows the first 126, second 127, third 128, fourth 129 information, first 130 and second 131 control and synchronizing 132 inputs, as well as the first 19 and second 20 information, the first 21 and second 22 control outputs.

The system operates as follows.

Data from the information output 140 of the video camera 25 (Fig. 2) is fed to the information input 144 of the block 27, where the road marking is recognized and a decision is made on whether the marking intersects or not. To this end, in block 27, the following sequence of operations [4] is implemented:

- receiving data from a video camera;

- breakdown of the video stream into frames;

- removing the effect of perspective using the inverse mapping method of perspective;

- border enhancement using a Gaussian filter;

- vectorization of data by the Hough transform;

- drawing lines on the original image;

- displaying the result on the screen.

From the output 29 of the module 1, the processed information in the form of codes of marking lines plotted on the original image is fed to the input 126 of the module 9, the elements 125 And of the group pass through and through the output 19 of the module 9 it is then displayed on the display screen.

In the event that the vehicle begins to cross the marking, then at the output 31 of module 1, a signal pulse appears, which through the input 40 of module 2 is fed to one input of the elements 38 and 39 AND, which are controlled by trigger 35.

In the event that the vehicle begins to cross the markings, and the turn signal is not turned on, then trigger 35 will be in the initial state in which, with a high potential from the inverse output of the trigger 35, element 39 And will be open, and element 38 And low potential from the direct exit trigger 35 will be closed.

The signal pulse from the input 40 of module 2, firstly, passes through the open element 39 And to the output 45 of module 2, and, secondly, is fed to the direct input of the trigger 36 and sets it to a single state, in which the trigger 36 from the direct output gives output 44 a high potential coming through the input 71 of module 4 to one of the inputs of the 65 And group elements, and opens them at one input.

In addition, the high potential from the output 44 of the module 2 through the input 130 of the module 9 also goes to one of the inputs of the elements 123 And groups, opening them one input.

In parallel, the signal pulse of crossing the marking lines from the output 45 of module 2 is fed to the synchronizing input 72 of module 4, as a result of which the angle of rotation of the corresponding front wheel from the angle sensor (not shown) is entered into the register 60 of module 4 from input 11 .

This code from the output of the register 60 passes through the open elements 65 AND groups, then the elements 69 OR groups and from the output 75 of the module 4 enters through the input 89 of the module 5 to the input of the decoder 80, which decrypts the received code and generates a high potential at one of its outputs, opening one input of one of the elements 83-85 AND, for example, element 85 I.

In parallel, the synchronizing pulse from the input 90 of the module 5 passes the OR element 86, is delayed by the delay element 87 for the response time of the register 60 and the decoder 80, and from the output of the delay element 87 it goes to another element 85 AND, from where it goes to the input of the fixed memory block 81, which contains the code of the amplitude of the signal required to control the drive to return the vehicle to a given lane. This code is read into the information register 82.

In parallel with this, the synchronizing pulse from the output of the element 87 of module 5 is delayed by the element 88 for the time of reading the code from the memory unit 81, and is supplied to the synchronizing input of the register 82, entering the read code into it.

From the output of the register 82, the read code of the signal amplitude through the output 92 of the module 5 is fed to the input 128 of the module 9, and then through the elements of the And group 123 it is fed to the input of the digital-analog converter 121, from the output of which goes to the output of the system 21 and then to the input of the steering control system a mechanism that returns the front wheels of the car to its original state.

In the event that the vehicle begins to cross the markings, and the turn indicator is turned on, in this case, a signal is received from the turn signal sensor 15 from the turn signal sensor to the input 41 of module 2, which sets the trigger 35 to a single state, at which high potential from the direct output of the trigger 35 element 38 And will be open, and element 39 And low potential from the inverse output of the trigger 35 will be closed.

In this case, the signal pulse from the input 40 of module 2 passes through the open element 38 AND, then the element 37 OR, and enters the installation input of the trigger 35 and sets it to its original state.

In addition, the pulse from the input 15 of the system enters the input 53 of the module 3, and then passes through the element 48 OR to the installation input of the trigger 46, also setting it to its original state.

Similarly, the system works when processing data coming from the output of the second 26 cameras.

In parallel with this process, through the information 13 and 14 inputs of the system, the coordinates X, Y of the vehicle positioning point are received, respectively, which are issued to the information inputs of the respective registers 62 and 63 of module 4.

If it is necessary to clarify the position of the vehicle relative to the road marking lines, the driver of the vehicle issues a control signal that enters through the input 17 of the system to the single input of the trigger 64 of module 4, which goes into a single state, in which the high potential from its direct output opens the elements And groups 67, 68, which connect the outputs of the registers 62 and 63 through the outputs 76 and 77 of module 4 and through the inputs 102, 103 of module 6 to the inputs of the decoder 93.

The decoder 93 determines the identification number of the road section using the coordinate codes and, with a high potential, opens the corresponding element 96-98 I. At the same time, the synchronizing pulse from input 17, delayed by the delay element 70 for the duration of the trigger 64 and the decoder 93, firstly, arrives at the installation input of the trigger 64 and returns it to its original state, and, secondly, from the output 78 of the module 4 it goes to the input 104 of the module 6 and then passes through the corresponding element 96-98 And opened by the decoder 93, and then the post paet fixed to a respective memory cell of ROM 94 and reads the contents of the information input of the register 95.

At the same time, the synchronizing pulse from the input 104 of the module 6 is delayed by the element 99 for the time of reading data from the ROM 94, and is supplied to the synchronizing input of the register 95, entering the input data of the codogram into it.

The structure of the read codegram has the following form:

The address of the road marking image in this section from the output 105 of module 6 is fed to the address input 133 of the memory unit 7, to the input of which is read 134 a synchronizing pulse is received from the output 107 of the module 3.

As a result, the output 135 of the memory unit 7 through the input 129 of the module 9 to the information input of the register 120 of the module 9 receives a digital image of the marking lines of the road scene, which is entered into the register 120 with a synchronizing pulse from the input 132 of the module 9 and then displayed on the system output 20 display.

In parallel, the code of the quantitative value of the length of the section with the displayed image of the road marking lines from the output 106 of the module 6 through the information input 116 of the module 8 is fed to the information input of the register 110, which is entered by the synchronizing pulse from the input 117.

In addition, the synchronizing pulse from the input 117 of the module 8 is supplied to the direct input of the trigger 113 and sets it to a single state, in which the element 114 And will be opened with a high potential from the direct output of the trigger 113, to the other input 118 of which pulses from a measurement sensor of the path traveled by the vehicle (not shown in the drawing).

As the vehicle moves, pulses from the sensor of the traveled path through the element 114 OR arrive at the counting input of the counter 112, which records at each moment in time the movement of the vehicle along the selected route. The readings of the counter 112 are fed to one information input of the comparator 111, to the other information input of which the readings of the register 110 are received.

In parallel with the input to the counter input of the counter 112, the synchronizing pulse from the output of the element 114 And is delayed by the element 115 for the response time of the counter 112, and then it goes to the synchronizing input of the comparator 111. With the arrival of this synchronizing pulse, the comparator 111 compares the increasing readings of the counter 112 with the readings of the register 110, and as soon as the vehicle passes the specified section, the readings of the counter 112 will become equal to the readings of the register 110 and a signal appears at the output of the comparator 111, which, firstly, from the output The ode 23 of the system is issued on the driver warning board, and, secondly, it enters the installation input of the trigger 113, returning it to its original state.

Thus, the introduction of new modules and new structural connections has significantly improved the reliability of the system by identifying the position of the vehicle relative to the marking lines of the road scene and warning the driver about crisis situations.

Sources of information taken into account when drawing up the description of the application:

1. RF patent No. 2268499, 2003

2. RF patent No. 2369905, 2008

3. RF patent No. 2520855, 2010 (prototype).

4. Konushin A., Kinshakov V., Krylov A. Algorithms for detecting markings and pavement defects. - M.: Publishing House of Moscow State University, 2009

Claims (1)

A scene recognition and analysis system containing a visual representation of a road scene, the first and second information outputs of which are connected to the first and second information inputs of the output signal integration module, the first information output of which is the first information output of the system designed to output the road scene data to the screen the driver’s display, the second information output of the output signal integration module is the second information output of the systems intended for issuing the image of the road marking lines on the display screen of the vehicle driver, the first and second control outputs of the output signal integration module are the control outputs of the system designed to provide signals to the input of the steering gear control system, a module for representing the current position of the vehicle, the first and second information inputs of which are the first and second information inputs of the system, designed to receive codes with of the corresponding sensors of the angular position of the left and right front wheels of the vehicle, the third and fourth information inputs of the module for representing the current position of the vehicle are the third and fourth information inputs of the system, respectively, designed to receive the coordinates of the vehicle positioning point on the road scene, and the control input of the module for presenting the current vehicle position is the control input of the system, designed to receive driver request signals, the first module for selecting signals for changing the direction of movement of the vehicle, the clock input of which is connected to the first clock output of the module for visual representation of the road scene, the first signal input of the first module for selecting signals for changing the direction of movement of the vehicle is the first signal input of the system for receiving data on the position of the rotation handle when it is set to the left position, the second signal input of the first signal selection module s of changing the direction of movement of the car is the second signal input of the system, designed to receive data on the position of the crank when it is set to the right position, a second module for selecting signals of changing the direction of movement of the vehicle, the clock input of which is connected to the second clock output of the visual representation module of the road scene, the first signal input of the second module of the selection of signals for changing the direction of movement of the vehicle is connected to the second signal an ode to the system, and the second signal input of the second module for selecting signals of changing the direction of movement of the vehicle is connected to the first signal input of the system, while the control outputs of the first and second modules for selecting signals for changing the direction of movement of the vehicle are connected to the first and second control inputs of the module for integrating output signals and to the first and second control inputs of the module representing the current position of the vehicle, respectively, the synchronizing output of the first the module for selecting signals of changing the direction of movement of the vehicle is connected to the installation input of the second module for the selection of signals for changing the direction of movement of the vehicle and with one synchronizing input of the module for representing the current position of the vehicle, and the synchronizing output of the second module for selecting signals for changing the direction of the vehicle is connected to the installation input of the first a module for selecting signals to change the direction of the vehicle and to another synchronizing input of the module representing the current position of the vehicle, characterized in that the system comprises a module for determining the magnitude of the signal for correcting the position of the vehicle, the information input of which is connected to the first information output of the module for representing the current position of the vehicle, the first and second synchronizing inputs of the module for determining the magnitude of the correction signal vehicle positions are connected to the synchronizing outputs of the first and second modules selection of signals for changing the direction of movement of the vehicle, respectively, and the information output of the module for determining the value of the signal for correcting the position of the vehicle is connected to the third information input of the integration module of the output signals, the identification module of the current sections of the marking of the road scene, the information inputs of which are connected to the second and third information outputs of the presentation module the current position of the vehicle, respectively, and the synchronizing input of the module is ide To identify the current sections of the marking of the road scene, it is connected to the synchronizing output of the module for representing the current position of the vehicle, a memory module whose address input is connected to the address output of the identification module of the current sections of the marking of the road scene, the first synchronizing output of which is connected to the read input of the memory module, and the information output of the module memory is connected to the fourth information input of the output signal integration module, a module for fixing the boundaries of the current sections of the road with prices, the information input of which is connected to the information output of the identification module of the current sections of the marking of the road scene, the synchronizing input is connected to the second synchronizing output of the identification module of the current sections of the marking of the road scene, the timing input is the clocking input of the system, designed to receive signals from the measurement sensor of the path traveled by the vehicle, and the output of the module for fixing the boundaries of the current sections of the road scene is a signal output of the system, intended for issue a warning signal to the driver of the vehicle, the second clock output of the module fixing portions road boundaries current stage is connected to the clock input module output signal integration.

Images