KR102568270B1 - Pedestrian communication system for self driving cars - Google Patents

Abstract

The present invention provides a pedestrian communication system for an autonomous vehicle in which the autonomous vehicle recognizes a pedestrian and promotes the safety of the pedestrian. Accordingly, a pedestrian communication system for an autonomous vehicle according to an aspect of the present invention includes a sensing unit recognizing a pedestrian and an output unit outputting a query to an area where the pedestrian is located, and the sensing unit responds to the pedestrian's inquiry. detect whether

Description

Pedestrian communication system for autonomous vehicles {PEDESTRIAN COMMUNICATION SYSTEM FOR SELF DRIVING CARS}

The present invention relates to a pedestrian communication system, and more particularly, to a pedestrian communication system of an autonomous vehicle between an autonomous vehicle and a pedestrian.

Vehicles may be classified into internal combustion engine vehicles, external combustion engine vehicles, gas turbine vehicles, electric vehicles, and the like, depending on the type of prime mover used.

Autonomous vehicles are vehicles that can drive automatically without human intervention. Driverless cars drive autonomously by recognizing the surrounding environment with radar, LIDAR (light detection and ranging), GPS, and camera and designating a destination. Unmanned vehicles that are already in practical use include unmanned vehicles operated by the Israeli military that patrol preset routes and unmanned operation systems such as dump trucks that are operated at overseas mines or construction sites.

The first core technologies of these self-driving vehicles are the unmanned car system and the actual system. It is a technology that builds an unmanned car system in reality as well as simulation in the laboratory. It implements driving devices such as accelerators, decelerators, and steering devices to suit unmanned operation, and enables control using computers, software, and hardware installed in unmanned cars. do.

The second key technology is to receive and process visual information using vision and sensors. It is the basis of autonomous driving for unmanned operation, and it is a technology that accepts image information and extracts necessary information from the image. This uses not only a CCD (charge-coupled device) camera, but also sensors such as ultrasonic sensors and range filters to fuse information necessary for distance and driving, allowing obstacle avoidance and unexpected situations to be dealt with through analysis and processing.

The third core technology is the integrated control system and operation monitoring fault diagnosis system technology. This technology establishes an operation monitoring system that monitors vehicle operation and gives appropriate commands according to frequently changing situations, analyzes various situations generated by individual processors and sensors, diagnoses system failures, and provides appropriate information to the operator. or to perform the function of notifying an alarm.

The fourth key technology is intelligent control and intelligent driving devices. This technology generates control commands based on mathematical analysis using actual vehicle models as an unmanned driving technique. Intelligent forward control is a system based on radar guide technology that maintains the distance from the vehicle in front or obstacles by adjusting the speed on its own without the driver manipulating the pedals. When the driver inputs the distance to the vehicle in front, the long-range radar attached to the front of the vehicle detects the location of the vehicle in front, maintains a constant speed, decelerates or accelerates, and comes to a complete stop if necessary, which is useful in poor visibility weather.

The fifth applied technology is the lane departure prevention system. This is a technology in which a camera installed inside detects the lane and informs the driver of an unintended deviation situation.

The sixth applied technology is a parking assist system. When the driver searches for the assist button, puts the reverse gear in and presses the brake pedal, the car adjusts the steering to assist in parallel parking in reverse. It automatically guides the vehicle from the starting point to the parking space based on infrastructure as well as vehicle-mounted sensors to save unnecessary time and energy when parking, thereby minimizing cost and environmental pollution.

The seventh applied technology is an automatic parking system. This is a technology in which the driver stops the car in front of the parking lot, turns off the engine, gets off, and presses the remote control lock switch twice in a row. am.

The eighth applied technology is a blind spot information guidance system. Sensors mounted on both sides of the car determine if there is another vehicle in the blind spot that is not visible through the side mirror and warn the driver. .

The biggest advantage of an unmanned car is that it helps fuel efficiency by avoiding repeated stops by making the control device more even because the driving speed and traffic management data match, and that it can be used by people who cannot drive, such as the elderly, children, and the disabled. will be. In addition, it has the advantage of solving fatigue caused by long-time driving, greatly reducing the risk of traffic accidents, speeding up traffic flow on the road and reducing traffic congestion.

Furthermore, self-driving vehicles are expected to solve numerous problems related to parking by not unnecessarily occupying a parking space in the absence of a person. In particular, when an accident occurs, the self-driving vehicle can move smartly and play a role of easily assisting in coping with the accident.

However, since discussions on additional user experiences using autonomous vehicles have not yet been actively conducted, there is a problem that pedestrians may have vague safety fears that autonomous vehicles are not controlled. Therefore, it is time to discuss in detail how to communicate with autonomous vehicles and pedestrians for safety.

Korean Patent Publication No. 2019-0049221 (2019. 05. 09.)

The present invention is to solve the problems of the prior art, and an object of the present invention is to provide a pedestrian communication system for an autonomous vehicle in which the autonomous vehicle recognizes a pedestrian and promotes the safety of the pedestrian.

A pedestrian communication system for an autonomous vehicle according to an aspect of the present invention includes a sensor for recognizing a pedestrian and an output unit for outputting a query to an area where the pedestrian is located, wherein the sensor determines whether or not the sensor answers the pedestrian's query. detect

In this case, the detection unit may detect whether the pedestrian is stopped, and the output unit may output an inquiry to the stop position.

In addition, the query item may be output to the area where the pedestrian is located so as to select one of the questions regarding the future movement of the pedestrian.

Also, the detector may detect a pedestrian's selection of an inquiry item.

In addition, the output unit may output an action guide to the front of the pedestrian when selection of the pedestrian with respect to the query item is detected.

The pedestrian communication system of the self-driving vehicle may further include a communication unit, and the communication unit may transmit information indicating that a query has been output to a pedestrian or an answer given by the pedestrian to an adjacent vehicle.

In addition, a neighboring vehicle receiving the information may output an action guide on its own.

Also, the output unit may output a maintenance button for another pedestrian to select, and the detection unit may detect an input of the maintenance button.

The present invention promotes the safety of pedestrians because the self-driving vehicle recognizes the pedestrian, queries what the pedestrian will do, and makes the autonomous vehicle considerate of the pedestrian according to the answer.

In addition, since the present invention allows the autonomous vehicle to output the pedestrian's action guide to the road, the pedestrian can perform a predetermined action in a safe state with predictability guaranteed.

1 is a block diagram of a pedestrian communication system for an autonomous vehicle according to an embodiment of the present invention.
FIG. 2 is a configuration diagram showing the sensing unit in FIG. 1 in more detail.
FIG. 3 is a configuration diagram showing the output unit in FIG. 1 in more detail.
4 is a configuration diagram showing the control unit in FIG. 1 in more detail.
5 to 7 are diagrams illustrating the operation of a pedestrian communication system of an autonomous vehicle according to an embodiment of the present invention.

Hereinafter, the embodiments disclosed in this specification will be described in detail with reference to the accompanying drawings, but the same or similar components are given the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted.

The suffixes "module" and "unit" for components used in the following description are given or used together in consideration of ease of writing the specification, and do not have meanings or roles that are distinct from each other by themselves.

In addition, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the embodiment disclosed in this specification, the detailed description thereof will be omitted.

In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in this specification, the technical idea disclosed in this specification is not limited by the accompanying drawings, and all changes included in the spirit and technical scope of the present invention , it should be understood to include equivalents or substitutes.

Terms including ordinal numbers, such as first and second, may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this application, terms such as "comprise" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Hereinafter, a pedestrian communication system 1000 for an autonomous vehicle according to an embodiment of the present invention will be described. 1 is a configuration diagram of a pedestrian communication system for an autonomous vehicle according to an embodiment of the present invention, FIG. 2 is a configuration diagram showing the sensing unit in FIG. 1 in more detail, and FIG. 3 is a configuration diagram showing the output unit in FIG. 1 in more detail. It is a detailed configuration diagram, and FIG. 4 is a configuration diagram showing the control unit in FIG. 1 in more detail.

Referring to the drawings, first, a pedestrian communication system 1000 for an autonomous vehicle according to an embodiment of the present invention includes an autonomous vehicle 100 and a server 200 that performs path management and remote control.

In this case, the autonomous vehicle 100 includes an input unit 110, a sensing unit 120, an output unit 130, a control unit 140, a communication unit 150, and a driving unit 160.

The input unit 110 is a device that receives a user input for driving. In the manual mode, the autonomous vehicle 100 may include a steering input device (not shown), an acceleration input device, and a brake input device of an input unit. In addition, the input unit 110 serves to input the destination of the autonomous vehicle, receives information from the server 200 according to the input destination, and continuously sets a route.

The detector 120 may include a radar 121 and a lidar 122, and may further include a camera 123. In this embodiment, a pedestrian, which is a main object, and movement and direction of movement of the pedestrian are detected using the radar 121, the lidar 122, and the camera 123.

First, the radar 121 may generate information about an object outside the self-driving vehicle 100 by using radio waves. The radar 121 may include an electromagnetic wave transmitter, an electromagnetic wave receiver, and at least one processor electrically connected to the electromagnetic wave transmitter and electromagnetic wave receiver, processing a received signal, and generating data about an object based on the processed signal. there is.

The radar 121 may be implemented in a pulse radar method or a continuous wave radar method in terms of radio wave emission principles. The radar 121 may be implemented in a frequency modulated continuous wave (FMCW) method or a frequency shift keyong (FSK) method according to a signal waveform among continuous wave radar methods. The radar 121 detects an object based on a Time of Flight (TOF) method or a phase-shift method through electromagnetic waves, and measures the position of the detected object, the distance to the detected object, and the relative speed. can be detected. In this case, the radar 121 may be disposed at an appropriate location outside the vehicle to detect an object located in front, rear or side of the vehicle.

Next, the lidar 122 may generate information about an object outside the self-driving vehicle 100 using laser light. The LIDAR 122 is electrically connected to the light transmitter (not shown), the light receiver (not shown), and the light transmitter and the light receiver, processes the received signal, and generates data for an object based on the processed signal. It may include at least one processor that

The lidar 122 may be implemented in a Time of Flight (TOF) method or a phase-shift method. The lidar 122 may be implemented as a driven or non-driven type. When implemented as a driven type, the lidar 122 is rotated by a motor and may detect objects around the autonomous vehicle 100. When implemented as a non-driving type, the lidar 122 may detect an object located within a predetermined range with respect to the vehicle by light steering. The autonomous vehicle 100 may include a plurality of non-driven lidars. The lidar 122 detects an object based on a time of flight (TOF) method or a phase-shift method using a laser light medium, and calculates the position of the detected object, the distance to the detected object, and the relative speed. can be detected. At this time, the lidar 122 may be disposed at an appropriate location outside the vehicle to detect an object located in the front, rear, or side of the vehicle.

Meanwhile, the camera 123 may generate information about an object outside the self-driving vehicle 100 by using an image. The camera 123 may include at least one lens, at least one image sensor, and at least one processor electrically connected to the image sensor to process a received signal and to generate object data based on the processed signal. can

The camera 123 may be at least one of a mono camera, a stereo camera, and an AVM (Around View Monitoring) camera. The camera 123 may obtain position information of an object, distance information with respect to the object, or relative speed information with the object by using various image processing algorithms.

For example, the camera 123 may obtain distance information and relative speed information with respect to the object based on a change in the size of the object over time in the obtained image. In addition, the camera 123 may obtain distance information and relative speed information with an object through a pinhole model, road profiling, and the like.

In addition, the camera 123 may obtain distance information and relative speed information with respect to an object based on disparity information in a stereo image obtained from a stereo camera. The camera 123 may be mounted in a position where a field of view (FOV) can be secured in the vehicle in order to photograph the exterior of the vehicle.

The camera 123 may be disposed close to the front windshield inside the vehicle to obtain an image of the front of the vehicle. Furthermore, the camera 123 may be disposed around a front bumper or a radiator grill. The camera 123 may be disposed close to the rear glass inside the vehicle to obtain an image behind the vehicle. In this case, the camera 123 may be disposed around a rear bumper, a trunk, or a tailgate. The camera 123 may be disposed close to at least one of the side windows in the interior of the vehicle in order to acquire an image of the side of the vehicle. Alternatively, the camera 123 may be disposed around side mirrors, fenders, or doors.

In addition, since the sensing unit 120 needs to utilize the location information of the self-driving vehicle, the GPS 124 is essentially further included. The GPS 124 may include at least one of a general Global Positioning System (GPS) and a Differential Global Positioning System (DGPS) to generate location data of the autonomous vehicle 100 . Location data of the self-driving vehicle 10 may be generated based on a signal generated by at least one of the GPS and DGPS.

In this case, the GPS 124 may correct the location data based on at least one of an Inertial Measurement Unit (IMU) and the camera 123 of the sensing unit 120 . Also, the GPS 124 may be referred to as a Global Navigation Satellite System (GNSS).

Meanwhile, the sensing unit 120 may further include a microphone 125 so that the occupant, in addition to the guardian, may intervene in autonomous driving by voice or the like even if the occupant does not perform a steering operation. Furthermore, the biometric information sensor 126 not only senses the heart rate, blood pressure, and brain waves of passengers to prepare for emergencies, but additionally performs a function to prevent erroneous boarding by sensing fingerprint and iris information when entering and exiting the vehicle. can Utilizing the biometric information sensor 126, the self-driving vehicle 100 can detect that a passenger has boarded or alighted.

The radar 121, the lidar 122, and the camera 123 as described above detect the pedestrian, which is a main object, and the location of the pedestrian, and detect the distance between the pedestrian and the autonomous vehicle 100 and the movement of the pedestrian. Furthermore, as will be described later, the camera 123 separately performs a role of detecting which region the pedestrian has selected with respect to the output query and also detecting whether another pedestrian presses the hold button.

The output unit 130 is normally disposed inside or outside the self-driving vehicle 100 to display a driving-related situation. Furthermore, any one of shape, shape, and color may be displayed on the outside of the vehicle to provide predictability to the owner of the vehicle and people around.

By the way, the output unit 130 according to an embodiment of the present invention includes a query item output module 131, a guide output module 132, and a maintenance button output module 133.

The query item output module 131 outputs the query item to the area where the pedestrian is located recognized by the sensor 120 described above. At this time, it is preferable to output the inquiry items in the front direction of the area where the pedestrian stopped in front of the autonomous vehicle is located. It is preferable that the questions be made in the form of short answers as they can predict the pedestrian's future movement.

For example, the query item output module 131 displays content asking whether a pedestrian passes in front of the self-driving vehicle 100, and furthermore, a “yes” or “no” form is displayed on the road in front of the pedestrian. to be published as an image. The pedestrian may answer the question by indicating either "yes" or "no" as an operation. After that, as described above, the camera 123 of the detector 120 detects whether the pedestrian's motion is “yes” or “no”.

Meanwhile, when the movement of the pedestrian is detected, the query item output module 131 may output the query items in front of the pedestrian according to the pedestrian's movement speed.

The guide output module 132 serves to output an action guide in front of the pedestrian when the pedestrian's selection on the inquiry item is detected. As an example, the action guide may output a crosswalk image on the road in a cross direction of the road or in front of the pedestrian when the selection of the pedestrian is “yes” in the above example.

The maintenance button output module 133 is a configuration provided to assume a case where there is another pedestrian. The maintenance button output module 133 outputs a maintenance button image on an area where another pedestrian is located when another pedestrian other than the original pedestrian is detected. At this time, when the other pedestrian selects the hold button as an action in the same way, the detector 120 senses the input. In this case, the guide output module 132 increases the output time of the output behavior guide or expands the output area of the output behavior guide. For example, if another pedestrian selects the hold button after the initial pedestrian selection, the output time of the crosswalk image is increased or the length or width of the crosswalk image is increased.

Meanwhile, the control unit 140 may be configured as a main ECU and controls the driving unit 160 of the self-driving vehicle 100.

In particular, the control unit 140 may include a power train driving control device, a chassis driving control device, a door/window driving control device, a safety device driving control device, a lamp driving control device, and an air conditioning driving control device. The power train driving control device may include a power source driving control device and a transmission driving control device. The chassis drive control device may include a steering drive control device, a brake drive control device, and a suspension drive control device. Meanwhile, the safety device driving control device may include a seat belt driving control device for controlling seat belts.

The controller 140 includes at least one electronic control device (eg, a control ECU (Electronic Control Unit)). In particular, the vehicle driving device may be controlled based on the received signal. For example, the controller 140 may control a power train, a steering device, and a brake device based on a signal received from the sensor 120 .

In addition to this, the control unit 140 according to an embodiment of the present invention includes a pedestrian determination module 141, an answer confirmation module 142, an output guide control module 143, and a communication control module 144.

The pedestrian determination module 141 receives signals detected by the radar 121 and the rider 122 to classify information, and the camera 123 converts color information input as an image into black and white. At this time, the lidar 122 converts the received information into a coordinate plane, and the camera 123 normalizes the histogram (frequency distribution table) in the converted information.

In addition, the ROI (region of interest) is set through the coordinate system converted by the LIDAR 122 and the histogram obtained from the camera. In addition, the ROI obtained in the setting step is subjected to HOG conversion to obtain pedestrian recognition information, and the LIDAR 122 obtains a distance value from information transmitted as an RF signal. Thereafter, the two pieces of information obtained by measuring the region of interest and the distance value are combined, and pedestrians/non-pedestrians are distinguished through SVM classification from the combined information.

Through classification in this classification step, pedestrians are classified, and information is stored in case of pedestrians, and if pedestrians can be classified, additionally return to the normalization step, and if there is no pedestrian, the algorithm is terminated. Accordingly, the self-driving vehicle may slow down or apply the brake to reduce the speed of the driving vehicle or avoid a pedestrian or cyclist.

The answer confirmation module 142 serves to determine the pedestrian's answer to the above-mentioned questions. In this embodiment, since it is checked whether the pedestrian has selected the image of the question item output on the road, it is preferable to make the determination based on the detected value transmitted from the camera 123. At this time, the region where the user's foot is located in the above-described "yes" or "no" image is determined according to the image of the camera, or the user's selection is determined based on the fact that a part of the "yes" or "no" image is not captured.

At this time, if the answer is not confirmed, it is preferable that the answer confirmation module 142 controls the question item output module 131 to output the question item again to the pedestrian's position to determine the correct selection of the pedestrian.

The output guide control module 143 controls the guide output module 132 and controls the output state of the action guide. For example, the duration of the virtual crosswalk output as an action guide and the width of the output are determined. As described above, even when another pedestrian is detected, the output guide control module 143 increases the output time of the crosswalk image or increases the length or width of the crosswalk image when another pedestrian selects the above-mentioned maintenance button. You can control it.

The communication control module 144 controls the communication unit 150, and the communication unit 150 transmits to an adjacent vehicle that a query has been output to the pedestrian or that the pedestrian has answered a selection. In this case, an adjacent vehicle receiving such information may output the above-described action guide on its own.

For example, if an autonomous vehicle initially outputs a crosswalk image as an action guide, a vehicle following after receiving this information can also display the action guide on the pedestrian's movement path. At this time, the color or size of the crosswalk, which is an action guide, is changed, or the crosswalk is displayed more clearly. Accordingly, it is possible for the pedestrian to move safely with respect to all vehicles in front.

Meanwhile, the communication unit 150 may exchange a signal with a device located outside the autonomous vehicle 100, and may exchange a signal with at least one of an infrastructure (eg, a server, a broadcasting station), another vehicle, and a terminal. there is. The communication unit 150 may include at least one of a transmission antenna, a reception antenna, a radio frequency (RF) circuit capable of implementing various communication protocols, and an RF element to perform communication. For example, the communication device may exchange signals with an external device based on C-V2X (Cellular V2X) technology. In addition, the communication unit 150 transmits external devices and signals based on Dedicated Short Range Communications (DSRC) technology based on IEEE 802.11p PHY/MAC layer technology and IEEE 1609 Network/Transport layer technology or Wireless Access in Vehicular Environment (WAVE) standard. can be exchanged.

Hereinafter, the operation of the present invention is exemplified and described. 5 to 7 are diagrams illustrating the operation of a pedestrian communication system of an autonomous vehicle according to an embodiment of the present invention.

Referring to FIG. 5 , the self-driving vehicle recognizing the pedestrian outputs an inquiry item on the road ahead of the pedestrian. In this case, the questions may be divided into questions output on the vehicle and options output on the road.

Next, referring to FIG. 6 , it is shown that the pedestrian selects the YES part among options. For these options, the self-driving vehicle will be aware and confirm that the pedestrian will cross the road.

Finally, referring to FIG. 7 , the self-driving vehicle outputs a crosswalk image, which is an action guide, according to the pedestrian's moving direction, and the pedestrian moves while feeling safe as long as the crosswalk image is maintained.

As such, those skilled in the art to which the present invention pertains will be able to understand that the present invention may be embodied in other specific forms without changing its technical spirit or essential features.

In addition, the present specification and drawings disclose preferred embodiments of the present invention, and although specific terms are used, they are only used in a general sense to easily explain the technical content of the present invention and help understanding of the present invention. It is not intended to limit the scope of the invention. It is obvious to those skilled in the art that other modified examples based on the technical spirit of the present invention can be implemented in addition to the embodiments disclosed herein.

1000: Pedestrian communication system for autonomous vehicles
100: autonomous vehicle
110: input unit
120: sensing unit
130: output unit
140: control unit
150: communication department
160: driving unit
200: server

Claims (8)

a detector for recognizing a pedestrian; and
an output unit outputting an inquiry to an area where the pedestrian is located;
including,
The detecting unit detects whether or not the pedestrian answers the inquiry,
The output unit outputs an action guide in front of the pedestrian when selection of the pedestrian is detected with respect to the query,
The output unit outputs a maintenance button for selection by another pedestrian, and when the detection unit detects that another pedestrian selects the maintenance button, the output unit increases the output time of the previously output behavior guide or outputs the output behavior. Pedestrian communication system of an autonomous vehicle, characterized by expanding the output area of the guide.
According to claim 1,
The detecting unit detects whether the pedestrian is stopped,
The output unit outputs an inquiry to the stop position.
According to claim 1,
The pedestrian communication system of the autonomous vehicle, characterized in that the query is output to the area where the pedestrian is located and output to select one of the items for the pedestrian's future movement.
According to claim 3,
The pedestrian communication system of the autonomous vehicle, characterized in that for detecting the selection of the pedestrian for the query.
delete According to claim 1,
The pedestrian communication system of the self-driving vehicle further includes a communication unit, wherein the communication unit transmits information about outputting a query to a pedestrian to an adjacent vehicle or an answer of the pedestrian. system.
According to claim 6,
A pedestrian communication system for an autonomous vehicle, characterized in that the neighboring vehicle receiving the information outputs an action guide on its own.
delete