KR20220115702A - Pedestrian communication system for self driving cars - Google Patents

Abstract

The present invention provides a pedestrian communication system of an autonomous vehicle, in which an autonomous vehicle recognizes a pedestrian and promotes safety of the pedestrian. The pedestrian communication system of an autonomous vehicle according to one aspect of the present invention comprises: a detection unit recognizing a pedestrian; and an output unit outputting a query in an area that the pedestrian is located. The detection unit detects whether the pedestrian answers the query or not.

Description

Pedestrian communication system for autonomous vehicles

The present invention relates to a pedestrian communication system, and more particularly, to a pedestrian communication system of an autonomous vehicle between an autonomous vehicle and a pedestrian.

An automobile may be classified into an internal combustion engine automobile, an external combustion engine automobile, a gas turbine automobile, an electric vehicle, or the like, according to a type of a prime mover used.

An autonomous vehicle is a vehicle that can drive automatically without human driving. Autonomous vehicles drive autonomously by recognizing their surroundings with radar, light detection and ranging (LIDAR), GPS, and cameras and specifying a destination. Unmanned vehicles that have already been put into practical use include unmanned vehicles that patrol preset routes operated by the Israeli military, and unmanned operation systems such as dump trucks operated in overseas mines and construction sites.

The first core technologies of these autonomous vehicles are the driverless vehicle system and the actual system. In addition to simulation in the laboratory, it is a technology that actually builds an unmanned vehicle system. It implements the accelerator, reducer, and steering device, which are driving devices, for unmanned operation, and enables control using the computer, software and hardware installed in the unmanned vehicle. do.

The second core technology is to receive and process visual information using vision and sensors. As the basis of autonomous driving for unmanned driving, it is a technology that accepts image information and extracts necessary information from this image. It uses CCD (charge-coupled device) cameras as well as ultrasonic sensors and range filters to fuse information necessary for distance and driving, so that it can avoid obstacles and cope with unexpected situations through analysis and processing.

The third core technology is the integrated control system and operation monitoring fault diagnosis system technology. This technology establishes a driving monitoring system that monitors vehicle operation and gives appropriate commands according to changing situations, and provides appropriate information to the operator by analyzing various situations occurring from individual processors and sensors to diagnose system failures. or to perform the function of notifying an alarm.

The fourth core technology is intelligent control and intelligent operation devices. This technology is an unmanned driving technique that generates control commands based on mathematical analysis using real vehicle models. Intelligent Forward Control is a system that maintains the distance from the vehicle in front or obstacles by adjusting the speed by itself without the driver operating the pedals based on radar guide technology. When the driver inputs the distance to the vehicle in front, the long-range radar attached to the front of the vehicle detects the position of the vehicle in front, maintains a constant speed, decelerates, accelerates, and stops completely if necessary, which is useful in weather where visibility is difficult.

The fifth applied technology is the lane departure prevention system. This is a technology that notifies the driver of an unintentional departure situation by detecting a lane with a camera installed inside.

The sixth application technology is the parking assist system. This is a system that helps the car park in reverse line by adjusting the steering system when the driver presses the brake pedal after searching for the assist button and inserts the reverse gear. Based on infrastructure as well as vehicle-mounted sensors, the vehicle is automatically guided from the departure point to the parking space, saving unnecessary time and energy when parking, thereby minimizing cost and environmental pollution.

The seventh application technology is the automatic parking system. This is a technology in which the driver stops the car in front of the parking lot, turns off the engine, gets off, and presses the remote control lock switch twice in succession. to be.

The eighth application technology is a blind spot information guidance system. It is used to change lanes by checking obstacles and vehicles on both sides in complex road conditions, as sensors mounted on both sides of the car determine whether there are other vehicles in the blind spot not seen by the side mirrors and warn the driver. .

The biggest advantage of driverless cars is that they help fuel efficiency by avoiding repeated stops by making the control device more even because the driving speed and traffic management data match. will be. In addition, it has the advantages of solving fatigue caused by long-term driving, greatly reducing the risk of traffic accidents, speeding up road traffic flow, and reducing traffic congestion.

Furthermore, autonomous vehicles are expected to solve a number of parking-related problems by not occupying a parking space unnecessarily in the absence of a person. In particular, when an accident occurs, the autonomous vehicle may move smartly and play a role of easily assisting in handling an accident.

However, there is a problem in that pedestrians may have a vague fear of safety that the autonomous vehicle is not controlled because the discussion on the additional user experience using the autonomous vehicle has not been actively conducted yet. Therefore, it is time for a detailed discussion on communication methods for safety between autonomous vehicles and pedestrians.

Korean Patent Publication No. 2019-0049221 (2019.05.09.)

The present invention is to solve the problems of the prior art, and an object of the present invention is to provide a pedestrian communication system for an autonomous vehicle in which the autonomous vehicle recognizes a pedestrian and promotes the safety of the pedestrian.

A pedestrian communication system for an autonomous vehicle according to an aspect of the present invention includes a sensing unit for recognizing a pedestrian, and an output unit for outputting a question to an area in which the pedestrian is located, wherein the sensing unit answers the pedestrian's question to detect

In this case, the sensing unit may detect whether the pedestrian is stopped, and the output unit may output a query to the stopping position.

In addition, the query may be output to the area where the pedestrian is located, and to select any one of the pedestrian's future movement.

Also, the sensing unit may detect a pedestrian's selection for the query.

Also, the output unit may output an action guide to the front of the pedestrian when the pedestrian's selection is sensed in response to the query.

In addition, the pedestrian communication system of the autonomous vehicle may further include a communication unit, and the communication unit may transmit information about outputting a question to a pedestrian to an adjacent vehicle or an answer from the pedestrian.

In addition, the adjacent vehicle receiving the information may output an action guide by itself.

In addition, the output unit may output a holding button for selecting another pedestrian, and the sensing unit may detect an input of the holding button.

The present invention promotes pedestrian safety because the autonomous vehicle recognizes the pedestrian and asks a question about the pedestrian's action so that the autonomous vehicle considers the pedestrian according to the answer.

In addition, since the present invention allows the autonomous vehicle to output the pedestrian's behavior guide to the road, the pedestrian can perform a predetermined action in a safe state with predictability guaranteed.

1 is a block diagram of a pedestrian communication system of an autonomous vehicle according to an embodiment of the present invention.
FIG. 2 is a configuration diagram illustrating the sensing unit of FIG. 1 in more detail.
FIG. 3 is a configuration diagram illustrating the output unit of FIG. 1 in more detail.
FIG. 4 is a configuration diagram illustrating the control unit in FIG. 1 in more detail.
5 to 7 are views for explaining the operation of the pedestrian communication system of the autonomous vehicle according to an embodiment of the present invention.

Hereinafter, the embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numbers regardless of reference numerals, and redundant description thereof will be omitted.

The suffixes "module" and "part" for components used in the following description are given or mixed in consideration of only the ease of writing the specification, and do not have distinct meanings or roles by themselves.

In addition, in describing the embodiments disclosed in the present specification, if it is determined that detailed descriptions of related known technologies may obscure the gist of the embodiments disclosed in the present specification, the detailed description thereof will be omitted.

In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in the present specification, and the technical spirit disclosed herein is not limited by the accompanying drawings, and all changes included in the spirit and scope of the present invention , should be understood to include equivalents or substitutes.

Terms including ordinal numbers such as first, second, etc. may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but it is understood that other components may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that the other element does not exist in the middle.

The singular expression includes the plural expression unless the context clearly dictates otherwise.

In the present application, terms such as “comprises” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that this does not preclude the possibility of the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Hereinafter, the pedestrian communication system 1000 of an autonomous vehicle according to an embodiment of the present invention will be described. 1 is a configuration diagram of a pedestrian communication system of an autonomous vehicle according to an embodiment of the present invention, FIG. 2 is a configuration diagram illustrating the sensing unit in FIG. 1 in more detail, and FIG. 3 is a diagram showing the output unit in FIG. 1 more It is a configuration diagram showing in detail, and FIG. 4 is a configuration diagram showing the control unit in FIG. 1 in more detail.

Referring to the drawings, the pedestrian communication system 1000 of an autonomous vehicle according to an embodiment of the present invention includes an autonomous vehicle 100 and a server 200 that manages routes and performs remote control.

In this case, the autonomous vehicle 100 includes an input unit 110 , a sensing unit 120 , an output unit 130 , a control unit 140 , a communication unit 150 , and a driving unit 160 .

The input unit 110 is a device for receiving a user input for driving. In the manual mode, the autonomous vehicle 100 may include a steering input device (not shown) of the input unit, an acceleration input device, and a brake input device. In addition, the input unit 110 serves to input the destination of the autonomous vehicle, receives information from the server 200 according to the input destination, and continuously sets a route.

The sensing unit 120 may include a radar 121 and a lidar 122 , and may further include a camera 123 . In this embodiment, using the radar 121, the lidar 122, and the camera 123, the movement and direction of movement and movement of pedestrians and pedestrians, which are main objects, are detected.

First, the radar 121 may generate information on an object outside the autonomous vehicle 100 using radio waves. The radar 121 may include at least one processor that is electrically connected to an electromagnetic wave transmitter, an electromagnetic wave receiver, and an electromagnetic wave transmitter and an electromagnetic wave receiver, processes a received signal, and generates data for an object based on the processed signal. have.

The radar 121 may be implemented in a pulse radar method or a continuous wave radar method in view of a radio wave emission principle. The radar 121 may be implemented in a frequency modulated continuous wave (FMCW) method or a frequency shift keyong (FSK) method according to a signal waveform among continuous wave radar methods. The radar 121 detects an object based on a time of flight (TOF) method or a phase-shift method using an electromagnetic wave as a medium, and calculates the position of the detected object, the distance from the detected object, and the relative speed. can be detected. In this case, the radar 121 may be disposed at an appropriate position outside the vehicle in order to detect an object located in the front, rear, or side of the vehicle.

Next, the lidar 122 may generate information on an object outside the autonomous vehicle 100 using laser light. The lidar 122 is electrically connected to the light transmitter (not shown), the light receiver (not shown), and the light transmitter and the light receiver, processes the received signal, and generates data for the object based on the processed signal. It may include at least one processor.

The lidar 122 may be implemented in a time of flight (TOF) method or a phase-shift method. The lidar 122 may be implemented as a driving type or a non-driven type. When the lidar 122 is implemented as a driving type, the lidar 122 is rotated by a motor and may detect an object around the autonomous vehicle 100 . When implemented as a non-driven type, the lidar 122 may detect an object located within a predetermined range with respect to the vehicle by light steering. The autonomous vehicle 100 may include a plurality of non-driven lidar. The lidar 122 detects an object based on a time of flight (TOF) method or a phase-shift method using a laser light medium, and calculates a position of the detected object, a distance from the detected object, and a relative speed. can be detected. In this case, the lidar 122 may be disposed at an appropriate position outside the vehicle in order to detect an object located in the front, rear, or side of the vehicle.

Meanwhile, the camera 123 may generate information about an object outside the autonomous vehicle 100 by using the image. The camera 123 may include at least one lens, at least one image sensor, and at least one processor that is electrically connected to the image sensor to process a received signal, and generate data about the object based on the processed signal. can

The camera 123 may be at least one of a mono camera, a stereo camera, and an Around View Monitoring (AVM) camera. The camera 123 may obtain position information of an object, distance information from an object, or relative speed information with an object by using various image processing algorithms.

For example, the camera 123 may acquire distance information and relative velocity information from an object based on a change in the size of the object over time in the acquired image. Also, the camera 123 may acquire distance information and relative speed information with respect to an object through a pinhole model, road surface profiling, or the like.

Also, the camera 123 may acquire distance information and relative velocity information from an object based on disparity information in a stereo image obtained from the stereo camera. The camera 123 may be mounted at a position where a field of view (FOV) can be secured in the vehicle to photograph the outside of the vehicle.

The camera 123 may be disposed adjacent to the front windshield in the interior of the vehicle to acquire an image of the front of the vehicle. Furthermore, the camera 123 may be disposed around the front bumper or radiator grill. The camera 123 may be disposed adjacent to the rear glass in the interior of the vehicle to acquire an image of the rear of the vehicle. In this case, the camera 123 may be disposed around the rear bumper, the trunk, or the tailgate. In order for the camera 123 to acquire an image of the side of the vehicle, it may be disposed adjacent to at least one of the side windows in the interior of the vehicle. Alternatively, the camera 123 may be disposed around a side mirror, a fender, or a door.

In addition, since the sensing unit 120 needs to utilize location information of the autonomous vehicle, it essentially further includes a GPS 124 . The GPS 124 may include at least one of a general global positioning system (GPS) and a differential global positioning system (DGPS) to generate location data of the autonomous vehicle 100 . Position data of the autonomous vehicle 10 may be generated based on a signal generated by at least one of the GPS and DGPS.

In this case, the GPS 124 may correct the location data based on at least one of an Inertial Measurement Unit (IMU) and the camera 123 of the sensing unit 120 . Also, the GPS 124 may be referred to as a Global Navigation Satellite System (GNSS).

On the other hand, the sensing unit 120 may further include a microphone 125 in order for the passenger to intervene in the autonomous driving in addition to the guardian by voice or the like even if the passenger does not perform the steering operation. Furthermore, the biometric information sensor 126 not only prepares for an emergency by sensing the passenger's heartbeat, pressure, brain waves, etc., but additionally senses fingerprint and iris information when entering and exiting the vehicle to prevent erroneous boarding. can By utilizing the biometric information sensor 126 , the autonomous vehicle 100 may detect that a passenger has boarded or alighted.

As described above, the radar 121 , the rider 122 , and the camera 123 detect the positions of pedestrians and pedestrians, which are main objects, and detect the distance between the pedestrian and the autonomous vehicle 100 and the movement of the pedestrian. Furthermore, as will be described later, the camera 123 also performs a separate role of detecting which area the pedestrian has selected in response to the output query and detecting whether another pedestrian has inputted the hold button.

The output unit 130 is normally disposed inside or outside the autonomous vehicle 100 to display driving-related situations. Furthermore, any one of a shape, a shape, and a color may be displayed on the exterior of the vehicle to provide predictability to the vehicle owner and surrounding people.

However, the output unit 130 according to an embodiment of the present invention comprises a query output module 131 , a guide output module 132 , and a maintenance button output module 133 .

The query output module 131 outputs a query to the area where the pedestrian recognized by the above-described sensing unit 120 is located. In this case, it is preferable to output the query in the forward direction of the area where the pedestrian stopped in front of the autonomous vehicle is located. It is desirable that the questions be in a short-answer format as they can predict the future movement of pedestrians.

For example, the query output module 131 displays an inquiry as to whether a pedestrian passes in front of the autonomous vehicle 100, and furthermore, the form of "yes" or "no" is displayed on the road in front of the pedestrian. to be output as an image. The pedestrian may answer a question by pointing either "yes" or "no" as an action. Thereafter, as described above, the camera 123 of the sensing unit 120 detects whether the pedestrian's motion is “yes” or “no”.

Meanwhile, when the movement of the pedestrian is detected, the query output module 131 may output the query to the front of the pedestrian in accordance with the movement speed of the pedestrian.

The guide output module 132 serves to output an action guide to the front of the pedestrian when the pedestrian's selection for the question is sensed. As an example, the behavior guide may output a crosswalk image on the road in front of the pedestrian or in the cross direction of the road when the selection of the pedestrian in the above example is “Yes”.

The maintenance button output module 133 is configured to assume that there is another pedestrian. When another pedestrian is detected in addition to the original pedestrian, the maintenance button output module 133 outputs a maintenance button image on the area where the other pedestrian is located. At this time, when the other pedestrian selects the hold button as an action in the same manner, the sensing unit 120 detects the input. In this case, the guide output module 132 increases the output time of the output action guide or expands the output area of the output action guide. For example, when another pedestrian selects the hold button after the first pedestrian is selected, the output time of the crosswalk image is increased or the length or width of the crosswalk image is increased.

Meanwhile, the control unit 140 may be configured as a main ECU, and controls the driving unit 160 of the autonomous vehicle 100 .

In particular, the controller 140 may include a power train drive control device, a chassis drive control device, a door/window drive control device, a safety device drive control device, a lamp drive control device, and an air conditioning drive control device. The power train drive control device may include a power source drive control device and a transmission drive control device. The chassis drive control device may include a steering drive control device, a brake drive control device, and a suspension drive control device. Meanwhile, the safety device drive control device may include a safety belt drive control device for seat belt control.

The controller 140 includes at least one electronic control device (eg, a control ECU (Electronic Control Unit)). In particular, on the basis of the received signal, it is possible to control the vehicle drive unit. For example, the controller 140 may control a power train, a steering device, and a brake device based on a signal received from the sensor 120 .

In addition, the control unit 140 according to an embodiment of the present invention includes a pedestrian determination module 141 , an answer confirmation module 142 , an output guide control module 143 , and a communication control module 144 .

The pedestrian determination module 141 receives the signal detected by the radar 121 and the rider 122 to classify the information, and the camera 123 converts the color information input as an image into black and white. At this time, the lidar 122 converts the received information into a coordinate plane, and the camera 123 normalizes the histogram (frequency distribution table) from the converted information.

In addition, an ROI (region of interest) is set through the coordinate system converted by the lidar 122 and the histogram obtained from the camera. Then, the ROI obtained in the setting step is converted to HOG to obtain pedestrian recognition information, and the lidar 122 obtains a distance value from the information transmitted as an RF signal. Thereafter, two pieces of information obtained by measuring the region of interest and the distance value are combined, and pedestrian/non-pedestrians are distinguished from the combined information through SVM classification.

Pedestrians are classified through the classification of the classification step, and information is stored in the case of pedestrians. If pedestrians can be classified, the normalization step is additionally returned. If not, the algorithm is terminated. Accordingly, the autonomous vehicle may slow down or apply the brake to reduce the speed of the driving vehicle or avoid pedestrians or cyclists.

The answer confirmation module 142 serves to confirm the pedestrian's answer to the above-mentioned questions. In this embodiment, since it is checked whether the pedestrian has selected the image of the query output on the road, it is preferable to make the determination based on the sensed value transmitted from the camera 123 . At this time, the user's selection is determined based on whether a portion of the "yes" or "no" image is not captured or the region where the user's foot is located in the above-mentioned "yes" or "no" image according to the image of the camera.

At this time, when the answer is not confirmed, it is preferable that the answer confirmation module 142 controls the question output module 131 to output the question again to the location of the pedestrian to determine the correct selection of the pedestrian.

The output guide control module 143 controls the output state of the action guide to control the guide output module 132 . For example, the duration of the virtual crosswalk output as an action guide, the width of the output, and the like are determined. As described above, even when another pedestrian is detected, the output guide control module 143 is configured to increase the output time of the crosswalk image or increase the length or width of the crosswalk image when another pedestrian selects the aforementioned hold button. can be controlled

The communication control module 144 controls the communication unit 150, and the communication unit 150 transmits to an adjacent vehicle that a question has been output to the pedestrian or that the pedestrian has answered the choice. In this case, the adjacent vehicle receiving such information may output the above-described action guide by itself.

For example, when an autonomous vehicle initially outputs a crosswalk image as a behavior guide, the vehicle following receiving this information can also display the behavior guide on the pedestrian's movement path. At this time, the color or size of the crosswalk, which is the action guide, is changed or the crosswalk is displayed more clearly. Accordingly, it is possible for pedestrians to move safely with respect to all vehicles in front.

Meanwhile, the communication unit 150 may exchange signals with devices located outside the autonomous vehicle 100 , and may exchange signals with at least one of an infrastructure (eg, a server, a broadcasting station), another vehicle, and a terminal. have. The communication unit 150 may include at least one of a transmit antenna, a receive antenna, a radio frequency (RF) circuit capable of implementing various communication protocols, and an RF element to perform communication. For example, the communication apparatus may exchange a signal with an external device based on C-V2X (Cellular V2X) technology. In addition, the communication unit 150 is based on the IEEE 802.11p PHY / MAC layer technology and the IEEE 1609 Network / Transport layer technology based on DSRC (Dedicated Short Range Communications) technology or WAVE (Wireless Access in Vehicular Environment) standard based on the external device and signal can be exchanged for

Hereinafter, the operation of the present invention will be exemplified and described. 5 to 7 are diagrams for explaining the operation of a pedestrian communication system of an autonomous vehicle according to an embodiment of the present invention.

Referring to FIG. 5 , the autonomous vehicle that recognizes the pedestrian outputs a query on the road in front of the pedestrian. In this case, the questions may be divided into a question output on the vehicle and an option output on the road.

Next, referring to FIG. 6 , it is shown that the pedestrian selects a YES part among the options. The autonomous vehicle will be aware of these options and will confirm that pedestrians will cross the road.

Finally, referring to FIG. 7 , the autonomous vehicle outputs a crosswalk image as an action guide according to the movement direction of the pedestrian, and the pedestrian moves while feeling a sense of safety as long as the crosswalk image is maintained.

As such, those skilled in the art to which the present invention pertains will understand that the present invention may be embodied in other specific forms without changing the technical spirit or essential characteristics thereof.

In addition, the present specification and drawings have been disclosed with respect to preferred embodiments of the present invention, and although specific terms are used, these are only used in a general sense to easily explain the technical content of the present invention and help the understanding of the present invention, It is not intended to limit the scope of the invention. It will be apparent to those of ordinary skill in the art to which the present invention pertains that other modifications based on the technical spirit of the present invention can be implemented in addition to the embodiments disclosed herein.

1000: Pedestrian communication system of autonomous vehicle
100: autonomous vehicle
110: input unit
120: detection unit
130: output unit
140: control unit
150: communication department
160: driving unit
200: server

Claims (8)

a sensing unit for recognizing a pedestrian; and
an output unit for outputting a query to an area where the pedestrian is located;
including,
The sensing unit is a pedestrian communication system of an autonomous vehicle, characterized in that it detects whether an answer to the pedestrian's question is answered.
According to claim 1,
The sensing unit detects whether the pedestrian is stopped,
and the output unit outputs a query to the stop position.
According to claim 1,
The query is output to the area where the pedestrian is located, and the pedestrian communication system of the autonomous driving vehicle, characterized in that the output is output to select any one of the pedestrian's future movement.
4. The method of claim 3,
The sensing unit is a pedestrian communication system of an autonomous vehicle, characterized in that it detects the pedestrian's selection for the query.
5. The method of claim 4,
The output unit outputs an action guide to the front of the pedestrian when the pedestrian's selection is detected in response to the query.
6. The method of claim 5,
The pedestrian communication system of the autonomous vehicle further includes a communication unit, and the communication unit transmits information about outputting a question to a pedestrian to an adjacent vehicle or a response from the pedestrian to an adjacent vehicle. system.
7. The method of claim 6,
A pedestrian communication system, characterized in that the adjacent vehicle receiving the information outputs a behavior guide by itself.
6. The method of claim 5,
Pedestrian communication system, characterized in that the output unit outputs a holding button for selecting another pedestrian, and the sensing unit detects an input of the holding button.

Images