WO2018008314A1 - 歩行者検出装置、歩行者検出方法 - Google Patents

歩行者検出装置、歩行者検出方法 Download PDF

Info

Publication number
WO2018008314A1
WO2018008314A1 PCT/JP2017/020809 JP2017020809W WO2018008314A1 WO 2018008314 A1 WO2018008314 A1 WO 2018008314A1 JP 2017020809 W JP2017020809 W JP 2017020809W WO 2018008314 A1 WO2018008314 A1 WO 2018008314A1
Authority
WO
WIPO (PCT)
Prior art keywords
pedestrian
roadway
crossing
target
target pedestrian
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2017/020809
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
真也 來山
弘和 大薮
正明 廣瀬
近藤　敏之
鎌田　忠
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Denso Corp
Original Assignee
Denso Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Denso Corp filed Critical Denso Corp
Priority to DE112017003414.0T priority Critical patent/DE112017003414T5/de
Priority to US16/315,411 priority patent/US11011062B2/en
Publication of WO2018008314A1 publication Critical patent/WO2018008314A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
    • B60Q9/00—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling
    • B60Q9/008—Arrangement or adaptation of signal devices not provided for in one of main groups B60Q1/00 - B60Q7/00, e.g. haptic signalling for anti-collision purposes
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R21/00—Arrangements or fittings on vehicles for protecting or preventing injuries to occupants or pedestrians in case of accidents or other traffic risks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/23—Recognition of whole body movements, e.g. for sport training
    • G06V40/25—Recognition of walking or running movements, e.g. gait recognition
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/005—Traffic control systems for road vehicles including pedestrian guidance indicator
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • G08G1/166—Anti-collision systems for active traffic, e.g. moving vehicles, pedestrians, bikes
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
    • B60Q1/00—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor
    • B60Q1/26—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor the devices being primarily intended to indicate the vehicle, or parts thereof, or to give signals, to other traffic
    • B60Q1/50—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor the devices being primarily intended to indicate the vehicle, or parts thereof, or to give signals, to other traffic for indicating other intentions or conditions, e.g. request for waiting or overtaking
    • B60Q1/54—Arrangement of optical signalling or lighting devices, the mounting or supporting thereof or circuits therefor the devices being primarily intended to indicate the vehicle, or parts thereof, or to give signals, to other traffic for indicating other intentions or conditions, e.g. request for waiting or overtaking for indicating speed outside of the vehicle
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion

Definitions

  • the present invention relates to a technique for detecting a pedestrian existing in the vicinity of a vehicle, and more particularly to a technique for detecting a pedestrian trying to cross a roadway before actually starting to cross the road.
  • Patent Document 1 a technique for determining whether or not a pedestrian is going to cross a roadway by detecting a time-series change in the position of the pedestrian and a time-series change in moving speed.
  • Patent Document 1 a technique for determining whether or not a pedestrian is going to cross a roadway by detecting a time-series change in the position of the pedestrian and a time-series change in moving speed.
  • the proposed technique above detects the movement of the pedestrian crossing the roadway, so it has already started crossing at the time of detection, and detects the pedestrian trying to cross before starting to cross. It's not possible.
  • This disclosure is intended to provide a technique capable of accurately detecting a pedestrian who is going to cross a roadway at a stage before starting to cross.
  • the pedestrian detection device and the pedestrian detection method detect a target pedestrian existing within a predetermined distance from the roadway in a photographed image of the in-vehicle camera. Then, it is determined based on the direction of the line of sight of the target pedestrian whether or not the target pedestrian has watched the crossing destination and whether or not safety has been confirmed for the vehicle traveling on the roadway. As a result, if the target pedestrian has performed either the gaze at the crossing destination or the safety check and then has performed the other movement within the specified time, the crossing willing to cross the road Detect as a front pedestrian.
  • a pedestrian trying to cross a roadway has a high probability of 100% prior to crossing. Gaze and perform safety checks on vehicles traveling on the roadway to be crossed. Therefore, if a target pedestrian who has observed a crossing destination and confirming safety with respect to the vehicle within a predetermined time is detected as a pre-crossing pedestrian who intends to cross the roadway, the pedestrian trying to cross the roadway Can be detected with high accuracy at the stage before actually starting to cross.
  • FIG. 1 shows a rough configuration of a vehicle 1 on which a pedestrian detection device 100 according to this embodiment is mounted.
  • the vehicle 1 in addition to the pedestrian detection device 100, the vehicle 1 is equipped with an in-vehicle camera 2, a monitor 3, a speaker 4, a navigation device (hereinafter referred to as a navigation device) 10, and the like.
  • the in-vehicle camera 2 captures the front situation at a predetermined time interval and outputs the obtained captured image to the pedestrian detection device 100.
  • the navigation device 10 stores map data, detects the current position of the vehicle 1, and outputs map data around the current position to the pedestrian detection device 100.
  • the pedestrian detection apparatus 100 analyzes the captured image received from the in-vehicle camera 2 and detects a pedestrian reflected in the captured image. Further, it is determined whether or not the detected pedestrian is a pedestrian trying to cross the roadway before the pedestrian starts crossing. A method for determining whether or not a pedestrian is going to cross before the start of crossing will be described in detail later. In the following, a pedestrian who is trying to cross the roadway but has not yet started crossing will be referred to as a “pre-crossing pedestrian”.
  • a navigation device 10 is also connected to the pedestrian detection device 100 of the present embodiment. Therefore, when determining whether or not the pedestrian is a pre-crossing pedestrian, information on the current position of the vehicle 1 and surrounding map information can be acquired from the navigation device 10 and used. And the pedestrian detection apparatus 100 outputs the result of having detected the pedestrian before crossing from the pedestrian reflected in the picked-up image using the monitor 3 or the speaker 4 to a driver
  • FIG. 2 shows an internal configuration of the pedestrian detection device 100 of the present embodiment.
  • the pedestrian detection device 100 includes a captured image acquisition unit 101, a pedestrian detection unit 102, a roadway detection unit 103, a target pedestrian detection unit 104, a line-of-sight direction acquisition unit 105, and a gaze determination unit. 106, a safety confirmation determination unit 107, and a pre-crossing pedestrian detection unit 108.
  • these “parts” are the pedestrian detection device 100, focusing on the function that the pedestrian detection device 100 of this embodiment provides for detecting a pedestrian before crossing from among pedestrians in a captured image. It is an abstract concept that categorizes the inside of the space for convenience. Therefore, it does not indicate that the pedestrian detection device 100 is physically divided into these “parts”.
  • These “units” can be realized as a computer program executed by the CPU, can be realized as an electronic circuit including an LSI or a memory, or can be realized by combining them. .
  • the captured image acquisition unit 101 is connected to the in-vehicle camera 2, acquires a captured image obtained by capturing the front of the vehicle 1 by the in-vehicle camera 2 at a certain time period, and sends it to the pedestrian detection unit 102 and the roadway detection unit 103. Output.
  • the pedestrian detection unit 102 detects a pedestrian in the captured image by analyzing the captured image received from the captured image acquisition unit 101.
  • various well-known methods such as detecting a part having the characteristics of a pedestrian in the image can be used.
  • the roadway detection unit 103 detects the roadway shown in the captured image by analyzing the captured image received from the captured image acquisition unit 101.
  • the roadway can be detected by detecting the white line from the captured image.
  • the navigation device 10 detects the current position of the vehicle 1 and stores map information including position information of the roadway. Therefore, when detecting the roadway in the captured image, the roadway detection unit 103 may acquire the current position and map information of the vehicle 1 from the navigation device 10 and detect the roadway while using these information.
  • the target pedestrian detection unit 104 detects the target pedestrian based on the detection result of the pedestrian acquired from the pedestrian detection unit 102 and the detection result of the roadway acquired from the roadway detection unit 103.
  • the target pedestrian is a pedestrian that is a target for determining whether or not the pedestrian is a pre-crossing pedestrian among the pedestrians detected by the pedestrian detection unit 102. For example, when a pedestrian is at a position sufficiently away from the roadway, it is obvious that the pedestrian is not trying to cross the roadway. In other words, for a pedestrian within a certain distance from the roadway, it may be determined whether the pedestrian is a pre-crossing pedestrian.
  • the target pedestrian detection unit 104 selects a pedestrian (that is, the target pedestrian) to be determined as to whether or not it is a pedestrian before crossing from the pedestrians detected by the pedestrian detection unit 102. It detects based on the information on the position of the road detected by the road detection unit 103.
  • the line-of-sight direction acquisition unit 105 acquires the direction of the line of sight for the target pedestrian detected by the target pedestrian detection unit 104.
  • the direction of the line of sight of the target pedestrian acquires the direction of the line of sight by analyzing the captured image acquired by the captured image acquisition unit 101 and detecting the direction of the face of the target pedestrian.
  • a dedicated device that detects the direction of the line of sight and wirelessly transmits it may be worn by the pedestrian to acquire the direction of the line of sight transmitted wirelessly from the pedestrian.
  • install a surveillance camera and an analysis device on the side of the road so that the direction of the line of sight obtained by analyzing the image of the pedestrian is transmitted wirelessly, and the direction of the line of sight transmitted wirelessly is acquired. You may do it.
  • the gaze determination unit 106 determines whether or not the target pedestrian gazes at the crossing destination existing on the other side of the roadway based on the direction of the line of sight of the target pedestrian and the information on the position of the roadway. To do.
  • the safety confirmation determination unit 107 determines whether the target pedestrian has performed safety confirmation on the vehicle traveling on the roadway based on the direction of the line of sight of the target pedestrian and the information on the position of the roadway. . A method of determining whether the target pedestrian has watched the crossing destination or whether the safety check has been performed will be described in detail later.
  • the pre-crossing pedestrian detection unit 108 crosses the target pedestrian when the target pedestrian performs the operation of watching the crossing destination and the operation of confirming the safety of the vehicle within a predetermined time. Detect as a front pedestrian.
  • a pedestrian trying to cross the roadway performs an operation of watching the destination of the crossing and an operation of confirming safety with respect to the vehicle. It is known to do before crossing. Therefore, if a target pedestrian who has performed these two movements within a predetermined time is detected as a pedestrian before crossing, the pedestrian who is about to cross the roadway is detected at a stage before the pedestrian starts crossing. It can be detected well.
  • the pre-crossing pedestrian detection unit 108 outputs the result of detecting the pre-crossing pedestrian using the monitor 3 or the speaker 4.
  • Pre-crossing pedestrian detection process 3 and 4 show detailed flowcharts of processing in which the pedestrian detection apparatus 100 of the present embodiment detects a pedestrian before crossing.
  • a captured image captured by the in-vehicle camera 2 is acquired (S100).
  • a pedestrian in the captured image is detected (S101), and further, a roadway is detected from the captured image (S102).
  • S101 a pedestrian in the captured image
  • S102 a roadway is detected from the captured image
  • a pedestrian in a captured image can be detected by searching for a part having the characteristics of a pedestrian in the captured image.
  • the roadway in the photographed image can be detected by extracting a portion where a white line existing on both sides of the roadway or a step on the road shoulder is extracted.
  • the target pedestrian is extracted from the pedestrians (S103).
  • the target pedestrian is a pedestrian that is a target for determining whether or not the pedestrian is before crossing.
  • a method for extracting the target pedestrian will be described using a specific example. For example, if the captured image is the image illustrated in FIG. 5, a pedestrian in the captured image is detected. An image 6a in FIG. 6 shows a state where eight pedestrians A to H are detected from the captured image. Subsequently, the roadway in the captured image is detected. In the image 6b of FIG. 6, the roadway detected from the captured image is displayed by hatching.
  • the in-vehicle camera 2 since the in-vehicle camera 2 is fixed with respect to the vehicle 1, the positions of the pedestrians A to H and the roadway on the photographed image can be read as relative positions based on the vehicle 1 in the real space. it can. Therefore, among the pedestrians A to H shown in the image 6a of FIG. 6, pedestrians whose distance to the roadway is within a predetermined distance are extracted as target pedestrians. This is because pedestrians who are far enough from the roadway are not considered to cross the roadway, so it is considered sufficient to extract pedestrians near the roadway as target pedestrians. Because. As a result, as shown in the image 6c of FIG. 6, seven pedestrians A to G excluding the pedestrian H among the eight pedestrians A to H in the photographed image are the target pedestrians. Extracted.
  • a situation where a pedestrian cannot be considered to cross the roadway may occur due to a fence provided at the center of the roadway.
  • a situation where a pedestrian is unlikely to cross the roadway because there is a sidewalk on one side of the roadway but no sidewalk on the other side. Therefore, when such a situation is detected by analyzing the captured image, other pedestrians may be extracted as target pedestrians except for pedestrians in such a situation.
  • a road range in which the pedestrian is unlikely to cross the roadway is stored in the map information of the navigation device 10. And even if it is a pedestrian within the predetermined distance from a roadway, you may extract a pedestrian other than that as an object pedestrian except the pedestrian in such a road range.
  • the target pedestrian is extracted from the pedestrians in the captured image as described above (S103 in FIG. 3), one target pedestrian is selected (S104), and the line-of-sight direction of the target pedestrian is acquired. (S105).
  • the line-of-sight direction is acquired by analyzing the image of the head of the target pedestrian in the captured image and detecting the orientation of the face.
  • the direction of the line of sight may be acquired by detecting the direction of the line of sight of the pedestrian using a monitoring camera and an analysis device installed on the side of the road and receiving the detection result transmitted wirelessly.
  • it is good also as receiving a pedestrian's gaze direction by detecting the direction of a gaze by attaching a dedicated apparatus to a pedestrian and transmitting a detection result wirelessly.
  • the target pedestrian is a pedestrian whose distance to the roadway is within a predetermined distance, there is a roadway near the target pedestrian. Therefore, it is determined whether or not the target pedestrian is looking beyond the roadway.
  • the line of sight of the target pedestrian 7a (face orientation in the present embodiment) is orthogonal to the road in a state where the road crosses the front of the body of the target pedestrian 7a. If it is, it can be determined that the line-of-sight direction of the target pedestrian 7a faces the direction of the crossing destination.
  • the line-of-sight direction of the target pedestrian 7b is relative to the direction orthogonal to the roadway.
  • the line-of-sight direction of the target pedestrian 7b faces the direction of the crossing destination.
  • the line-of-sight direction of the target pedestrian 7c is perpendicular to the roadway.
  • it can be determined that the line-of-sight direction of the target pedestrian 7c does not face the direction of the crossing destination.
  • the gaze flag is a flag indicating that the target pedestrian gazes at the crossing destination existing on the other side of the roadway.
  • the gaze flag is set for each target pedestrian (in the example shown in FIG. 6, for each of the pedestrians A to G).
  • a predetermined time or longer for example, 100 msec or longer. Therefore, if the gaze direction of the target pedestrian stays in the direction of the crossing destination that exists on the other side of the roadway for more than a predetermined gaze time, it is considered that the crossing destination is watched. good. Therefore, in such a case, the gaze flag for the selected target pedestrian is set to ON.
  • the gaze flag for the selected target pedestrian is set to OFF (S109). If the line-of-sight direction does not point in the direction of the crossing destination in the first place (S106: no), the selected target pedestrian without determining whether or not the line-of-sight direction remains for more than the gaze time. Is set to OFF (S109).
  • the pedestrian detection device 100 of the present embodiment determines whether or not the target pedestrian has confirmed the safety of the vehicle traveling on the roadway (S110 in FIG. 4). That is, as described above, since the roadway exists near the target pedestrian, it is determined whether or not the target pedestrian has confirmed the safety of the vehicle traveling on the roadway. Whether or not the target pedestrian has confirmed safety is determined as follows based on the line-of-sight direction of the target pedestrian.
  • the line-of-sight direction of the target pedestrian moves in one direction more than a predetermined angle on the side of the roadway with respect to the target pedestrian within a predetermined moving speed range. If the target pedestrian makes a correct movement, it is determined that the target pedestrian has confirmed safety. This is because such a characteristic line-of-sight movement appears when a vehicle traveling on a roadway is chased with the eyes, so if such a line-of-sight movement is performed, safety for a vehicle traveling on the roadway This is because it is considered that confirmation was made.
  • the other direction also exceeds the confirmation time in the other direction.
  • a characteristic movement of staying over it is determined that the target pedestrian has confirmed safety.
  • even when such movement of the line of sight is repeated it is determined that the target pedestrian has confirmed safety. This is because such a characteristic line-of-sight movement appears when a pedestrian confirms left and right safety, so if he / she has such a line-of-sight movement, he will drive on the road This is because it is considered that the vehicle safety was confirmed.
  • the predetermined confirmation time used for determining whether or not the pedestrian has confirmed left and right safety can be the same time as the above-described gaze time (for example, a time of 100 msec or more).
  • the target pedestrian's line-of-sight direction moves in a direction along the roadway and moves beyond the state orthogonal to the roadway.
  • the target pedestrian is judged to have confirmed safety. This is because such a characteristic line-of-sight movement appears when a pedestrian confirms the safety of the rear, so if such a line-of-sight movement is performed, the safety for the vehicle traveling on the roadway It is because it is thought that it confirmed.
  • the line-of-sight direction before such line-of-sight movement and the line-of-sight direction after moving beyond the state orthogonal to the roadway are at an angle separated by a predetermined turning angle (typically 120 °) or more. It is desirable to confirm that there is.
  • the safety confirmation flag for the selected target pedestrian (that is, the target pedestrian selected in S104 in FIG. 3) is set to ON (S111).
  • the safety confirmation flag is a flag indicating that the target pedestrian has confirmed the safety of the vehicle traveling on the roadway.
  • the safety confirmation flag is also set for each target pedestrian, similarly to the gaze flag described above.
  • the safety confirmation flag for the selected target pedestrian is set to OFF (S112).
  • the target pedestrian corresponds to the pedestrian before crossing based on the history of the gaze flag and safety confirmation flag set so far It is determined whether or not (S113).
  • S113 A method for determining whether or not the target pedestrian corresponds to the pre-crossing pedestrian will be described later.
  • the pedestrian before crossing is a pedestrian who has an intention to cross the roadway but has not yet started crossing.
  • S114 when it judges that it corresponds to a pedestrian before crossing (S113: yes), it recognizes that the object pedestrian under selection is a pedestrian before crossing (S114).
  • the selected target pedestrian is determined not to be the pre-crossing pedestrian (S115).
  • the pedestrian detection apparatus 100 determines whether or not the target pedestrian corresponds to a pre-crossing pedestrian in the process of S113.
  • 11A to 11C show a basic concept for determining whether or not a target pedestrian corresponds to a pedestrian before crossing.
  • a pedestrian who crosses a roadway has a case where he / she pays attention to a destination for crossing immediately before the start of the crossing and a case where he / she checks safety on a vehicle traveling on the roadway.
  • the pedestrian does not confirm the safety to the vehicle.
  • the pedestrian detection device 100 determines whether or not the target pedestrian corresponds to a pre-crossing pedestrian based on the above-described concept.
  • FIG. 12A to 12C are explanatory diagrams showing a method for determining whether or not the target pedestrian corresponds to a pedestrian before crossing. As mentioned in S113 of FIG. 4, whether or not the target pedestrian corresponds to a pre-crossing pedestrian is determined based on the history of the gaze flag and the safety confirmation flag set so far.
  • FIG. 12A conceptually shows how the setting of the gaze flag and the safety confirmation flag switches over time for each of the target pedestrians A to G illustrated in FIG.
  • the black straight line extending from the left side to the right side indicates that neither the gaze flag nor the safety confirmation flag is set (that is, neither flag is set to OFF).
  • the attached part indicates that the gaze flag is set to ON.
  • a portion with a fine diagonal line indicates that the safety confirmation flag is set to ON.
  • the gaze flag is a flag that indicates that the crossing destination is watched
  • the safety confirmation flag is a flag that indicates that the safety confirmation of the vehicle traveling on the roadway has been performed. Since the gaze at the crossing destination and the safety confirmation to the vehicle cannot be performed at the same time, the gaze flag and the safety confirmation flag are not set to ON at the same time.
  • the safety confirmation flag is set to ON after a while after the gaze flag is set to ON. Therefore, as shown in FIG. 12B, at the time point a when the safety confirmation flag is set to ON, it is determined whether or not the gaze flag is set to ON while going back a predetermined retroactive time (for example, 3 seconds). To do. As shown in the figure, for the target pedestrian A, the gaze flag is not set to ON within the retroactive time that goes back from the time point a. For this reason, since one of the safety confirmation flag and the gaze flag is set to ON and the other flag is not turned ON within the retroactive time, it is determined not to correspond to the pedestrian before crossing.
  • the safety confirmation flag is set to ON immediately after the gaze flag is set to ON. For this reason, as shown in FIG. 12C, the gaze flag is set to ON within the retroactive time retroactive from the time point b when the safety confirmation flag is set to ON. Therefore, it is determined that the target pedestrian C corresponds to the pedestrian before crossing at the time point b. Further, for the target pedestrian C, the gaze flag is set to ON within the period that goes back the retroactive time as long as the safety confirmation flag is set to ON even after the time point b. For this reason, it is determined that the target pedestrian C corresponds to the pre-crossing pedestrian even after the time point b.
  • the gaze flag is turned on within the period of retroactive time from the time when the second gaze flag is turned on. Since it is set, it is determined that the target pedestrian C corresponds to the pedestrian before crossing. And after the second gaze flag is turned off, until the time point c, the gaze flag is set to ON and the safety confirmation flag is set to ON within the period of retroactive time. And exist. For this reason, the target pedestrian C continues to be determined to be a pedestrian before crossing until the time point c.
  • S113 of FIG. 4 it is determined whether or not the target pedestrian being selected (that is, the pedestrian selected in S104 of FIG. 3) corresponds to the pedestrian before crossing by the above method.
  • the target pedestrian being selected that is, the pedestrian selected in S104 of FIG. 3 corresponds to the pedestrian before crossing by the above method.
  • the selected target pedestrian is recognized as the pre-crossing pedestrian and the fact is stored (S114).
  • the selected target pedestrian is recognized as not being a pre-crossing pedestrian and is stored as such (S115).
  • FIG. 13 illustrates a state in which the extraction result of the pedestrian before crossing is output on the screen of the monitor 3.
  • a photographed image obtained by photographing the front of the vehicle 1 with the in-vehicle camera 2 is displayed.
  • eight pedestrians A to H are shown in the photographed image.
  • a marker image m indicated by a broken-line rectangle is displayed. Overlaid. Therefore, the driver of the vehicle 1 can immediately recognize that the pedestrian G who must pay particular attention among the eight pedestrians A to H is the pedestrian G.
  • the pre-crossing pedestrian exists as a result of extracting the pre-crossing pedestrian (S118: yes).
  • the speaker 4 outputs a sound to the effect (for example, “a pedestrian is about to cross”).
  • the pedestrian detection apparatus 100 of this embodiment can detect the pedestrian who is going to cross (that is, the pedestrian before crossing) before the pedestrian actually starts to cross the roadway.
  • the movement of the pedestrian watching the destination of the crossing and the movement of confirming the safety of the vehicle traveling on the roadway Is detected based on whether or not it has been performed within a predetermined time, so that it is possible to detect a pedestrian before crossing with high accuracy.
  • the pedestrian who is going to cross the roadway can be detected with high accuracy and at the stage before actually starting to cross the road.
  • the monitoring burden on pedestrians can be greatly reduced. For example, in the example shown in FIG.
  • the pedestrian detection apparatus 100 of the present embodiment outputs a sound to that effect from the speaker 4 (see FIG. 13). For this reason, the driver can recognize immediately that the pedestrian before crossing has been extracted even if he / she is driving while visually checking the front as usual. If the screen of the monitor 3 is seen at that stage, it can be immediately recognized by the marker image m which pedestrian is the pre-crossing pedestrian.
  • the direction of the target pedestrian's line of sight is the characteristic movement shown in FIG.
  • the target pedestrian estimated that the vehicle traveling on the road was chased with his eyes, and determined that the vehicle had been confirmed for safety.
  • the position of the vehicle traveling on the roadway may be detected to determine whether or not the direction of the target pedestrian's line of sight faces the direction of the vehicle. Then, when the direction of the target pedestrian's line of sight is also moving as the position of the vehicle is moved, the target pedestrian may determine that the safety check for the vehicle has been performed.
  • FIG. 14 shows an internal configuration of the pedestrian detection apparatus 150 of the first modification example.
  • the pedestrian detection device 150 of the first modification shown in FIG. 14 has a vehicle detection unit 109 added to the pedestrian detection device 100 of the present embodiment described above with reference to FIG. About a point, it is the same as that of the pedestrian detection apparatus 100 of this embodiment.
  • the vehicle detection unit 109 included in the pedestrian detection device 150 of the first modification analyzes the captured image of the in-vehicle camera 2 received from the captured image acquisition unit 101, detects the position of the vehicle in the captured image, and results thereof. Is output to the safety confirmation determination unit 107.
  • the vehicle detection unit 109 is described as detecting the position of the vehicle by analyzing the captured image, but various known methods are used as a method for the vehicle detection unit 109 to detect the position of the vehicle. be able to.
  • the position of the vehicle may be detected based on an output from a radar such as a millimeter wave or a sonar.
  • the safety confirmation determination unit 107 acquires the gaze direction of the target pedestrian from the gaze direction acquisition unit 105 and acquires the position of the vehicle from the vehicle detection unit 109. Then, when the line-of-sight direction of the target pedestrian moves following the movement of the vehicle, it is determined that the target pedestrian has confirmed safety with respect to the vehicle.
  • the pedestrian detection device 150 can detect the pedestrian before crossing with higher accuracy.
  • the safety confirmation flag and the gaze flag are ON within a period that extends a predetermined retroactive time from the current time point.
  • the target pedestrian was detected as a pedestrian before crossing.
  • the ON state continues to be valid for each of the safety confirmation flag and the gaze flag.
  • An effective duration may be set. And it is good also as detecting the object pedestrian whose ON state is continuing about any of a safety confirmation flag and a gaze flag as a pedestrian before crossing.
  • FIG. 15A to FIG. 15C are explanatory views showing a state in which a pre-crossing pedestrian is detected from the target pedestrians in the pre-crossing pedestrian detection process of the second modification described above.
  • the effective duration of a predetermined time is set at a time point d when the gaze flag of the target pedestrian is turned from ON to OFF.
  • the target pedestrian is detected as a pre-crossing pedestrian while the safety confirmation flag is ON within the period from the time point d until the effective duration time elapses.
  • the safety confirmation flag remains ON even after the effective duration time has elapsed from the time point d.
  • the effective continuation time during which the ON state of the gaze flag continues effectively ends, the pedestrian is not detected as a pre-crossing pedestrian after the effective continuation time has elapsed from the time point d.
  • an effective duration of a predetermined time is set at the time point e when the gaze flag of the target pedestrian is turned from ON to OFF, and within the period from the time point e until the effective duration time elapses.
  • the safety confirmation flag While the safety confirmation flag is ON, the target pedestrian is detected as a pedestrian before crossing.
  • the safety confirmation flag changes from ON to OFF the effective duration of the safety confirmation flag is set from this time f.
  • the effective duration of the safety confirmation flag is not shown. For this reason, even after the safety confirmation flag is turned off, the gaze flag and the safety confirmation flag remain on until the effective duration of the gaze flag elapses. The target pedestrian is detected as a pedestrian before crossing.
  • the case where the gaze flag is turned on first and then the safety confirmation flag is turned on has been described as an example.
  • the safety confirmation flag is turned on first, and then the gaze flag is turned on.
  • the same description applies to the case of turning on.
  • the effective duration may be made different between the gaze flag and the safety confirmation flag.
  • the effective duration of the safety confirmation flag may be set shorter than the effective duration of the gaze flag illustrated in FIG. 15A or 15B.
  • the gaze flag is a flag that indicates that the target pedestrian has watched the crossing destination
  • the safety confirmation flag is a flag that indicates that the target pedestrian has confirmed the safety of the vehicle. is there. And once the crossing destination is confirmed, it is unlikely that the situation will change over time, so pedestrians who want to cross the roadway will have to wait for a while after confirming the crossing destination. Even after elapse, it is easy to start crossing without confirming again.
  • the safety confirmation flag is set to be shorter than the effective duration of the gaze flag, it is possible to detect the pedestrian before crossing with higher accuracy.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Mechanical Engineering (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Traffic Control Systems (AREA)
  • Control Of Driving Devices And Active Controlling Of Vehicle (AREA)
PCT/JP2017/020809 2016-07-07 2017-06-05 歩行者検出装置、歩行者検出方法 Ceased WO2018008314A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
DE112017003414.0T DE112017003414T5 (de) 2016-07-07 2017-06-05 Fußgängererfassungsvorrichtung und Fußgängererfassungsverfahren
US16/315,411 US11011062B2 (en) 2016-07-07 2017-06-05 Pedestrian detection apparatus and pedestrian detection method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2016135404A JP6617651B2 (ja) 2016-07-07 2016-07-07 歩行者検出装置、歩行者検出方法
JP2016-135404 2016-07-07

Publications (1)

Publication Number Publication Date
WO2018008314A1 true WO2018008314A1 (ja) 2018-01-11

Family

ID=60912099

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2017/020809 Ceased WO2018008314A1 (ja) 2016-07-07 2017-06-05 歩行者検出装置、歩行者検出方法

Country Status (4)

Country Link
US (1) US11011062B2 (enExample)
JP (1) JP6617651B2 (enExample)
DE (1) DE112017003414T5 (enExample)
WO (1) WO2018008314A1 (enExample)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019156678A1 (en) * 2018-02-09 2019-08-15 Ford Global Technologies, Llc Pedestrian management systems and methods
WO2019240070A1 (ja) * 2018-06-11 2019-12-19 本田技研工業株式会社 動作確認システム
FR3088279A1 (fr) * 2018-11-12 2020-05-15 Alstom Transport Technologies Procede d'envoi d'une information a un individu situe dans l'environnement d'un vehicule
US10817730B2 (en) 2017-06-01 2020-10-27 Honda Motor Co., Ltd. Prediction apparatus, vehicle, prediction method, and non-transitory computer-readable storage medium
CN112249021A (zh) * 2020-10-14 2021-01-22 广州汽车集团股份有限公司 一种道路行人碰撞风险预测方法及其系统
CN113793497A (zh) * 2021-08-04 2021-12-14 东南大学 一种考虑多因素的行人过街行为预测方法
US11373534B2 (en) 2018-12-05 2022-06-28 Volkswagen Aktiengesellschaft Method for providing map data in a transportation vehicle, transportation vehicle and central data processing device
CN115984318A (zh) * 2023-03-20 2023-04-18 宝略科技(浙江)有限公司 一种基于特征最大关联概率的跨摄像头行人跟踪方法

Families Citing this family (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018204917A1 (en) 2017-05-05 2018-11-08 Ball Aerospace & Technologies Corp. Spectral sensing and allocation using deep machine learning
CN111052126B (zh) * 2017-09-04 2024-06-04 华为技术有限公司 行人属性识别与定位方法以及卷积神经网络系统
WO2019089591A1 (en) * 2017-10-30 2019-05-09 Mobileye Vision Technologies Ltd. Vehicle navigation based on human activity
JP7100998B2 (ja) * 2018-03-08 2022-07-14 本田技研工業株式会社 車両制御装置、車両制御方法、およびプログラム
JP6989418B2 (ja) * 2018-03-12 2022-01-05 矢崎総業株式会社 車載システム
US11182672B1 (en) * 2018-10-09 2021-11-23 Ball Aerospace & Technologies Corp. Optimized focal-plane electronics using vector-enhanced deep learning
US20200202133A1 (en) * 2018-12-21 2020-06-25 Continental Automotive Systems Inc. Pedestrian tracking system and method
US11851217B1 (en) 2019-01-23 2023-12-26 Ball Aerospace & Technologies Corp. Star tracker using vector-based deep learning for enhanced performance
US11412124B1 (en) 2019-03-01 2022-08-09 Ball Aerospace & Technologies Corp. Microsequencer for reconfigurable focal plane control
US11488024B1 (en) 2019-05-29 2022-11-01 Ball Aerospace & Technologies Corp. Methods and systems for implementing deep reinforcement module networks for autonomous systems control
US11303348B1 (en) 2019-05-29 2022-04-12 Ball Aerospace & Technologies Corp. Systems and methods for enhancing communication network performance using vector based deep learning
US11828598B1 (en) 2019-08-28 2023-11-28 Ball Aerospace & Technologies Corp. Systems and methods for the efficient detection and tracking of objects from a moving platform
WO2021059747A1 (ja) * 2019-09-26 2021-04-01 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 情報処理方法、プログラム及び情報処理装置
CN112329684B (zh) * 2020-11-16 2024-04-30 常州大学 一种基于注视检测和交通场景识别的行人穿越马路意图识别方法
US11597314B2 (en) * 2021-02-08 2023-03-07 Ford Global Technologies, Llc Vehicle lamp system comprising a computer adjusting the color or direction of a lamp based on a road user's gaze direction
US12269512B2 (en) 2021-06-03 2025-04-08 Y.E. Hub Armenia LLC Method for training a machine learning algorithm for predicting an intent parameter for an object on a terrain
CN115303300B (zh) * 2022-08-30 2024-11-01 重庆长安汽车股份有限公司 车辆控制方法、装置、设备及存储介质
JP2024114483A (ja) * 2023-02-13 2024-08-23 トヨタ自動車株式会社 情報処理装置
CN119649321B (zh) * 2025-02-20 2025-06-06 贵州道坦坦科技股份有限公司 基于计算机视觉的道路行人检测系统

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002074595A (ja) * 2000-08-29 2002-03-15 Hitachi Ltd 車両の安全運転支援システム
JP2005165422A (ja) * 2003-11-28 2005-06-23 Denso Corp 衝突可能性判定装置
JP2011123636A (ja) * 2009-12-10 2011-06-23 Hitachi Ltd 人横断支援車両システム及び人横断支援方法
JP2014059841A (ja) * 2012-09-19 2014-04-03 Daimler Ag 運転支援装置
JP2015185075A (ja) * 2014-03-26 2015-10-22 株式会社デンソー 街灯システム
WO2016098238A1 (ja) * 2014-12-19 2016-06-23 株式会社日立製作所 走行制御装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4536674B2 (ja) 2006-03-27 2010-09-01 本田技研工業株式会社 歩行者認識装置
JP4935795B2 (ja) 2008-10-22 2012-05-23 株式会社豊田中央研究所 歩行者飛び出し予測装置及びプログラム
US9507998B2 (en) 2011-06-13 2016-11-29 Toyota Jidosha Kabushiki Kaisha Pedestrian motion predicting device
US9604639B2 (en) * 2015-08-28 2017-03-28 Delphi Technologies, Inc. Pedestrian-intent-detection for automated vehicles
JP6279006B2 (ja) 2016-04-28 2018-02-14 株式会社三共 スロットマシン

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002074595A (ja) * 2000-08-29 2002-03-15 Hitachi Ltd 車両の安全運転支援システム
JP2005165422A (ja) * 2003-11-28 2005-06-23 Denso Corp 衝突可能性判定装置
JP2011123636A (ja) * 2009-12-10 2011-06-23 Hitachi Ltd 人横断支援車両システム及び人横断支援方法
JP2014059841A (ja) * 2012-09-19 2014-04-03 Daimler Ag 運転支援装置
JP2015185075A (ja) * 2014-03-26 2015-10-22 株式会社デンソー 街灯システム
WO2016098238A1 (ja) * 2014-12-19 2016-06-23 株式会社日立製作所 走行制御装置

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10817730B2 (en) 2017-06-01 2020-10-27 Honda Motor Co., Ltd. Prediction apparatus, vehicle, prediction method, and non-transitory computer-readable storage medium
WO2019156678A1 (en) * 2018-02-09 2019-08-15 Ford Global Technologies, Llc Pedestrian management systems and methods
JPWO2019240070A1 (ja) * 2018-06-11 2021-07-29 本田技研工業株式会社 動作確認システム
WO2019240070A1 (ja) * 2018-06-11 2019-12-19 本田技研工業株式会社 動作確認システム
JP7523347B2 (ja) 2018-06-11 2024-07-26 本田技研工業株式会社 動作確認システム
WO2020099270A1 (fr) * 2018-11-12 2020-05-22 Alstom Transport Technologies Procédé d'envoi d'une information à un individu situé dans l'environnement d'un véhicule
US11745647B2 (en) 2018-11-12 2023-09-05 Alstom Transport Technologies Method for sending information to an individual located in the environment of a vehicle
FR3088279A1 (fr) * 2018-11-12 2020-05-15 Alstom Transport Technologies Procede d'envoi d'une information a un individu situe dans l'environnement d'un vehicule
US11373534B2 (en) 2018-12-05 2022-06-28 Volkswagen Aktiengesellschaft Method for providing map data in a transportation vehicle, transportation vehicle and central data processing device
CN112249021A (zh) * 2020-10-14 2021-01-22 广州汽车集团股份有限公司 一种道路行人碰撞风险预测方法及其系统
CN112249021B (zh) * 2020-10-14 2022-06-24 广州汽车集团股份有限公司 一种道路行人碰撞风险预测方法及其系统
CN113793497A (zh) * 2021-08-04 2021-12-14 东南大学 一种考虑多因素的行人过街行为预测方法
CN115984318A (zh) * 2023-03-20 2023-04-18 宝略科技(浙江)有限公司 一种基于特征最大关联概率的跨摄像头行人跟踪方法
CN115984318B (zh) * 2023-03-20 2023-06-13 宝略科技(浙江)有限公司 一种基于特征最大关联概率的跨摄像头行人跟踪方法

Also Published As

Publication number Publication date
DE112017003414T5 (de) 2019-03-21
US20190213887A1 (en) 2019-07-11
JP2018005793A (ja) 2018-01-11
US11011062B2 (en) 2021-05-18
JP6617651B2 (ja) 2019-12-11

Similar Documents

Publication Publication Date Title
JP6617651B2 (ja) 歩行者検出装置、歩行者検出方法
CN109758167B (zh) 驾驶员状态检测装置
CN110419211B (zh) 信息处理装置、信息处理方法和计算机可读的存储介质
JP5863233B2 (ja) 運転者注視方向連動の前方衝突危険警報システム、その方法及びそれを利用した車両
JP5345350B2 (ja) 車両の運転支援装置
CN105083265B (zh) 物体识别装置及车辆
CN105365819B (zh) 驾驶辅助装置和驾驶辅助方法
JP3304687B2 (ja) 車両用走行車線認識装置及び障害物検出装置・道路逸脱報知装置
JP5863607B2 (ja) 歩行者警告装置
US11074818B2 (en) Drive assist device and drive assist method
US10618467B2 (en) Stereo image generating method using mono cameras in vehicle and providing method for omnidirectional image including distance information in vehicle
JP6651031B2 (ja) 車両ドア衝突防止装置および車両ドア衝突防止方法
JP4885793B2 (ja) 障害物監視装置
KR20200131832A (ko) 정보 처리 장치, 이동 장치 및 방법, 그리고 프로그램
CN106405835A (zh) 显示装置
CN108263376B (zh) 自动泊车系统和自动泊车方法
WO2014109016A1 (ja) 車両周辺表示装置
CN108327719A (zh) 辅助车辆行驶的方法及装置
JP4936060B2 (ja) 車両用走行支援装置
JP2017220178A (ja) 物体検出装置、及び物体検出方法
JP2017073008A (ja) 車両用運転支援システムおよび運転支援方法
JP2018084873A (ja) 逆走防止装置、逆走防止方法および逆走防止システム
KR101103526B1 (ko) 스테레오 카메라를 이용한 충돌회피 방법
JP2010097335A (ja) 運転意思推定装置
JP2011103058A (ja) 誤認識防止装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17823921

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 17823921

Country of ref document: EP

Kind code of ref document: A1