WO2021111858A1 - Moving body detection method, roadside device, and vehicle-mounted device - Google Patents

Abstract

[Problem] To enable accurately determining whether or not moving bodies detected by two observation devices (a roadside device, a vehicle-mounted device) are the same. [Solution] This roadside device 4 detects a moving body (a pedestrian, etc.) present on the road on the basis of the output of a roadside sensor 42, acquires position information of the moving body and detects behavior characteristics of the moving body, and transmits the position information and information relating to the behavior characteristics to the vehicle-mounted terminal 2. Further, the vehicle-mounted terminal detects a moving body present on the road on the basis of the output of a vehicle-mounted sensor 11, acquires position information of the moving body and detects behavior characteristics of the moving body, and performs identification processing to determine whether or not the moving bodies detected by the roadside device and the local device are the same, in accordance with whether or not the behavior characteristics received from the roadside device and the behavior characteristics generated by the local device match.

Description

Mobile detection methods, roadside devices, and in-vehicle devices

The present disclosure relates to a moving body detection method for detecting a moving body existing in the vicinity, a roadside device, and an in-vehicle device.

In vehicles such as self-driving cars, there is known a technology for detecting obstacles (pedestrians, vehicles, etc.) existing around the own vehicle by using in-vehicle sensors such as radar, lidar (LiDAR), and camera. On the other hand, in a safe driving support radio system using ITS (Intelligent Transport System), a roadside machine installed at an intersection exists on the road around the intersection using a roadside sensor such as a radar. It detects moving objects (pedestrians, vehicles, etc.) and provides information on moving objects to vehicles through road-to-vehicle communication. As a result, it is possible for the vehicle side to recognize a moving object that is out of sight from the vehicle.

However, when the information of the moving body detected by the roadside machine is provided to a vehicle such as an autonomous driving vehicle, it is determined that the moving body detected by the roadside machine and the moving body detected by the in-vehicle terminal are different. , Recognize moving objects doubly. In addition, it is not possible to determine whether or not the moving body detected by the roadside machine is out of sight, that is, whether or not it is actually visible from the vehicle. Therefore, in a vehicle such as an autonomous driving vehicle, appropriate driving control cannot be performed. Therefore, it is necessary to identify whether or not the pedestrians detected by the in-vehicle terminal and the roadside machine are the same.

In order to identify such a moving body, in addition to the position information of the moving body, information unique to the moving body is acquired by each of the roadside unit and the in-vehicle terminal in order to identify the moving body, and the distance between road vehicles is obtained. It is advisable to use communication to exchange the information between the roadside unit and the in-vehicle terminal.

As a technique for exchanging the information acquired by each of the roadside unit and the in-vehicle terminal between the roadside unit and the in-vehicle terminal, conventionally, the information obtained on the vehicle side and the information obtained on the infrastructure side are exchanged. A technique for processing as a common data format on the vehicle side is known (see Patent Document 1).

Japanese Unexamined Patent Publication No. 2001-283380

By the way, in the identification process for determining whether or not the moving bodies detected by the roadside unit and the in-vehicle terminal are the same, the position information, the moving direction, and the moving locus of the moving body are generally used as the identification information. The mobile body is identified by comparing the identification information acquired by each of the roadside unit and the in-vehicle terminal. However, in such an identification method, when the positioning error of the position information becomes large, the same moving body is erroneously determined as another moving body. Therefore, a technique capable of accurately performing the identification process of the moving body is desired, but in the above-mentioned conventional technique, there is no proposal for suppressing such an erroneous determination of the identification process, and the identification is performed accurately. There was a problem that it could not be done.

Therefore, in the present disclosure, a mobile body detection method, a roadside device, and a roadside device capable of accurately determining whether or not the mobile bodies detected by each of the two observation devices (roadside device, in-vehicle terminal) are the same. The main purpose is to provide an in-vehicle device.

In the moving body detection method of the present disclosure, the first observation device and the second observation device detect a moving body existing on the road based on the output of the sensor and acquire the position information of the moving body. At the same time, the behavior characteristics of the moving body are detected, and one of the first observation device, the second observation device, and a processing device different from the first and second observation devices is the first. The moving object detected by each of the first observation device and the second observation device depends on whether or not the behavior characteristics generated by each of the observation device and the second observation device match. The configuration is such that an identification process for determining whether or not they are the same is performed.

Further, the roadside device of the present disclosure detects a moving body existing on the road based on the output of the roadside sensor, acquires the position information of the moving body, and detects the behavior characteristics of the moving body. The configuration is such that the position information and the information regarding the behavior characteristics are transmitted to the in-vehicle device as mobile identification information.

Further, the in-vehicle device of the present disclosure detects a moving body existing on the road based on the output of the in-vehicle sensor, acquires the position information of the moving body, and detects the behavior characteristics of the moving body. Whether or not the moving body detected by each of the roadside device and the own device is the same, depending on whether or not the behavior characteristic received from the roadside device and the behavior characteristic generated by the own device match. The configuration is such that the identification process for determination is performed.

According to the present disclosure, by comparing the behavioral characteristics of the moving body detected by each of the two observation devices, the identification of the moving body, that is, the moving body detected by each of the two observation devices is the same. Whether or not it can be determined accurately.

Overall configuration diagram of the mobile detection system according to the first embodiment Explanatory drawing which shows the outline of the moving body detection processing and identification processing performed by the roadside machine 4 which concerns on 1st Embodiment Explanatory drawing which shows the outline of the behavior characteristic acquisition processing performed by the roadside machine 4 which concerns on 1st Embodiment Explanatory drawing which shows the outline of the radio wave characteristic image generation processing performed by the roadside machine 4 which concerns on 1st Embodiment Explanatory drawing which shows the outline of the image composition processing performed by the roadside machine 4 which concerns on 1st Embodiment Explanatory drawing which shows the outline of the moving body image extraction processing performed by the roadside machine 4 which concerns on 1st Embodiment Explanatory drawing which shows the outline of the position determination processing performed by the vehicle-mounted terminal 2 and the identification processing performed by the roadside machine 4 according to the first embodiment. A sequence diagram showing a procedure of processing performed by the mobile detection system according to the first embodiment. A block diagram showing a schematic configuration of a pedestrian terminal 5 according to the first embodiment. A block diagram showing a schematic configuration of the roadside machine 4 according to the first embodiment. A block diagram showing a schematic configuration of the vehicle 1 according to the first embodiment. A flow chart showing an operation procedure of the pedestrian terminal 5 according to the first embodiment. A flow chart showing an operation procedure of the roadside machine 4 according to the first embodiment. A flow chart showing an operation procedure of the roadside machine 4 according to the first embodiment. A flow chart showing an operation procedure of the in-vehicle terminal 2 according to the first embodiment. A flow chart showing an operation procedure of the in-vehicle terminal 2 according to the first embodiment. Explanatory drawing which shows the outline of the moving body detection system which concerns on 2nd Embodiment A block diagram showing a schematic configuration of a roadside machine 4 according to a second embodiment. A block diagram showing a schematic configuration of the vehicle 1 according to the second embodiment. A flow chart showing an operation procedure of the roadside machine 4 according to the second embodiment. A flow chart showing an operation procedure of the in-vehicle terminal 2 according to the second embodiment. Explanatory drawing which shows the outline of the moving body detection system which concerns on 3rd Embodiment A flow chart showing an operation procedure of the pedestrian terminal 5 according to the third embodiment. A flow chart showing an operation procedure of the roadside machine 4 according to the third embodiment. A flow chart showing an operation procedure of the in-vehicle terminal 2 according to the third embodiment. Explanatory drawing which shows the outline of the moving body detection system which concerns on 4th Embodiment Explanatory drawing which shows the outline of the moving body detection system which concerns on 5th Embodiment A block diagram showing a schematic configuration of a pedestrian terminal 5 according to a sixth embodiment.

In the first invention made to solve the above problems, the first observation device and the second observation device detect a moving body existing on the road based on the output of the sensor, and the moving body is detected. Any of the first observation device, the second observation device, and a processing device different from the first and second observation devices, which acquires the position information of the moving object and detects the behavior characteristics of the moving object. In each of the first observation device and the second observation device, depending on whether or not the behavior characteristics generated by each of the first observation device and the second observation device match. The configuration is such that an identification process is performed to determine whether or not the detected moving objects are the same.

According to this, by comparing the behavior characteristics of the moving body detected by each of the two observation devices, the identification of the moving body, that is, whether or not the moving body detected by each of the two observation devices is the same. It is possible to accurately determine whether or not.

Further, in the second invention, the first observation device and the second observation device have at least one of the amplitude of the vertical movement of the moving body, the period of the vertical movement, the moving direction, and the moving speed as the behavior characteristics. It is configured to detect whether or not.

According to this, the moving body can be accurately identified based on the behavior characteristics of the moving body.

Further, in the third invention, the terminal device held by the mobile body transmits an identification signal having radio wave characteristics assigned to the own device, and the first observation device and the second observation device receive the identification signal. The moving body is identified based on the radio wave characteristics of the identification signal.

According to this, in addition to the comparison of the behavior characteristics of the moving object detected by each of the first observation device and the second observation device, the radio wave characteristics detected by each of the two observation devices are compared. , The identification of the moving body can be performed more accurately. Further, even in a state of being shielded by an obstacle, the moving body can be identified based on the position of the radio wave transmission source, so that the accuracy of identifying the moving body can be improved.

Further, in the fourth invention, the first observation device and the second observation device generate a radio wave characteristic image that visualizes the radio wave characteristics of the received identification signal, and the position of the radio wave transmission source of the identification signal. Based on the information, the radio wave characteristic image is superimposed and drawn on the image captured by the camera to generate a moving body image including the radio wave characteristic image, and the first observation device, the second observation device, and the above. One of the processing devices is configured to compare the moving body image including the radio wave characteristic image acquired by each of the first observation device and the second observation device in the identification process.

According to this, by extracting the moving object image from the image taken by the camera, it is possible to appropriately acquire the radio wave characteristic image corresponding to the moving object. Further, by comparing the behavior characteristics of the moving body detected by each of the first observation device and the second observation device, and by comparing the moving body image including the radio wave characteristic image detected by each device. , It is possible to further improve the accuracy of identification of a moving object by distinguishing it from the reflected wave from the radio wave transmission source. Further, even when the radio wave characteristic image is the same for a plurality of moving bodies, the moving body can be identified based on the comparison of the moving body images including the radio wave characteristic image, so that the accuracy of identifying the moving body can be improved.

Further, in the fifth invention, the terminal device transmits the identification signal at a timing corresponding to the behavior characteristics of the moving body, and the first observation device and the second observation device receive the identification signal. The configuration is such that the behavior characteristics of the moving body are acquired based on the timing.

According to this, the behavior characteristics of the moving body such as vertical movement can be notified from the terminal device to the first observation device and the second observation device.

Further, in the sixth aspect of the present invention, the first observation device and the second observation device display the moving body on the captured image of the camera based on the position information of the moving body acquired based on the output of the sensor. By setting a detection frame and cutting out a region of the detection frame from the captured image, a moving object image is extracted, and any one of the first observation device, the second observation device, and the processing device can be used. In the identification process, the moving body images acquired by each of the first observation device and the second observation device are compared.

According to this, in addition to the comparison of the behavior characteristics of the moving body detected by each of the first observation device and the second observation device, the moving body images acquired by each of the two observation devices are compared. The identification of the moving body can be performed more accurately.

Further, in the seventh invention, any one of the first observation device, the second observation device, and the processing device of the first observation device and the second observation device in the identification process. The configuration is such that a plurality of the moving body images acquired at each time are compared.

According to this, it is possible to more appropriately identify a moving body based on a plurality of moving body images.

Further, in the eighth invention, the indicator light held by the moving body lights up with the lighting characteristics assigned to the own device, and the first observation device recognizes the indicator light from the captured image of the camera. The lighting characteristic is detected, and the moving body is identified based on the lighting characteristic.

According to this, it is possible to accurately identify a moving object by using the lighting characteristics of the indicator light as identification information.

Further, in the ninth invention, any of the first observation device, the second observation device, and the processing device is a moving body detected by the first observation device and the second observation device. When it is determined that the moving body detected in is the same, the first observation device is based on the distance from the first observation device to the moving body and the second observation device is based on the distance from the moving body. The position of the moving body is determined by selecting either the position information of the moving body acquired by the observation device 1 or the position information of the moving body acquired by the second observation device.

According to this, the position of the moving body can be determined with the more accurate position information.

Further, the tenth invention is a tracking mode in which, when any one of the first observation device, the second observation device, and the processing device succeeds in the identification process, the position of the moving body is determined by the tracking process. If the tracking process fails, the mode returns to the identification mode in which the position of the moving body is determined by the identification process.

According to this, the processing load can be reduced by omitting the identification process.

Further, in the eleventh invention, the first observation device determines whether or not the moving body that is the source of the message and the moving body detected by using the sensor are the same. When the identification process is performed and the first identification process is successful, a message including instruction information for ignoring the message from the terminal device held by the mobile body as having been identified is sent to the second observation device. A second identification process of transmitting and determining whether or not the moving body detected by the first observing device and the moving body detected by the own device are the same by the second observing device. And, based on the instruction information, ignoring the message from the terminal device held by the identified mobile body, the position information of the mobile body notified from the first observation device and the own device The position of the moving body is determined based on the acquired position information.

According to this, the highly accurate position information acquired by either the first observation device or the second observation device is adopted, and the less accurate position information notified from the terminal device of the mobile body is not adopted. can do.

Further, in the twelfth invention, the first observation device is a roadside device, and the second observation device is an in-vehicle device.

According to this, it is possible to accurately determine whether or not the moving objects detected by the roadside unit and the in-vehicle terminal are the same.

Further, in the thirteenth invention, the first observation device and the second observation device are roadside devices.

According to this, it is possible to accurately determine whether or not the moving objects detected by each of the two roadside machines are the same.

Further, in the fourteenth invention, based on the output of the roadside sensor, a moving body existing on the road is detected, the position information of the moving body is acquired, and the behavior characteristics of the moving body are detected. The configuration is such that the position information and the information related to the behavior characteristics are transmitted to the in-vehicle device as mobile identification information.

According to this, as in the first invention, it is possible to accurately determine whether or not the moving objects detected by the roadside machine and the in-vehicle terminal are the same.

Further, the fifteenth invention detects a moving body existing on the road based on the output of the vehicle-mounted sensor, acquires the position information of the moving body, detects the behavior characteristics of the moving body, and detects the roadside. It is determined whether or not the moving body detected by each of the roadside device and the own device is the same, depending on whether the behavior characteristic received from the device and the behavior characteristic generated by the own device match. The configuration is such that the identification process is performed.

According to this, as in the first invention, it is possible to accurately determine whether or not the moving objects detected by the roadside machine and the in-vehicle terminal are the same.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

(First Embodiment)
FIG. 1 is an overall configuration diagram of a mobile body detection system according to the first embodiment.

This moving object detection system detects moving objects (pedestrians, vehicles, etc.) existing on the road and supports the driving of vehicle 1 (autonomous driving vehicle). This moving object detection system includes an in-vehicle terminal 2 (vehicle-mounted device, second observation device) mounted on the vehicle 1, an automatic driving ECU 3 (travel control device), and a roadside device 4 (roadside device,) installed on the road. It includes a first observation device) and a pedestrian terminal 5 (pedestrian device) possessed by a pedestrian.

ITS communication is performed between the in-vehicle terminal 2, the pedestrian terminal 5, and the roadside device 4. This ITS communication is a wireless communication using a frequency band (for example, 700 MHz band or 5.8 GHz band) adopted in a safe driving support wireless system using ITS (Intelligent Transport System). In this ITS communication, a message including necessary information such as the position information of the vehicle 1 and the pedestrian is transmitted and received.

Of these ITS communications, the one performed between the in-vehicle terminals 2 is referred to as vehicle-to-vehicle communication, and the one performed between the roadside unit 4 and the in-vehicle terminal 2 is referred to as road-to-vehicle communication. Further, the in-vehicle terminal 2 and the roadside device 4 can also perform ITS communication (communication between pedestrians and vehicles, communication between pedestrians) with the pedestrian terminal 5.

The in-vehicle terminal 2 transmits and receives a message including position information and the like to and from another in-vehicle terminal 2 by ITS communication (vehicle-to-vehicle communication), determines the risk of collision between vehicles 1, and determines the risk of collision. If there is, a warning activation operation is performed for the driver. The alert activation operation may be performed using a car navigation device (not shown) connected to the in-vehicle terminal 2. Further, the in-vehicle terminal 2 transmits and receives a message to and from the pedestrian terminal 5 by ITS communication (pedestrian-vehicle communication), and determines the risk of collision between the pedestrian and the vehicle 1.

The automatic driving ECU 3 detects obstacles around the vehicle 1 based on the output of the vehicle-mounted sensor 11, detects the state of the vehicle 1, and controls the running of the vehicle 1.

The roadside unit 4 notifies the in-vehicle terminal 2 and the pedestrian terminal 5 of the existence of a vehicle 1 or a pedestrian located in the vicinity of the own device by ITS communication (road-to-vehicle communication, road-to-walk communication). This makes it possible to prevent a collision when turning left or right at an intersection outside the line of sight. In addition to this, the roadside machine 4 distributes traffic information to the in-vehicle terminal 2 and the pedestrian terminal 5.

The pedestrian terminal 5 transmits and receives a message including position information and the like to and from the in-vehicle terminal 2 by ITS communication (pedestrian-vehicle communication), determines the risk of collision between the pedestrian and the vehicle 1, and determines the risk of collision. If there is a danger, a warning activation action for pedestrians will be performed.

The vehicle 1 is equipped with an in-vehicle sensor 11 and an in-vehicle camera 12. The in-vehicle sensor 11 is a radar, a rider, or the like. The in-vehicle terminal 2 detects a pedestrian (moving body) existing on the road around the own vehicle based on the output of the in-vehicle sensor 11. The in-vehicle camera 12 photographs the road around the own device. The vehicle-mounted camera 12 can also be used as the vehicle-mounted sensor 11.

The roadside unit 4 includes a roadside sensor 42 and a roadside camera 43 in addition to an antenna 41 that transmits and receives radio waves for ITS communication. The roadside sensor 42 is a radar, a rider, a camera, or the like. The roadside machine 4 detects a pedestrian (moving body) existing on the road around the own device based on the output of the roadside sensor 42. The roadside camera 43 photographs the road around the own device. The roadside camera 43 can also be used as the roadside sensor 42.

In the present embodiment, an example in which the moving body to be processed is a pedestrian will be described, but the moving body to be processed may be a vehicle.

Next, the moving object detection process and the identification process performed by the roadside machine 4 according to the first embodiment will be described. FIG. 2 is an explanatory diagram showing an outline of a moving object detection process and an identification process performed by the roadside machine 4. The same process is performed on the in-vehicle terminal 2.

In the present embodiment, the roadside machine 4 detects a pedestrian (moving body) existing on the road around the own device based on the output of the roadside sensor 42 (moving body detection process).

At this time, first, based on the output of the roadside sensor 42, the relative position information of the pedestrian with respect to the own device, that is, the direction in which the pedestrian exists as seen from the roadside machine 4, and the walking from the roadside machine 4. The distance to the person is required. Then, the absolute position information (latitude, longitude) of the pedestrian is obtained based on the relative position information of the pedestrian and the position information (latitude, longitude) of the installation position of the own device.

Further, in the present embodiment, three-dimensional position information of the pedestrian, that is, two-dimensional position information of the pedestrian and height information are required based on the relative position information (direction, distance) of the pedestrian. .. As the two-dimensional position information, absolute position information (latitude, longitude) may be used, but position information on the horizontal plane (XY plane) of an appropriate coordinate system may be used. Further, the height information is information on the height from the road surface under the feet of the representative point of the pedestrian's body. The road surface under the feet of a pedestrian is set as a reference horizontal plane (XY plane). In addition, in order to detect the vertical movement according to the walking of the person, it is preferable to use the center point of the upper body of the person as a representative point.

Further, in the present embodiment, pedestrian distance information (distance from the roadside machine 4 to the pedestrian) is required based on the position information regarding the installation position of the roadside machine 4 and the position information of the pedestrian.

In the moving object detection process, the roadside machine 4 detects a pedestrian (moving object) from the image captured by the camera by image recognition using a machine learning model such as deep learning, and the pedestrian on the captured image. The position information (coordinates) of the pedestrian may be acquired, and the absolute position information (latitude, longitude) of the pedestrian may be acquired from the position information of the pedestrian on the captured image.

Further, in the present embodiment, in the vehicle-mounted terminal 2, it is determined whether or not the pedestrians detected by the vehicle-mounted terminal 2 and the roadside machine 4 are the same (identification process). In this identification process, the moving body identification information for identifying a pedestrian (moving body) is acquired by each of the in-vehicle terminal 2 and the roadside machine 4, and the moving body identification acquired by each of the in-vehicle terminal 2 and the roadside machine 4. Pedestrians are identified by comparing the information.

First, in the present embodiment, the pedestrian is identified by using the position information of the pedestrian acquired by each of the in-vehicle terminal 2 and the roadside machine 4 as the moving object identification information. Further, in the present embodiment, the pedestrian is identified by using the behavior characteristics of the pedestrian (amplitude and period of vertical movement of the body, moving direction, moving speed, etc.) as moving body identification information. Further, in the present embodiment, the pedestrian terminal 5 transmits an identification signal of the radio wave characteristic assigned to each of them, and the radio wave characteristic of the identification signal is used as the moving object identification information to identify the pedestrian. ..

That is, in the present embodiment, the position information of the pedestrian acquired by each of the in-vehicle terminal 2 and the roadside device 4 is compared, the behavior characteristics of the pedestrian are compared, and the radio wave characteristics are compared. If the items match, it is determined that the pedestrians detected by the in-vehicle terminal 2 and the roadside device 4 are the same. Therefore, even if it is determined that some of the comparison items do not match due to a measurement error or the like, it is sufficient if the other comparison items match, and the pedestrian can be identified accurately.

Here, when the pedestrian detected by the roadside machine 4 is not among the pedestrians detected by the in-vehicle terminal 2, that is, due to a shield such as another moving body (pedestrian, vehicle) or a building. Pedestrians who cannot be seen from the vehicle are excluded. In the example shown in FIG. 2, the pedestrian A is detected by both the roadside machine 4 and the in-vehicle terminal 2, but the pedestrian B is blocked by another vehicle and is not detected by the in-vehicle terminal 2, and is therefore the target of identification. Be outside.

When comparing the moving body identification information (position information, behavior characteristics, radio wave characteristics) acquired by each of the in-vehicle terminal 2 and the roadside machine 4, the moving body acquired by each of the in-vehicle terminal 2 and the roadside machine 4 is compared. When the difference in the identification information is within the permissible range (within the range of the error), it may be determined that the pedestrians are the same.

Further, in the present embodiment, the position information, behavior characteristics, and radio wave characteristics of the moving body (pedestrian) are used as the moving body identification information to identify the pedestrian, but other features such as detection are performed. It is also possible to use the size of the moved body as the moving body identification information.

Next, the behavior characteristic acquisition process performed by the roadside machine 4 according to the first embodiment will be described. FIG. 3 is an explanatory diagram showing an outline of the behavior characteristic acquisition process performed by the roadside machine 4. The same process is performed on the in-vehicle terminal 2.

In the present embodiment, the behavior characteristics of the pedestrian (moving body) detected by each of the vehicle-mounted terminal 2 and the roadside machine 4 are compared, and the pedestrians detected by each of the vehicle-mounted terminal 2 and the roadside machine 4 are the same. Whether or not it is determined (identification process).

In the present embodiment, the behavior characteristics include the amplitude and period of the pedestrian's body movement (vertical movement) in the vertical direction (Z-axis direction), and the pedestrian's movement direction and movement speed on the horizontal plane (XY plane). get. Since such behavior characteristics differ for each pedestrian (moving body), it can be used as pedestrian identification information. For example, in the case of a pedestrian, the body moves up and down during walking, so that the vertical movement is detected, but in the case of a vehicle, there is almost no vertical movement. In addition, the amplitude of vertical movement is small in the case of children, but the amplitude of vertical movement is large in the case of adults.

Further, the behavior characteristics of the pedestrian (moving body) are detected by using the three-dimensional position information of the pedestrian, that is, the two-dimensional position information of the pedestrian and the height information. Specifically, the amplitude and cycle of the vertical movement of the pedestrian's body are detected based on the change state of the pedestrian's height information. Further, the movement direction and the movement speed of the pedestrian are obtained based on the change state of the two-dimensional position information of the pedestrian.

In the present embodiment, the behavior characteristics of the pedestrian are detected based on the three-dimensional position information of the pedestrian acquired from the detection results of the roadside sensor 42 and the in-vehicle sensor 11, but the radio wave transmission acquired by the radio wave transmission source detection process is performed. The behavior characteristics of the pedestrian may be detected based on the position information of the source, that is, the position information of the radio wave characteristic image.

Alternatively, the movement locus obtained by connecting the positions of the pedestrians at each time may be acquired as a behavior characteristic. In this case, in addition to the two-dimensional movement locus of the pedestrian on the horizontal plane (XY plane), the three-dimensional movement locus of the pedestrian on the XYZ space may be acquired.

Next, the radio wave characteristic image generation process performed by the roadside machine 4 according to the first embodiment will be described. FIG. 4 is an explanatory diagram showing an outline of radio wave characteristic image generation processing performed by the roadside machine 4. The same process is performed on the in-vehicle terminal 2.

In the present embodiment, the pedestrian terminal 5 transmits an identification signal of the radio wave characteristic assigned to each person, and the pedestrian is identified by using this radio wave characteristic as the moving object identification information. In particular, in the present embodiment, the roadside machine 4 generates a radio wave characteristic image that visualizes the radio wave characteristics of the identification signal. This radio wave characteristic image is drawn in a color and shape corresponding to the radio wave characteristics of each radio wave transmission source (pedestrian terminal 5).

In the example shown in FIG. 4, the radio wave characteristic image is composed of a circular ring and a + mark. The + mark indicates the position of the detected radio wave transmission source (pedestrian terminal 5). The ring represents the frequency of the identification signal and the transmission cycle of the identification signal (the interval at which the identification signal is transmitted). The ring is drawn around the + mark.

For example, when the frequency is 700 MHz, the ring is drawn in red, and when the frequency is 800 MHz, the ring is drawn in blue. When the frequency is switched between 700 MHz and 800 MHz, the red ring and the blue ring are drawn in an overlapping state.

Further, when the transmission cycle is 100 ms, a concentric double ring is drawn, and when the transmission cycle is 1 s, a single ring is drawn. When the transmission cycle is switched between 100 ms and 1 s, the double ring and the single ring are drawn alternately.

In the present embodiment, the radio wave characteristic image is composed of a circular ring and a + mark, and the frequency and transmission cycle of the identification signal are the ring color (red, blue) and the form (single, double). However, this is just an example, and the radio wave characteristic image is not limited to such a configuration, and if the frequency and transmission cycle of the identification signal can be identified, other images can be used. Configuration is also possible.

Next, the image composition process performed by the roadside machine 4 according to the first embodiment will be described. FIG. 5 is an explanatory diagram showing an outline of the image composition processing performed by the roadside machine 4. The same process is performed on the in-vehicle terminal 2.

In the present embodiment, in the roadside unit 4, the relative radio wave transmission source based on the roadside unit 4 (observing subject) is based on the reception status of the identification signal transmitted from the pedestrian terminal 5 (radio wave transmission source). Position information can be obtained. Next, the roadside machine 4 acquires the position information (coordinates) of the radio wave transmission source on the captured image of the roadside camera 43 shown in FIG. 5A based on the relative position information of the radio wave transmission source. .. Then, based on the position information of the radio wave transmission source on the captured image, the radio wave characteristic image is superimposed and drawn on the position of the radio wave transmission source on the captured image, and the composite image shown in FIG. 5B is generated. ..

Also, when multiple radio wave characteristic images for each pedestrian are displayed at overlapping positions, the radio wave characteristic images are drawn in a semi-transparent state. As a result, the radio wave characteristics of each pedestrian can be recognized from the radio wave characteristic images even when the radio wave characteristic images of each pedestrian overlap.

Here, when the radio wave characteristic image is located within the detection frame of the pedestrian on the image captured by the camera, the radio wave characteristic image visualizes the identification signal transmitted from the pedestrian terminal 5. On the other hand, the radio wave characteristic image located outside the detection frame of the pedestrian is a radio wave of a radio wave transmission source other than the pedestrian terminal 5 or a reflected wave in which the identification signal transmitted from the pedestrian terminal 5 is reflected by a wall or the like. Therefore, it is possible to determine whether or not the radio wave transmission source of the radio wave characteristic image is the pedestrian terminal 5 depending on whether or not the radio wave characteristic image is located within the detection frame of the pedestrian.

Next, the moving body image extraction process performed by the roadside machine 4 according to the first embodiment will be described. FIG. 6 is an explanatory diagram showing an outline of a moving body image extraction process performed by the roadside machine 4. The same process is performed on the in-vehicle terminal 2.

In the present embodiment, in the roadside machine 4, the position information of the pedestrian on the captured image of the roadside camera 43 is obtained based on the relative position information (direction, distance) of the pedestrian acquired from the output of the roadside sensor 42. Desired. Then, as shown in FIG. 6A, a pedestrian detection frame is set on the captured image based on the position information of the pedestrian on the captured image (frame setting process).

Next, as shown in FIGS. 6 (B-1) and 6 (B-2), the image area of the detection frame of the moving body is cut out from the captured image, and the moving body image is extracted. At this time, the moving body image in which the radio wave characteristic image is superimposed and drawn can be acquired by cutting out the image area of the detection frame of the moving body in the state where the radio wave characteristic image is superimposed and drawn on the captured image.

By the way, by comparing the moving body images (including the radio wave characteristic images) acquired by each of the in-vehicle terminal 2 and the roadside machine 4, are the pedestrians detected by each of the in-vehicle terminal 2 and the roadside machine 4 the same? Determine whether or not (identification process). At this time, if the pedestrian (moving body) is hidden by a shield such as another pedestrian, the pedestrian cannot be sufficiently seen from the moving body image alone, so that the pedestrian cannot be properly identified. Pedestrians can be properly identified when the radio wave characteristic image is superimposed and drawn. Even when the pedestrian is hidden by a shield or the pedestrian terminal is hidden (for example, the pedestrian terminal is in a bag or pocket), the pedestrian terminal 5 due to the radio wave transmission characteristic. Can be detected as a radio wave transmission source, so that a radio wave characteristic image can be appropriately drawn on the image taken by the camera.

Next, the position determination process performed by the in-vehicle terminal 2 and the identification process performed by the roadside machine 4 according to the first embodiment will be described. FIG. 7 is an explanatory diagram showing an outline of a position determination process performed by the in-vehicle terminal 2 and an identification process performed by the roadside machine 4.

In the present embodiment, first, in the vehicle-mounted terminal 2, whether or not the pedestrian detected by the roadside machine 4 using the roadside sensor 42 and the pedestrian detected by the vehicle-mounted terminal 2 using the vehicle-mounted sensor 11 are the same. Is judged. When it is determined that the pedestrians detected by each of the roadside unit 4 and the in-vehicle terminal 2 are the same, the in-vehicle terminal 2 is one of the pedestrian position information acquired by each of the roadside unit 4 and the in-vehicle terminal 2. One is selected based on the high accuracy (reliability), and the position of the pedestrian is determined.

On the other hand, when it is determined that the pedestrians detected by the roadside machine 4 and the vehicle-mounted terminal 2 are not the same, the vehicle-mounted terminal 2 uses the position information acquired by each of the roadside unit 4 and the vehicle-mounted terminal 2 as a different person and mounts the vehicle. The position of the pedestrian is determined by the position information acquired by the terminal 2.

In particular, in the present embodiment, the vehicle-mounted terminal 2 selects the position information based on the distance from the observing subject (vehicle-mounted terminal 2, roadside unit 4) to the pedestrian (moving body). Specifically, it is considered that the closer the distance from the observing subject to the pedestrian is, the higher the accuracy (reliability) of the position information is in the in-vehicle terminal 2, and the distance from the roadside machine 4 to the pedestrian and the distance from the vehicle. Compare with the distance to the pedestrian, and select the position information acquired by the observer who has the shorter distance to the pedestrian.

Here, since the vehicle and the pedestrian move, which of the pedestrian position information acquired by the roadside machine 4 and the pedestrian position information acquired by the in-vehicle terminal 2 is selected is determined by the vehicle and the pedestrian. , And changes according to the positional relationship of the roadside machine 4. For example, when the vehicle is away from the pedestrian, the position information acquired by the roadside machine 4 is selected, and when the vehicle approaches the pedestrian, the position information acquired by the in-vehicle terminal 2 is selected.

Further, in the in-vehicle terminal 2, when it is determined that the pedestrian detected by the roadside machine 4 and the pedestrian detected by the in-vehicle terminal 2 are not the same, both position information is selected with each pedestrian as a different person. ..

In the present embodiment, the position information is selected based on the distance from the observing subject (vehicle-mounted terminal 2 and roadside device 4) to the pedestrian. For example, the roadside device 4 is always the vehicle-mounted terminal 2. When the accuracy of the position information is higher, the position information acquired by the roadside machine 4 may always be selected.

Further, in the present embodiment, the pedestrian terminal 5 periodically broadcasts a message including the position information of the pedestrian. Then, the in-vehicle terminal 2 and the roadside device 4 can acquire the position information of the pedestrian by receiving the message transmitted from the pedestrian terminal 5. However, the accuracy of the pedestrian position information acquired by the pedestrian terminal 5 is low. Therefore, in the in-vehicle terminal 2, the pedestrian detected by the roadside unit 4 using the roadside sensor 42 and the pedestrian possessing the pedestrian terminal 5 from which the message is transmitted are determined to be different persons, and the pedestrian May be recognized twice.

Therefore, in the present embodiment, the roadside machine 4 determines whether or not the pedestrian possessing the pedestrian terminal 5 from which the message is transmitted and the pedestrian detected by the roadside machine 4 are the same. Perform processing. Then, when the identification is successful, the roadside machine 4 instructs the in-vehicle terminal 2 to ignore the message from the pedestrian terminal 5 possessed by the identified pedestrian. Specifically, in the message transmitted from the roadside unit 4 to the in-vehicle terminal 2, the roadside unit 4 includes the pedestrian's position information, behavior characteristic information, and radio wave characteristic image as detection results for the identified pedestrian. The message ID (ignore message ID) given to the message transmitted from the pedestrian terminal 5 is added as instruction information to ignore the message from the corresponding pedestrian terminal 5. The message ID is information that identifies the pedestrian terminal 5 that is the source of the message.

As a result, in order to ignore the message from the identified pedestrian, the in-vehicle terminal 2 possesses the pedestrian detected by the roadside machine 4 using the roadside sensor 42 and the pedestrian terminal 5 from which the message is transmitted. It is possible to avoid recognizing a pedestrian twice by judging the pedestrian as a different person.

Next, the procedure of the processing performed by the mobile detection system according to the first embodiment will be described. FIG. 8 is a sequence diagram showing a procedure of processing performed by the mobile detection system.

The pedestrian terminal 5 periodically transmits an identification signal based on the radio wave characteristics (frequency, transmission cycle) assigned to the own device. Further, the pedestrian terminal 5 periodically transmits a message of ITS communication.

This message includes pedestrian position information (latitude, longitude), radio wave characteristic information (frequency, transmission cycle) regarding the identification signal transmitted from the own device, and a message ID that identifies the source of the message. ..

In the roadside machine 4, the above-mentioned mobile object detection process and behavior characteristic acquisition process are periodically performed. Further, when the roadside machine 4 receives the identification signal from the pedestrian terminal 5, the radio wave transmission source detection process, the radio wave characteristic image generation process, the image composition process, and the moving body image extraction process are performed. Further, when the roadside machine 4 receives the message from the pedestrian terminal 5, the identification process is performed. Then, the roadside machine 4 transmits a message of ITS communication.

This message includes pedestrian position information (latitude, longitude) detected by the roadside machine 4, a moving body image including a radio wave characteristic image, radio wave characteristic information, and moving body behavior information (amplitude, period, etc.). The message ID (message ID) included in the message received from the pedestrian terminal 5 as instruction information for ignoring the distance information (distance from the own device to the pedestrian) and the message from the pedestrian terminal 5 possessed by the identified pedestrian. Ignore message ID) and. If the identification is not successful, that is, if the pedestrian possessing the pedestrian terminal 5 from which the message is sent is different from the pedestrian detected by using the roadside sensor 42, the ignored message ID is Not added.

In the in-vehicle terminal 2, similarly to the roadside machine 4, the moving object detection process and the behavior characteristic acquisition process are periodically performed. Further, when the in-vehicle terminal 2 receives the identification signal from the pedestrian terminal 5, the radio wave transmission source detection process, the radio wave characteristic image generation process, the image composition process, and the moving body image extraction process are performed as in the roadside device 4. .. Further, when the in-vehicle terminal 2 receives the message from the pedestrian terminal 5 and the message from the roadside machine 4, the identification process and the position determination process are performed.

Next, the schematic configuration of the pedestrian terminal 5 according to the first embodiment will be described. FIG. 9 is a block diagram showing a schematic configuration of the pedestrian terminal 5.

The pedestrian terminal 5 includes an ITS communication unit 51, an identification signal transmission unit 52, a positioning unit 53, a memory 54, and a processor 55.

The ITS communication unit 51 transmits and receives a message between the in-vehicle terminal 2 and the roadside unit 4 by ITS communication (communication between pedestrians and roads). When the message is transmitted, the message is broadcast.

The identification signal transmission unit 52 transmits an identification signal based on the radio wave characteristics (frequency, transmission cycle) assigned to the own device at regular intervals.

The positioning unit 53 measures the position of its own device by a satellite positioning system such as GNSS (Global Navigation Satellite System), that is, GPS (Global Positioning System) or QZSS (Quasi-Zenith Satellite System), and position information of its own device. Get (latitude, longitude).

The memory 54 stores a program or the like executed by the processor 55.

By executing the program stored in the memory 54, the processor 55 performs various processes, for example, processes related to message transmission / reception by the ITS communication unit 51, processes related to transmission of the identification signal by the identification signal transmission unit 52, and the like. ..

Next, the schematic configuration of the roadside machine 4 according to the first embodiment will be described. FIG. 10 is a block diagram showing a schematic configuration of the roadside machine 4.

The roadside machine 4 includes an ITS communication unit 44, an identification signal receiving unit 45, a memory 46, and a processor 47, in addition to the roadside sensor 42 and the roadside camera 43.

The ITS communication unit 44 transmits and receives a message between the in-vehicle terminal 2 and the pedestrian terminal 5 by ITS communication (road-to-vehicle communication, road-to-step communication). When the message is transmitted, the message is broadcast.

The identification signal receiving unit 45 receives the identification signal transmitted from the pedestrian terminal 5.

The memory 46 stores a program or the like executed by the processor 47.

The processor 47 performs various processes related to information collection by executing the program stored in the memory 46. In the present embodiment, the processor 47 performs mobile detection processing, behavior characteristic acquisition processing, radio wave transmission source detection processing, radio wave characteristic image generation processing, image composition processing, mobile image extraction processing, identification processing, and the like.

In the moving object detection process, the processor 47 detects a pedestrian (moving object) based on the output of the roadside sensor 42 and acquires the position information of the pedestrian. Specifically, the processor 47 first acquires the relative position information (direction, distance) of the pedestrian with respect to the roadside machine 4 based on the output of the roadside sensor 42. Next, the processor 47 determines the absolute position information (latitude) of the pedestrian based on the relative position information (direction, distance) of the pedestrian and the position information (latitude, longitude) of the installation position of the own device. , Longitude).

Further, in the moving object detection process, the processor 47 acquires the three-dimensional position information of the pedestrian based on the relative position information (direction, distance) of the pedestrian. Further, the processor 47 bases the pedestrian's position information (latitude, longitude) and the position information of the installation position of the own device (latitude, longitude) on the pedestrian's distance information (distance from the own device to the pedestrian). ) To get.

In the behavior characteristic acquisition process, the processor 47 detects the behavior characteristics of the pedestrian (moving body) (amplification and period of vertical movement of the body, moving direction, moving speed, etc.) based on the three-dimensional position information of the pedestrian. And acquire the behavior characteristic information of the pedestrian.

In the radio wave transmission source detection process, the processor 47 detects the radio wave transmission source (pedestrian terminal 5) based on the reception status of the identification signal in the identification signal reception unit 45, and acquires the position information of the radio wave transmission source. ..

In the radio wave characteristic image generation process, the processor 47 detects the radio wave characteristics (frequency and transmission cycle) of the identification signal received by the identification signal receiving unit 45, and generates a radio wave characteristic image that visualizes the radio wave characteristics of the identification signal. ..

In the image composition process, the processor 47 generates a composite image in which the radio wave characteristic image is superimposed and drawn on the captured image of the roadside camera 43.

In the moving body image extraction process, the processor 47 detects the position information (detection) of the pedestrian on the captured image of the roadside camera 43 based on the relative position information (direction, distance) of the pedestrian with respect to the own device. Get the frame coordinates). Then, the processor 47 cuts out the image area of the detection frame of the moving body from the captured image based on the position information of the pedestrian on the captured image, and acquires the moving body image. At this time, the moving body image including the radio wave characteristic image can be acquired by cutting out the image area of the detection frame of the moving body in the state where the radio wave characteristic image is superimposed and synthesized on the captured image.

In the identification process, the processor 47 depends on whether or not the radio wave characteristics (frequency and transmission cycle) included in the message received from the pedestrian terminal 5 match the radio wave characteristics of the identification signal received from the pedestrian terminal 5. Therefore, it is determined whether or not the pedestrian possessing the pedestrian terminal 5 from which the message is transmitted and the pedestrian detected by the own device using the roadside sensor 42 are the same.

Next, the schematic configuration of the vehicle 1 according to the first embodiment will be described. FIG. 11 is a block diagram showing a schematic configuration of the vehicle 1.

The vehicle 1 includes a steering ECU 15, a driving ECU 16, and a braking ECU 17 in addition to the vehicle-mounted terminal 2, the automatic driving ECU 3, the vehicle-mounted sensor 11, and the vehicle-mounted camera 12.

The automatic driving ECU 3 is connected to the steering ECU 15, the driving ECU 16, and the braking ECU 17, and controls the steering ECU 15, the driving ECU 16, and the braking ECU 17 based on the detection result of the in-vehicle sensor 11 to automatically drive the vehicle 1 (autonomous driving). ) Is realized.

Here, the steering ECU 15 controls the steering mechanism of the own vehicle, the drive ECU 16 controls the drive mechanism (engine, electric motor, etc.) of the own vehicle, and the braking ECU 17 brakes the own vehicle. It controls the mechanism.

The in-vehicle terminal 2 includes an ITS communication unit 21, an identification signal receiving unit 22, a positioning unit 23, a memory 24, and a processor 25.

The ITS communication unit 21 transmits and receives a message between the pedestrian terminal 5, another in-vehicle terminal 2, and the roadside device 4 by ITS communication (pedestrian-to-vehicle communication, vehicle-to-vehicle communication, road-to-vehicle communication). When the message is transmitted, the message is broadcast.

The positioning unit 23 measures the position of its own device by GNSS, that is, a satellite positioning system such as GPS or QZSS, and acquires the position information (latitude, longitude) of its own device.

The identification signal receiving unit 22 receives the identification signal transmitted from the pedestrian terminal 5.

The memory 24 stores a program or the like executed by the processor 25.

The processor 25 performs various processes related to information collection by executing the program stored in the memory 24. In the present embodiment, the processor 25 performs mobile detection processing, behavior characteristic acquisition processing, radio wave transmission source detection processing, radio wave characteristic image generation processing, image composition processing, mobile image extraction processing, identification processing, position determination processing, and the like. Do.

In the moving object detection process, the processor 25 detects a pedestrian (moving object) based on the output of the in-vehicle sensor 11 and acquires the position information of the pedestrian. Specifically, the processor 25 first acquires the relative position information (direction, distance) of the pedestrian with respect to the own device based on the output of the vehicle-mounted sensor 11. Next, the processor 25 determines the absolute position information (latitude) of the pedestrian based on the relative position information (direction, distance) of the pedestrian and the position information (latitude, longitude) of the current position of the own device. , Longitude).

Further, in the moving object detection process, the processor 25 acquires the three-dimensional position information of the pedestrian based on the relative position information (direction, distance) of the pedestrian. Further, the processor 25 bases the pedestrian's position information (latitude, longitude) and the position information of the current position of the own device (latitude, longitude) on the pedestrian's distance information (distance from the own device to the pedestrian). ) To get.

In the behavior characteristic acquisition process, the processor 25 detects the behavior characteristics of the pedestrian (moving body) (amplification and period of vertical movement of the body, moving direction, moving speed, etc.) based on the three-dimensional position information of the pedestrian. And acquire the behavior characteristic information of the pedestrian.

In the radio wave transmission source detection process, the processor 25 detects the radio wave transmission source (pedestrian terminal 5) based on the reception status of the identification signal in the identification signal reception unit 22, and acquires the position information of the radio wave transmission source. ..

In the radio wave characteristic image generation process, the processor 25 detects the radio wave characteristics (frequency and transmission cycle) of the identification signal received by the identification signal receiving unit 22, and generates a radio wave characteristic image that visualizes the radio wave characteristics of the identification signal. ..

In the image composition process, the processor 25 generates a composite image in which the radio wave characteristic image is superimposed and drawn on the captured image of the vehicle-mounted camera 12.

In the moving body image extraction process, the processor 25 detects the position information (detection) of the pedestrian on the captured image of the in-vehicle camera 12 based on the relative position information (direction, distance) of the pedestrian with respect to the own device. Get the frame coordinates). Then, the processor 25 cuts out the image area of the detection frame of the moving body from the captured image based on the position information of the pedestrian on the captured image, and acquires the moving body image. At this time, the moving body image including the radio wave characteristic image can be acquired by cutting out the image area of the detection frame of the moving body in the state where the radio wave characteristic image is superimposed and synthesized on the captured image.

In the identification process, the processor 25 determines whether or not the pedestrian detected by the roadside machine 4 and the pedestrian detected by the own device are the same. At this time, the position information of the pedestrian included in the message received from the roadside machine 4 and the position information of the pedestrian acquired by the own device are compared, and the moving body included in the message received from the roadside machine 4 is also compared. The image (including the radio wave characteristic image) and the moving body image (including the radio wave characteristic image) generated by the own device are compared, and the pedestrian behavior characteristic information included in the message received from the roadside machine 4 is also used. , The behavior characteristic information of the pedestrian generated by the own device is compared with, and it is determined whether or not the pedestrians detected by the roadside machine 4 and the own device are the same.

In the position determination process, the processor 25 selects either the pedestrian position information acquired by the roadside machine 4 or the pedestrian position information acquired by the own device according to the result of the identification process, and selects the road. Determine the position of the pedestrians above. Specifically, when the pedestrians detected by the roadside machine 4 and the own device are the same, the pedestrian position information acquired by the roadside machine 4 and the pedestrian position information acquired by the in-vehicle terminal 2 Of these, the position information with the shorter distance to the pedestrian (higher accuracy) is selected. On the other hand, if the pedestrians detected by the roadside machine 4 and the own device are not the same, the position information acquired by the in-vehicle terminal 2 is selected.

In the present embodiment, the distance from the roadside machine 4 to the pedestrian is notified from the roadside machine 4 to the in-vehicle terminal 2, but the processor 25 of the in-vehicle terminal 2 and the position information regarding the installation position of the roadside machine 4 are used. , The distance from the roadside machine 4 to the pedestrian may be calculated based on the position information of the pedestrian.

Further, in the present embodiment, an example in which the moving body to be processed is a pedestrian is described, but the moving body to be processed may be a vehicle, and in this case, the identification signal is transmitted to the in-vehicle terminal 2. An identification signal transmitting unit is provided to transmit.

Next, the operation procedure of the pedestrian terminal 5 according to the first embodiment will be described. FIG. 12 is a flow chart showing an operation procedure of the pedestrian terminal 5.

In the pedestrian terminal 5, the identification signal transmitting unit 52 transmits an identification signal based on the radio wave characteristics (frequency, transmission cycle) assigned to the own device (ST101). Further, the positioning unit 53 measures the position of its own device by GNSS and acquires the position information of a pedestrian (ST102).

Next, in the pedestrian terminal 5, the processor 55 generates an ITS communication message (ST103). Then, the ITS communication unit 51 transmits a message (ST104). This message includes pedestrian position information (latitude, longitude), radio wave characteristic information (frequency, transmission cycle) regarding the identification signal transmitted from the own device, and a message ID that identifies the source of the message. ..

Next, the operation procedure of the roadside machine 4 according to the first embodiment will be described. 13 and 14 are flow charts showing the operation procedure of the roadside machine 4.

As shown in FIG. 13A, in the roadside machine 4, the processor 47 detects a pedestrian (moving object) existing on the surrounding road based on the output of the roadside sensor 42, and puts it on the surrounding road. It is determined whether or not there is a pedestrian (ST201).

Here, when a pedestrian is present on the surrounding road (Yes in ST201), the processor 47 determines the relative position information of the pedestrian with respect to the own device based on the output of the roadside sensor 42 (Yes). Orientation, distance) is acquired. Further, the processor 47 determines the pedestrian's position information (latitude, longitude) and the pedestrian's height based on the pedestrian's relative position information and the position information (latitude, longitude) of the current position of the own device. The information and the distance information of the pedestrian (distance from the own device to the pedestrian) are acquired (moving object detection process) (ST202).

Next, the processor 47 detects the behavior characteristics of the pedestrian (moving body) based on the position information and the height information (three-dimensional position information) of the pedestrian, and the behavior characteristic information of the pedestrian (vertical movement). Acquire (behavior characteristic acquisition processing) (ST203) to acquire the amplitude and period, the moving direction, and the moving speed.

Further, as shown in FIG. 13 (B), in the roadside machine 4, when the identification signal receiving unit 45 receives the identification signal transmitted from the pedestrian terminal 5 (Yes in ST211), the processor 47 receives the received identification. The radio wave characteristics (frequency and transmission cycle) of the signal are detected, and the position information of the radio wave transmission source is acquired (radio wave transmission source detection processing) (ST212).

Next, the processor 47 generates a radio wave characteristic image (radio wave source visualization image) that visualizes the radio wave characteristics of the identification signal for each pedestrian (radio wave characteristic image generation processing) (ST213).

Further, the processor 47 acquires a photographed image from the roadside camera 43 that photographs the surrounding road (ST214). Next, the processor 47 acquires the position information (coordinates) of the pedestrian on the captured image of the roadside camera 43 based on the relative position information (direction, distance) of the pedestrian with respect to the roadside machine 4. (ST215).

Next, the processor 47 generates a composite image in which the radio wave characteristic image is superimposed and drawn on the captured image of the roadside camera 43 based on the position information of the radio wave transmission source (image synthesis processing) (ST216). Then, the processor 47 cuts out an image area of the pedestrian detection frame from the image captured by the camera based on the position information of the pedestrian on the image captured by the camera, and obtains a moving body image (including a radio wave characteristic image). Acquire (moving body image extraction process) (ST217).

Further, as shown in FIG. 14, in the roadside machine 4, when the ITS communication unit 44 receives the message transmitted from the pedestrian terminal 5 (Yes in ST221), the processor 47 receives the message from the pedestrian terminal 5. A pedestrian who possesses the pedestrian terminal 5 from which the message is transmitted, depending on whether or not the radio wave characteristics (frequency and transmission cycle) included in the above match with the radio wave characteristics of the identification signal received from the pedestrian terminal 5. And, it is determined whether or not the pedestrian detected by the own device using the roadside sensor 42 is the same (identification process) (ST222).

Here, if the pedestrian possessing the pedestrian terminal 5 from which the message is transmitted and the pedestrian detected by using the roadside sensor 42 are the same (Yes in ST222), the processor 47 has been identified. As instruction information for ignoring the message from the pedestrian terminal 5 possessed by the pedestrian, a message of ITS communication to which a message ID (ignore message ID) included in the message received from the pedestrian terminal 5 is added is created ( ST223). Then, the ITS communication unit 44 transmits a message to the in-vehicle terminal 2 (ST225).

On the other hand, if the pedestrian possessing the pedestrian terminal 5 from which the message is transmitted and the pedestrian detected by using the roadside sensor 42 are different (No in ST222), the message to which the ignore message ID is not added is not added. Is created (ST224). Then, the ITS communication unit 44 transmits a message to the in-vehicle terminal 2 (ST225).

It should be noted that the message transmitted to the in-vehicle terminal 2 includes the pedestrian position information (latitude, longitude) detected by the roadside machine 4 and the movement including the radio wave characteristic image regardless of whether or not the ignore message ID is added. Body image, radio wave characteristic information, moving body behavior information, and distance information (distance from own device to pedestrian) are included.

Next, the operation procedure of the in-vehicle terminal 2 according to the first embodiment will be described. 15 and 16 are flow charts showing an operation procedure of the in-vehicle terminal 2.

As shown in FIG. 15A, in the vehicle-mounted terminal 2, the processor 25 detects a pedestrian (moving body) existing on the surrounding road based on the output of the vehicle-mounted sensor 11 and puts it on the surrounding road. It is determined whether or not there is a pedestrian (ST301).

Here, when a pedestrian is present on the surrounding road (Yes in ST301), the processor 25 determines the relative position information of the pedestrian with respect to the own device based on the output of the in-vehicle sensor 11 (Yes). Orientation, distance) is acquired. Further, the processor 25 determines the pedestrian position information (latitude, longitude) and the pedestrian height based on the pedestrian relative position information and the position information (latitude, longitude) of the current position of the own device. The information and the distance information of the pedestrian (distance from the own device to the pedestrian) are acquired (moving object detection process) (ST302).

Next, the processor 25 detects the behavior characteristics of the pedestrian (moving body) based on the position information and the height information (three-dimensional position information) of the pedestrian, and the behavior characteristics information of the pedestrian (vertical movement). Acquire (behavior characteristic acquisition processing) (ST303) to acquire the amplitude and period, the moving direction, and the moving speed.

Further, as shown in FIG. 15 (B), in the vehicle-mounted terminal 2, when the identification signal receiving unit 22 receives the identification signal transmitted from the pedestrian terminal 5 (Yes in ST311), the processor 25 receives the received identification. The radio wave characteristics (frequency and transmission cycle) of the signal are detected, and the position information of the radio wave transmission source is acquired (radio wave transmission source detection processing) (ST312).

Next, the processor 25 generates a radio wave characteristic image (radio wave source visualization image) that visualizes the radio wave characteristics of the identification signal for each pedestrian (radio wave characteristic image generation processing) (ST313).

Further, the processor 25 acquires a photographed image from the in-vehicle camera 12 that photographs the surrounding roads (ST314). Next, the processor 25 acquires the position information (coordinates) of the pedestrian on the captured image of the in-vehicle camera 12 based on the relative position information (direction, distance) of the pedestrian with respect to the vehicle (). ST315).

Next, the processor 25 generates a composite image in which the radio wave characteristic image is superimposed and drawn on the captured image of the vehicle-mounted camera 12 based on the position information of the radio wave transmission source (image composition processing) (ST316). Then, the processor 25 cuts out an image area of the pedestrian detection frame from the image captured by the camera based on the position information of the pedestrian on the image captured by the in-vehicle camera 12, and includes a moving body image (including a radio wave characteristic image). ) (Movement image extraction process) (ST317).

Further, as shown in FIG. 16, in the in-vehicle terminal 2, the ITS communication unit 21 receives the message transmitted from the pedestrian terminal 5 (Yes in ST321), and further receives the message transmitted from the roadside unit 4. Then (Yes in ST322), the processor 25 determines whether or not the message ID included in the message received from the pedestrian terminal 5 is the same as the ignored message ID included in the message received from the roadside device 4 (ST323). ..

Here, if the message ID included in the message of the pedestrian terminal 5 is the same as the ignore message ID (Yes in ST323), the processor 25 ignores the message of the pedestrian terminal 5 and the pedestrian. The location information included in the message of the terminal 5 is excluded from the target (ST324).

Next, the processor 25 determines whether or not the pedestrian detected by the roadside machine 4 and the pedestrian detected by the own device are the same (identification process) (ST325). At this time, the position information of the pedestrian included in the message received from the roadside machine 4 and the position information of the pedestrian acquired by the own device are compared, and the moving body included in the message received from the roadside machine 4 is also compared. The image (including the radio wave characteristic image) and the moving body image (including the radio wave characteristic image) generated by the own device are compared, and the pedestrian behavior characteristic information included in the message received from the roadside machine 4 is also used. , The behavior characteristic information of the pedestrian generated by the own device is compared with, and it is determined whether or not the pedestrian detected by the roadside machine 4 and the pedestrian detected by the own device are the same.

Here, if the moving body detected by the roadside machine 4 and the moving body detected by the own device are the same (Yes in ST325), the processor 25 acquires the position information of the pedestrian by the roadside machine 4. , And the position information of the pedestrian with the shorter distance (high accuracy) to the pedestrian is selected from the position information of the pedestrian acquired by the in-vehicle terminal 2 (moving body position determination process) (ST326).

On the other hand, if the pedestrian detected by the roadside machine 4 and the pedestrian detected by the own device are not the same (No in ST325), the pedestrian position information acquired by the roadside machine 4 and the pedestrian position information acquired by the processor 25, and The position information of the pedestrian acquired by the in-vehicle terminal 2 is used as another person, and the position information acquired by the in-vehicle terminal 2 is selected (moving body position determination process) (ST327).

The automatic driving ECU 3 acquires the position information of the pedestrian from the in-vehicle terminal 2, and controls the traveling of the own vehicle based on the position information of the pedestrian so as to avoid a collision with the pedestrian.

(Second Embodiment)
Next, the second embodiment will be described. It should be noted that the points not particularly mentioned here are the same as those in the above-described embodiment. FIG. 17 is an explanatory diagram showing an outline of the mobile body detection system according to the second embodiment.

In the first embodiment, the vehicle-mounted terminal 2 identifies a pedestrian by comparing the moving body images including the radio wave characteristic images generated by the vehicle-mounted terminal 2 and the roadside device 4, that is, the vehicle-mounted terminal 2 and the roadside device 4. It is determined whether or not the pedestrians detected in each of 4 are the same (identification process).

On the other hand, in the present embodiment, the vehicle-mounted terminal 2 identifies a pedestrian by comparing the moving body images that do not include the radio wave characteristic images generated by the vehicle-mounted terminal 2 and the roadside machine 4. The first point is to compare the position information of the pedestrian detected by each of the in-vehicle terminal 2 and the roadside machine 4, and to compare the behavior characteristics of the pedestrian detected by each of the in-vehicle terminal 2 and the roadside machine 4. It is the same as the embodiment.

Further, in the present embodiment, when the positional relationship with respect to the pedestrian is opposite between the roadside machine 4 and the in-vehicle terminal 2, one of the moving body images is flipped left and right and then compared.

Further, in the present embodiment, if the in-vehicle terminal 2 succeeds in identifying the pedestrian, that is, if it is determined that the pedestrian detected by the roadside machine 4 and the pedestrian detected by the in-vehicle terminal 2 are the same. After that, the latest position information and moving object image of the pedestrian are acquired by the tracking process of the pedestrian. Then, if the tracking process fails, the re-identification process is performed.

In the tracking process, pedestrians are tracked by image recognition. Specifically, the similarity between the person detected this time and the person detected last time is calculated by image recognition for the moving object image, and the person detected this time is associated with the person detected last time based on the similarity. This makes it possible to acquire the position information and the moving body image of the same pedestrian (moving body ID).

Next, the schematic configuration of the roadside machine 4 according to the second embodiment will be described. FIG. 18 is a block diagram showing a schematic configuration of the roadside machine 4.

In the present embodiment, similarly to the first embodiment (see FIG. 10), the roadside machine 4 includes the roadside sensor 42, the roadside camera 43, the ITS communication unit 44, the memory 46, and the processor 47, but the identification signal The receiving unit 45 is omitted.

The processor 47 performs the mobile detection process, the behavior characteristic acquisition process, the mobile image extraction process, and the identification process as in the first embodiment, but the radio wave source detection process, the radio wave characteristic image generation process, and the image synthesis. No processing is performed.

Next, the schematic configuration of the vehicle 1 according to the second embodiment will be described. FIG. 19 is a block diagram showing a schematic configuration of the vehicle 1.

In the present embodiment, as in the first embodiment (see FIG. 11), the vehicle 1 includes an in-vehicle terminal 2, an automatic driving ECU 3, an in-vehicle sensor 11, an in-vehicle camera 12, a steering ECU 15, a drive ECU 16, and a braking ECU 17. ..

The in-vehicle terminal 2 includes an ITS communication unit 21, a positioning unit 23, a memory 24, and a processor 25, as in the first embodiment, but the identification signal receiving unit 22 is omitted.

The processor 25 performs mobile detection processing, behavior characteristic acquisition processing, moving body image extraction processing, identification processing, and position determination processing, but does not perform radio wave source detection processing, radio wave characteristic image generation processing, and image composition processing. ..

Next, the operation procedure of the roadside machine 4 according to the second embodiment will be described. FIG. 20 is a flow chart showing an operation procedure of the roadside machine 4.

In the roadside machine 4, the processor 47 detects pedestrians (moving objects) existing on the surrounding roads based on the output of the roadside sensor 42, and determines whether or not there are pedestrians on the surrounding roads. (ST401).

Here, when a pedestrian is present on the surrounding road (Yes in ST401), the processor 47 determines the relative position information of the pedestrian with respect to the own device based on the output of the roadside sensor 42 (Yes). Orientation, distance) is acquired. Further, the processor 47 determines the pedestrian position information (latitude, longitude) and the pedestrian height based on the pedestrian relative position information and the position information (latitude, longitude) of the installation position of the own device. The information and the distance information of the pedestrian (distance from the own device to the pedestrian) are acquired (moving object detection process) (ST402).

Next, the processor 47 detects the behavior characteristics of the pedestrian (moving body) based on the position information and the height information (three-dimensional position information) of the pedestrian, and the behavior characteristic information of the pedestrian (vertical movement). Acquire (behavior characteristic acquisition process) (ST403) to acquire amplitude and period, movement direction, movement speed).

Further, the processor 47 acquires a photographed image from the roadside camera 43 that photographs the surrounding road (ST404). Next, the processor 47 acquires the position information (coordinates) of the pedestrian on the captured image of the roadside camera 43 based on the relative position information (direction, distance) of the pedestrian with respect to the roadside machine 4. (ST405).

Next, the processor 47 cuts out an image area of the pedestrian detection frame from the captured image of the roadside camera 43 based on the position information of the pedestrian on the captured image, and acquires a moving body image (moving body image). Extraction process) (ST406).

Next, the processor 47 generates a message for ITS communication (ST407). Then, the ITS communication unit 44 transmits a message to the in-vehicle terminal 2 (ST408). This message includes position information (latitude, longitude) about a pedestrian detected by the roadside machine 4, a moving body image, moving body behavior information, and distance information (distance from the own device to the moving body). Is done.

Next, the operation procedure of the in-vehicle terminal 2 according to the second embodiment will be described. FIG. 21 is a flow chart showing an operation procedure of the in-vehicle terminal 2.

As shown in FIG. 21 (A), in the vehicle-mounted terminal 2, the processor 25 detects a pedestrian (moving body) existing on the surrounding road based on the output of the vehicle-mounted sensor 11 and puts it on the surrounding road. It is determined whether or not there is a pedestrian (ST501).

Here, when a pedestrian is present on the surrounding road (Yes in ST501), the processor 25 determines the relative position information of the pedestrian with respect to the own device based on the output of the in-vehicle sensor 11 (Yes). Orientation, distance) is acquired. Further, the processor 25 determines the pedestrian position information (latitude, longitude) and the pedestrian height based on the pedestrian relative position information and the position information (latitude, longitude) of the current position of the own device. The longitude information and the distance information of the pedestrian (distance from the own device to the pedestrian) are acquired (moving object detection process) (ST502).

Next, the processor 25 detects the behavior characteristics of the pedestrian (moving body) based on the position information and the height information (three-dimensional position information) of the pedestrian, and the behavior characteristics information of the pedestrian (vertical movement). Acquire (behavior characteristic acquisition processing) (ST503) to acquire amplitude and period, movement direction, movement speed).

Further, the processor 25 acquires a photographed image from the in-vehicle camera 12 that photographs the surrounding roads (ST504). Next, the processor 25 acquires the position information (coordinates) of the pedestrian on the captured image of the in-vehicle camera 12 based on the relative position information (direction, distance) of the pedestrian with respect to the own vehicle. (ST505).

Next, the processor 25 cuts out an image area of the pedestrian detection frame from the image captured by the camera based on the position information of the pedestrian on the image captured by the vehicle-mounted camera 12, and acquires a moving body image (movement). Body image extraction process) (ST506).

Further, as shown in FIG. 21 (B), in the in-vehicle terminal 2, when the ITS communication unit 21 receives the message transmitted from the roadside unit 4 (Yes in ST511), the processor 25 is detected by the roadside unit 4. It is determined whether or not the moving body and the moving body detected by the own device are the same (identification process) (ST512). At this time, the position information of the pedestrian included in the message received from the roadside machine 4 and the position information of the pedestrian acquired by the own device are compared, and the moving body included in the message received from the roadside machine 4 is also compared. The image is compared with the moving object image generated by the own device, and the pedestrian behavior characteristic information included in the message received from the roadside machine 4 and the pedestrian behavior characteristic information generated by the own device are Is compared, and it is determined whether or not the moving body detected by the roadside machine 4 and the pedestrian detected by the own device are the same.

Here, if the pedestrian detected by the roadside machine 4 and the pedestrian detected by the own device are the same (Yes in ST512), the processor 25 acquires the position information of the pedestrian by the roadside machine 4. , And the position information of the pedestrian with the shorter distance (high accuracy) to the pedestrian is selected from the position information of the pedestrian acquired by the in-vehicle terminal 2 (moving body position determination process) (ST513).

Next, the processor 25 shifts from the identification mode in the initial state to the tracking mode (ST515). In this tracking mode, pedestrians are tracked by image recognition. Then, if the pedestrian tracking fails (Yes in ST516), the mode returns to the identification mode (ST517).

On the other hand, if the pedestrian detected by the roadside machine 4 and the pedestrian detected by the own device are not the same (No in ST512), the pedestrian position information acquired by the roadside machine 4 and the pedestrian position information acquired by the processor 25 With the pedestrian position information acquired by the in-vehicle terminal 2 as another person, both position information is selected (moving body position determination process) (ST514).

(Third Embodiment)
Next, the third embodiment will be described. It should be noted that the points not particularly mentioned here are the same as those in the above-described embodiment. FIG. 22 is an explanatory diagram showing an outline of the mobile body detection system according to the third embodiment.

In the first embodiment, the pedestrian (moving body) is identified based on the radio wave characteristics of the identification signal transmitted from the pedestrian terminal 5, but in the present embodiment, the pedestrian terminal 5 possessed by the pedestrian is used. An indicator light 57 is provided, and a pedestrian is identified based on the lighting characteristics (lighting color, etc.) of the indicator light 57.

The indicator light 57 is provided with a light source such as an LED. The indicator lamp 57 performs a lighting operation according to the lighting characteristics assigned to each of them. That is, the lighting characteristic of the indicator light 57 becomes the pedestrian terminal 5, that is, the information for identifying the pedestrian (terminal identification information, moving object identification information).

The characteristic of the indicator lamp 57 may be a lighting cycle or a blinking pattern in addition to the lighting color, or may be a combination of the lighting color and the lighting cycle.

Further, the indicator light 57 is attached to the outside of the pedestrian's clothes or luggage so that the pedestrian can be easily recognized from the outside by wireless communication or wired communication separately from the housing of the pedestrian terminal 5. The configuration may be such that it is connected to the terminal 5, but the indicator light 57 may be integrally provided in the housing of the pedestrian terminal 5.

In the roadside machine 4, the processor 47 performs a moving body detection process, a moving body image extraction process, a lighting characteristic acquisition process, an identification process, and the like.

In the moving object detection process, the processor 47 detects a pedestrian (moving object) existing on the surrounding road based on the output of the roadside sensor 42, and acquires the relative position information of the pedestrian.

In the moving body image extraction process, the processor 47 acquires the position information of the pedestrian on the captured image of the roadside camera 43 based on the relative position information of the pedestrian. Next, the processor sets a pedestrian detection frame on the captured image based on the position information of the pedestrian on the captured image (frame setting process). Then, the processor 47 cuts out the image area of the detection frame of the moving body from the captured image and extracts the moving body image.

In the lighting characteristic acquisition process, the processor 47 detects the pedestrian indicator light 57 by image recognition of the moving object image extracted by the moving object image extraction process, and acquires the lighting characteristic of the indicator light 57.

In the identification process, does the processor 47 match the lighting characteristics of the indicator light 57 included in the message received from the pedestrian terminal 5 with the lighting characteristics of the indicator light 57 recognized by the own device from the captured image of the roadside camera 43? Depending on whether or not, it is determined whether or not the pedestrian possessing the pedestrian terminal 5 from which the message is transmitted and the pedestrian detected by the own device using the roadside sensor 42 are the same.

In the in-vehicle terminal 2, the processor 25 performs identification processing and the like.

In the identification process, the processor 25 determines whether or not the message ID included in the message directly received from the pedestrian terminal 5 and the message ID of the pedestrian terminal 5 included in the message received from the roadside machine 4 match. Therefore, it is determined whether or not the pedestrian possessing the pedestrian terminal 5 from which the message is transmitted and the pedestrian notified of the highly accurate position information from the roadside machine 4 are the same.

Next, the operation procedure of the pedestrian terminal 5 according to the third embodiment will be described. FIG. 23 is a flow chart showing an operation procedure of the pedestrian terminal 5.

In the pedestrian terminal 5, the indicator light 57 performs a lighting operation with the lighting characteristics (color, etc.) assigned to the own device (ST601). Further, the positioning unit 53 measures the position of the own device and acquires the position information of the pedestrian (ST602).

Next, in the pedestrian terminal 5, the processor 55 creates a message for ITS communication (ST603). Then, the ITS communication unit 51 transmits a message (ST604). This message includes pedestrian position information (latitude, longitude), information on the lighting characteristics of the indicator lamp 57, and a message ID that identifies the source of the message. The message transmitted from the pedestrian terminal 5 is received by both the roadside machine 4 and the in-vehicle terminal 2.

Next, the operation procedure of the roadside machine 4 according to the third embodiment will be described. FIG. 24 is a flow chart showing an operation procedure of the roadside machine 4.

As shown in FIG. 24 (A), in the roadside machine 4, the processor 47 detects a pedestrian (moving body) existing on the surrounding road based on the output of the roadside sensor 42, and on the surrounding road. It is determined whether or not there is a pedestrian (ST701).

Here, when a pedestrian is present on the surrounding road (Yes in ST701), the processor 47 determines the relative position information of the pedestrian with respect to the own device based on the output of the roadside sensor 42 (Yes). Orientation, distance) is acquired. Further, the processor 47 acquires (moves) the pedestrian's position information (latitude, longitude) and the like based on the pedestrian's relative position information and the position information (latitude, longitude) of the current position of the own device. Body detection processing) (ST702).

Further, the processor 47 acquires a photographed image from the roadside camera 43 that photographs the surrounding road (ST703). Next, the processor 47 acquires the position information (coordinates) of the pedestrian on the captured image of the roadside camera 43 based on the relative position information (direction, distance) of the pedestrian with respect to the roadside machine 4. (ST704).

Next, the processor 47 cuts out an image area of the pedestrian detection frame from the image captured by the camera based on the position information of the pedestrian on the image captured by the roadside camera 43, and acquires a moving body image (movement). Body image extraction process) (ST705).

Next, the processor 47 acquires the lighting characteristics of the indicator lamp 57 possessed by the pedestrian reflected in the moving body image by image recognition for the moving body image (lighting characteristic acquisition process) (ST706).

Further, as shown in FIG. 24 (B), in the roadside machine 4, when the ITS communication unit 44 receives the message transmitted from the pedestrian terminal 5 (Yes in ST711), the processor 47 starts from the pedestrian terminal 5. The lighting characteristics of the notified indicator light 57, that is, the lighting characteristics of the indicator light 57 included in the message received from the pedestrian terminal 5, and the lighting characteristics of the indicator light 57 recognized from the captured image of the roadside camera 43 by the own device. Is determined whether or not they match (ST712).

Here, if the lighting characteristics of the indicator light 57 notified from the pedestrian terminal 5 and the lighting characteristics of the indicator light 57 recognized by the own device match (Yes in ST712), the processor 47 uses the pedestrian terminal. Create an ITS communication message with the message ID (identified message ID) added to the message from No. 5 (ST713). Then, the ITS communication unit 44 transmits a message to the in-vehicle terminal 2 (ST715).

On the other hand, if the lighting characteristics of the indicator light 57 notified from the pedestrian terminal 5 and the lighting characteristics of the indicator light 57 recognized by the own device do not match (No in ST712), the processor 47 determines the identified message ID. Creates a message for ITS communication to which is not added (ST714). Then, the ITS communication unit 44 transmits a message to the in-vehicle terminal 2 (ST715).

The message transmitted to the in-vehicle terminal 2 includes, in addition to the identified ID, pedestrian position information acquired by the own device using the roadside sensor 42.

Next, the operation procedure of the in-vehicle terminal 2 according to the third embodiment will be described. FIG. 25 is a flow chart showing an operation procedure of the in-vehicle terminal 2.

In the in-vehicle terminal 2, when the ITS communication unit 21 receives the message transmitted from the pedestrian terminal 5 (Yes in ST801) and further receives the message transmitted from the roadside device 4 (Yes in ST802), the processor 25 However, it is determined whether or not the message ID included in the message directly received from the pedestrian terminal 5 matches the identified message ID included in the message received from the roadside device 4 (identification process) (ST803).

Here, if the message ID included in the message directly received from the pedestrian terminal 5 matches the identified message ID (Yes in ST803), the processor 25 ignores the message from the pedestrian terminal 5. , The position information of the pedestrian notified from the roadside machine 4, that is, the position information included in the message received from the roadside machine 4 is selected (moving body position determination process) (ST804).

On the other hand, if the message ID included in the message directly received from the pedestrian terminal 5 does not match the identified message ID (No in ST803), the processor 25 determines the position information of the pedestrian notified from the pedestrian terminal 5. That is, both the position information included in the message directly received from the pedestrian terminal 5 and the position information of the pedestrian notified from the roadside machine 4 are selected (moving body position determination process) (ST805).

In the example shown in FIG. 25, as an example of the case where the pedestrian indicator light 57 cannot be seen from the vehicle, the indicator light 57 is recognized only by the roadside machine 4, but the pedestrian indicator light 57 is recognized from the vehicle. When the 57 can be seen, the in-vehicle terminal 2 also recognizes the indicator light 57. In this case, the pedestrian is identified by comparing the lighting characteristics of the indicator lamp 57 acquired by each of the roadside machine 4 and the own device. Further, also in the present embodiment, as in the first embodiment, the position information of the pedestrian acquired by each of the roadside machine 4 and the own device is compared and the behavior characteristics of the pedestrian are compared.

(Fourth Embodiment)
Next, the fourth embodiment will be described. It should be noted that the points not particularly mentioned here are the same as those in the above-described embodiment. FIG. 26 is an explanatory diagram showing an outline of the mobile body detection system according to the fourth embodiment.

In the present embodiment, as in the second embodiment, the vehicle-mounted terminal 2 compares the moving body images (not including the radio wave characteristic image) generated by each of the vehicle-mounted terminal 2 and the roadside unit 4, so that the roadside unit 4 It is determined whether or not the pedestrian detected in 1 and the pedestrian detected by the in-vehicle terminal 2 are the same (identification process).

Particularly in this embodiment, pedestrians are identified using a plurality of moving body images. That is, the latest predetermined number of moving body images, specifically, the latest moving body image included in the latest predetermined period and the moving body image at each time in the past are to be compared. The roadside machine 4 and the in-vehicle terminal 2 can collect moving object images at each time for each pedestrian by tracking a person by image recognition or the like. Further, a plurality of moving body images to be compared may be compared in all combinations and determined to be the same in any combination.

In the example shown in FIG. 26, the roadside machine 4 obtains a moving body image at time t, t + 1, t + 2, and the in-vehicle terminal 2 obtains a moving body image at time t + 1, t + 2, and these moving body images are compared. By doing so, the pedestrian is identified. Further, since the positional relationship with respect to the pedestrian is opposite between the roadside machine 4 and the in-vehicle terminal 2, for example, the moving object image at time t + 1 of the roadside camera 43 and the moving object image at time t + 2 of the in-vehicle camera 12 are compared. When doing so, the moving body image at time t + 1 of the roadside camera 43 is flipped left and right and then compared.

The configuration of the roadside machine 4 according to this embodiment is the same as that of the second embodiment (see FIG. 18). The configuration of the in-vehicle terminal 2 is also the same as that of the second embodiment (see FIG. 19).

Further, the operation procedure of the roadside machine 4 is substantially the same as that of the second embodiment (see FIG. 20), but after the mobile image extraction process (ST406), the processor 47 moves to the pedestrian detected by the own device. A body ID is assigned, and the pedestrian's moving body ID, position information, and moving body image are stored in the memory 46. Then, the processor 47 collects the latest predetermined number of moving body images of the pedestrian based on the moving body ID, and generates an ITS communication message. This message includes the latest position information of the pedestrian, the latest predetermined number of moving body images of the pedestrian, and the like.

The operation procedure of the in-vehicle terminal 2 is substantially the same as that of the second embodiment (see FIG. 21), but after the mobile image extraction process (ST506), the processor 47 gives the pedestrian ID detected by the own device. Is added, and the moving body ID, the position information, and the moving body image of the pedestrian are stored in the memory 24. Further, in the identification process (ST512), the processor 47 compares the plurality of moving body images included in the message received from the roadside machine 4 with the plurality of moving body images generated by the own device.

(Fifth Embodiment)
Next, the fifth embodiment will be described. It should be noted that the points not particularly mentioned here are the same as those in the above-described embodiment. FIG. 27 is an explanatory diagram showing an outline of the mobile body detection system according to the fifth embodiment.

In the above embodiment, in the identification process, it is determined whether or not the moving bodies (pedestrians) detected by each of the in-vehicle terminal 2 and the roadside machine 4 are the same. On the other hand, in the present embodiment, the moving bodies (pedestrians) detected by each of the two roadside machines 4 (first and second observation devices) installed on both sides (diagonal positions) of the intersection are the same. Judge whether or not.

In this case, a moving body (pedestrian, vehicle) that cannot be detected by one roadside machine 4 because it is shielded by another moving body (vehicle or the like) or a shield such as a building may be detected by the other roadside machine 4. is there. Therefore, in one roadside machine 4, if the observation results of the two roadside machines 4 are integrated and transmitted to the in-vehicle terminal 2, the accuracy of identification of the moving body can be further improved.

(Sixth Embodiment)
Next, the sixth embodiment will be described. It should be noted that the points not particularly mentioned here are the same as those in the above-described embodiment. FIG. 28 is a block diagram showing a schematic configuration of the pedestrian terminal 5 according to the sixth embodiment.

In the above embodiment, the roadside machine 4 and the vehicle-mounted terminal 2 detect the behavior characteristics of the pedestrian based on the three-dimensional position information acquired from the outputs of the roadside sensor 42 and the vehicle-mounted sensor 11.

On the other hand, in the present embodiment, the pedestrian terminal 5 detects the behavior characteristics of the pedestrian, and the pedestrian terminal 5 notifies the roadside machine 4 and the in-vehicle terminal 2 of information on the behavior characteristics of the pedestrian. In particular, in the present embodiment, the pedestrian terminal 5 is provided with a behavior sensor 58. The behavior sensor 58 is an acceleration sensor, a gyro sensor, or the like, and detects the movement of the pedestrian's body.

In this case, for example, if the pedestrian terminal 5 transmits the identification signal according to the walking tempo of the pedestrian, the roadside unit 4 and the in-vehicle terminal 2 each transmit the identification signal based on the reception timing of the identification signal. Behavior characteristics (cycle of vertical movement) can be acquired. Further, when the radio wave characteristic image is drawn on the captured image of the roadside camera 43 or the in-vehicle camera 12 in response to the reception of the identification signal in each of the roadside unit 4 and the in-vehicle terminal 2, the radio wave characteristic image appears in the moving body image. It is possible to acquire the behavior characteristics of a pedestrian based on the timing of the image.

Further, in the pedestrian terminal 5, by changing the radio wave characteristics of the identification signal according to the moving direction of the pedestrian, walking is performed based on the radio wave characteristics of the identification signal received by each of the roadside unit 4 and the in-vehicle terminal 2. It is possible to acquire the behavior characteristics (movement direction) of a person. The logic of the identification signal, the transmission frequency, and the like may be changed according to the moving direction of the pedestrian.

By the way, in the above-described embodiment, the two observation devices perform the moving object detection process, and one of the observation devices acquires the detection result of the moving object from the other observation device and performs the identification process. Specifically, in the first embodiment and the like, the in-vehicle terminal 2 performs the identification process, and in the fifth embodiment, the roadside machine 4 performs the identification process.

On the other hand, a processing device different from the observation devices (vehicle-mounted terminal 2, roadside device 4) may acquire the detection results of the moving object from the two observation devices and perform the identification process. Here, the processing device is, for example, a server device connected to the observation device via an appropriate communication medium (for example, a cellular communication network). In this case, the processing device may also perform the position determination process.

Further, in the fifth embodiment, the two roadside devices 4 serve as observation devices that perform the moving object detection process, and one of the roadside devices 4 performs the identification process, but the two in-vehicle terminals 2 move. It may be an observation device that performs body detection processing, and one of the vehicle-mounted terminals 2 may perform identification processing. That is, one in-vehicle terminal 2 may acquire the detection result of the moving object from the other in-vehicle terminal 2 and perform the identification process.

Further, in the above-described embodiment, the roadside device 4 and the in-vehicle terminal 2 are observation devices that perform mobile object detection processing, but the pedestrian terminal 5 can be an observation device that performs mobile object detection processing. .. In this case, the pedestrian terminal 5 is provided with a sensor (radar or the like) for detecting the moving body and a camera for photographing the moving body.

Further, the two pedestrian terminals 5 may be an observation device that performs the moving object detection process, and one of the pedestrian terminals 5 may perform the identification process. Further, the roadside machine 4 and the pedestrian terminal 5 may be an observation device that performs a moving object detection process, and the pedestrian terminal 5 among them may perform the identification process. Further, the in-vehicle terminal 2 and the pedestrian terminal 5 may be an observation device that performs the moving object detection process, and either the in-vehicle terminal 2 or the pedestrian terminal 5 may perform the identification process.

As described above, an embodiment has been described as an example of the technology disclosed in this application. However, the technique in the present disclosure is not limited to this, and can be applied to embodiments in which changes, replacements, additions, omissions, etc. have been made. It is also possible to combine the components described in the above embodiments to form a new embodiment.

For example, in the above-described embodiment, the position information (latitude, longitude) of the pedestrian (moving body), the behavior characteristics (amplification and period of vertical movement of the body, the moving direction, the moving speed, etc.), and the radio wave characteristics of the identification signal are obtained. The pedestrian (moving body) is identified by using it as the moving body identification information for identifying the pedestrian (moving body), but other information may be used as the moving body identification information.

Further, the information collected by the roadside machine 4 or the in-vehicle terminal 2 can be used in the roadside machine 4 or the in-vehicle terminal 2 for other purposes in addition to being used for the identification process of the moving body. For example, in the case of children who go to and leave school in groups, their behavior histories (position, speed, direction, etc.) are similar to each other. In addition, in the case of a physically handicapped person, the behavior history is similar to that of an accompanying caregiver or guide dog. Therefore, when there are a plurality of moving objects having similar behavior histories, it is assumed that the person has a high risk such as a child or a physically handicapped person. Therefore, it is possible to detect a pedestrian with a high degree of risk by having similar behavior histories in a plurality of adjacent moving objects. As a result, a person with a high degree of risk can be recognized at an early stage on the vehicle side.

In addition, information on pedestrians with a high degree of risk may be detected by infrastructure equipment such as communication equipment installed in traffic lights and the like. In this case, a person with a high risk can be recognized earlier by setting the transmission power higher than usual from the infrastructure equipment to the vehicle and transmitting the power.

In addition to performing identification processing based on vehicle behavior characteristic information (speed, trajectory, etc.), vehicles that perform dangerous driving (sudden acceleration, deceleration, meandering, etc.) by determining the stability of behavior. Can be detected. As a result, it is possible to detect a vehicle that is driving in a hurry or drunk driving at an early stage and notify the driver or the police.

In addition, vehicle information for driving in a hurry or drunk driving may be notified to infrastructure equipment such as communication equipment installed at a traffic light or the like. As a result, it is possible to notify the pedestrian of the vehicle information at an early stage, and further, it is possible to avoid danger such as urging the pedestrian to stay away from the relevant part.

The moving body detection method, the roadside device, and the in-vehicle device according to the present disclosure accurately determine whether or not the moving bodies detected by each of the two observation devices (roadside device, in-vehicle terminal) are the same. It has the effect of being able to perform, and is useful as a moving body detection method for detecting moving bodies existing in the vicinity, a roadside device, an in-vehicle device, and the like.

1 Vehicle 2 In-vehicle terminal (in-vehicle device, second observation device, terminal device)
3 Automatic operation ECU (travel control device)
4 Roadside machine (roadside device, first observation device)
5 Pedestrian terminal (pedestrian device, terminal device)
11 In-vehicle sensor 12 In-vehicle camera 21 ITS communication unit 22 Identification signal reception unit 23 Positioning unit 24 Memory 25 Processor 42 Roadside sensor 43 Roadside sensor 44 ITS communication unit 45 Identification signal reception unit 46 Memory 47 Processor 51 ITS communication unit 52 Identification signal transmission unit 53 Positioning unit 54 Memory 55 Processor 57 Indicator light 58 Behavior sensor

Claims (15)

1. The first observation device and the second observation device
   Based on the output of the sensor, it detects a moving body existing on the road, acquires the position information of the moving body, and detects the behavior characteristics of the moving body.
   One of the first observation device, the second observation device, and a processing device different from the first and second observation devices.
   It was detected by each of the first observation device and the second observation device, depending on whether or not the behavior characteristics generated by each of the first observation device and the second observation device match. A moving object detection method characterized by performing an identification process for determining whether or not the moving objects are the same.
2. The first observation device and the second observation device
   The moving body detection method according to claim 1, wherein at least one of the vertical movement amplitude, the vertical movement cycle, the moving direction, and the moving speed of the moving body is detected as the behavior characteristic.
3. The terminal device held by the mobile body
   Sends an identification signal with the radio wave characteristics assigned to the own device,
   The first observation device and the second observation device
   The mobile object detection method according to claim 1, wherein a mobile object is identified based on the radio wave characteristics of the received identification signal.
4. The first observation device and the second observation device
   A radio wave characteristic image that visualizes the radio wave characteristics of the received identification signal is generated.
   Based on the position information of the radio wave transmission source of the identification signal, the radio wave characteristic image is superimposed and drawn on the image taken by the camera to generate a moving body image including the radio wave characteristic image.
   Any of the first observation device, the second observation device, and the processing device
   The mobile body detection method according to claim 3, wherein in the identification process, a mobile body image including the radio wave characteristic image acquired by each of the first observation device and the second observation device is compared. ..
5. The terminal device
   The identification signal is transmitted at a timing according to the behavior characteristics of the moving body, and the identification signal is transmitted.
   The first observation device and the second observation device
   The moving body detection method according to claim 3, wherein the behavior characteristics of the moving body are acquired based on the reception timing of the identification signal.
6. The first observation device and the second observation device
   Based on the position information of the moving body acquired based on the output of the sensor, the detection frame of the moving body is set on the captured image of the camera.
   By cutting out the region of the detection frame from the captured image, a moving body image is extracted.
   Any of the first observation device, the second observation device, and the processing device
   The mobile body detection method according to claim 1, wherein in the identification process, the moving body images acquired by each of the first observation device and the second observation device are compared.
7. Any of the first observation device, the second observation device, and the processing device
   The mobile body detection method according to claim 6, wherein in the identification process, a plurality of the moving body images acquired by each of the first observation device and the second observation device are compared. ..
8. The indicator light held by the moving body
   Lights up with the lighting characteristics assigned to the own device,
   The first observation device
   The moving body detection method according to claim 1, wherein the indicator lamp is recognized from an image captured by a camera, its lighting characteristics are detected, and a moving body is identified based on the lighting characteristics.
9. Any of the first observation device, the second observation device, and the processing device
   When it is determined that the moving body detected by the first observing device and the moving body detected by the second observing device are the same, the distance from the first observing device to the moving body, And, based on the distance from the second observation device to the moving body, either the position information of the moving body acquired by the first observation device or the position information of the moving body acquired by the second observation device. The moving body detection method according to claim 1, wherein the position of the moving body is determined by selecting the above.
10. Any of the first observation device, the second observation device, and the processing device
    When the identification process is successful, the process shifts to the tracking mode in which the position of the moving object is determined by the tracking process.
    The moving object detection method according to claim 1, wherein when the tracking process fails, the identification mode returns to the identification mode in which the position of the moving object is determined by the identification process.
11. The first observation device
    The first identification process for determining whether or not the moving body that is the source of the message and the moving body detected by using the sensor are the same is performed.
    When the first identification process is successful, a message including instruction information including instruction information for ignoring the message from the terminal device held by the moving body as having identified the corresponding moving body is transmitted to the second observing device.
    The second observation device
    A second identification process for determining whether or not the moving body detected by the first observation device and the moving body detected by the own device are the same is performed, and based on the instruction information, the second identification process is performed. Ignoring the message from the terminal device held by the identified mobile body, the movement is based on the position information of the moving body notified from the first observation device and the position information acquired by the own device. The moving object detection method according to claim 1, wherein the position of the body is determined.
12. The first observation device is a roadside device.
    The moving object detection method according to claim 1, wherein the second observation device is an in-vehicle device.
13. The moving object detection method according to claim 1, wherein the first observation device and the second observation device are roadside devices.
14. Based on the output of the roadside sensor, it detects a moving body existing on the road, acquires the position information of the moving body, and detects the behavior characteristics of the moving body.
    A roadside device characterized in that information regarding the position information and the behavior characteristics is transmitted to an in-vehicle device as moving object identification information.
15. Based on the output of the in-vehicle sensor, it detects a moving body existing on the road, acquires the position information of the moving body, and detects the behavior characteristics of the moving body.
    Whether or not the moving body detected by each of the roadside device and the own device is the same, depending on whether or not the behavior characteristic received from the roadside device and the behavior characteristic generated by the own device match. An in-vehicle device characterized by performing a determination identification process.

Images