WO2023188807A1 - Information processing device, information processing method, and computer program - Google Patents

Abstract

A device according to one embodiment of the present invention comprises: a communication unit that receives image data in which a traffic signal at an intersection is included in the depicted scene; and a control unit that generates signal information representing the color of the traffic signal light on the basis of the image data and transmits the signal information to the communication unit, wherein the image data is image data generated by a roadside camera installed so as to include the traffic signal in the angle of view.

Description

Information processing device, information processing method, and computer program

The present disclosure relates to an information processing device, an information processing method, and a computer program. This application claims priority based on Japanese Application No. 2022-56321 filed on March 30, 2022, and incorporates all the contents described in the said Japanese application.

Patent Document 1 discloses a technology that acquires position information of a signal lamp from map information and uses image data acquired by an on-vehicle camera that photographs the signal lamp existing at the relevant position to determine the current light color of the signal lamp. Are listed.
Patent Document 2 describes a technology that prompts a vehicle to stop when the current light color of a signal light device is determined by an on-vehicle camera to be yellow.

JP2020-24715A JP2017-37528A

A device according to an aspect of the present disclosure includes a communication unit that receives image data in which a traffic light device at an intersection is included in a scene, and a communication unit that generates signal information representing a light color state of the traffic light device based on the image data, and An information processing device comprising: a control unit that transmits signal information to the communication unit, wherein the image data is image data generated by a roadside camera installed so that the angle of view includes the signal lamp device. .

A device according to another aspect of the present disclosure includes a communication unit that receives data indicating a change in light color of a signal light device at an intersection, and a communication unit that generates signal information representing a light color state of the signal light device based on the data, and generates signal information representing a light color state of the signal light device based on the data. An information processing device comprising: a control unit that causes the communication unit to transmit information; the communication unit is capable of transmitting a communication packet including the signal information; and the communication packet is configured to generate the signal information. Contains additional information about the data used.

A computer program according to an aspect of the present disclosure includes a communication unit that receives image data in which a traffic light device at an intersection is included in a scene, and generates signal information representing a light color state of the traffic light device based on the image data. , a control unit that transmits the signal information to the communication unit, and a computer program that causes a computer to function as an information processing device, wherein the image data is installed so that the angle of view includes the signal lamp device. This is image data generated by a roadside camera.

A method according to an aspect of the present disclosure includes, as steps executed by an information processing device, a step of receiving image data in which a traffic light device at an intersection is included in a scenery, and determining a light color state of the traffic light device based on the image data. An information processing method comprising the steps of: generating signal information representing the signal; and transmitting the signal information to the outside, wherein the image data is acquired by a roadside camera installed so that the angle of view includes the signal lamp device. This is generated image data.

The present disclosure can be realized not only as a system and a device having the above-described characteristic configurations, but also as a program for causing a computer to execute such characteristic configurations. Further, the present disclosure can be realized as a semiconductor integrated circuit that realizes part or all of a system and a device.

FIG. 1 is a road plan view showing an example of the configuration of an information providing system. FIG. 2 is a block diagram showing an example of the hardware configuration of an edge computer. FIG. 3 is a block diagram showing an example of the functional configuration of an edge computer. FIG. 4 is an explanatory diagram illustrating an example of a method for setting a reference point for a lamp equipment area. FIG. 5 is a flowchart showing an example of the first check process. FIG. 6 is a flowchart showing a processing example of the second check. FIG. 7 is a flowchart showing a processing example of the third check. FIG. 8 is a flowchart showing a processing example of the fourth check. FIG. 9 is a flowchart showing a processing example of the fifth check. FIG. 10 is a flowchart showing a processing example of the sixth check. FIG. 11 is an explanatory diagram showing an example of the data structure of a communication packet for signal information.

<Issues that this disclosure seeks to solve>
In Patent Documents 1 and 2, the current light color of a traffic light device is determined based on image data captured by an on-vehicle camera. However, with vehicle-mounted cameras, it is difficult to capture signal lights due to large vehicles in front, and even if they are captured, the capture time is short. For this reason, it is difficult to accurately determine the current light color of the signal lamp device, and inaccurate signal information may be distributed.
In view of such conventional problems, the present disclosure aims to provide an information processing device and the like that can distribute accurate signal information.

<Effects of this disclosure>
According to the present disclosure, accurate signal information can be distributed.

<Summary of embodiments of the present disclosure>
Hereinafter, an overview of the embodiments of the present disclosure will be listed and explained.
(1) A device according to an aspect of the present embodiment includes a communication unit that receives image data in which a traffic light device at an intersection is included in a scene, and a communication unit that receives image data that includes a traffic light device at an intersection, and a communication unit that receives signal information representing a light color state of the traffic light device based on the image data. and a control unit that causes the signal information to be generated and to transmit the signal information to the communication unit, wherein the image data is generated by a roadside camera installed so that the angle of view includes the signal lamp device. This is image data.

According to the information processing device of the present embodiment, the image data used to generate the signal information is image data generated by a roadside camera installed so that the angle of view includes the signal lamp, and therefore is acquired by an on-vehicle camera. Compared to the case of using image data that has been captured, accurate image data with a longer capture time in which the signal lamp is reliably reflected can be obtained.
Therefore, the accuracy of determining the current light color of the signal lamp device can be improved, and accurate signal information can be distributed.

(2) In the information processing device of this embodiment, the communication unit is capable of transmitting a communication packet including the signal information, and the communication packet includes additional information related to a generation method of the signal information. Good too.

In this case, for example, by defining additional information representing the information source of the image data as a signal information generation method, the vehicle that receives the communication packet can determine whether the signal information is the roadside camera image data based on the above additional information. It can be seen that the information is based on Therefore, it is possible to notify the vehicle that the signal information is highly reliable information.

(3) In the information processing device of the present embodiment, the control unit determines an extraction range of the traffic light device, which is a pixel range that includes a reference point set in the image data by the user, and extracts pixel data within the extraction range. The current light color of the signal lamp device may be determined based on the above.

In this way, compared to the case where the current light color of the signal light device is determined from all the pixel data included in the image data, it is possible to suppress extraction errors of the signal light device and determine the current light color with high accuracy. In addition to being able to determine the current light color at high speed and with low load.

(4) In the information processing device of this embodiment, the control unit may execute a time consistency check to check the temporal consistency of the operation of the signal lamp device.

In this way, it is possible to prevent signal information from being generated based on the operation of a signal lamp device that lacks temporal consistency, and it is possible to improve the accuracy of signal information.

(5) In the information processing device of this embodiment, the time consistency check may include a first check of checking whether the change in light color of the signal lamp device corresponds to a prohibited operation.

In this way, it is possible to prevent signal information from being generated based on inaccurate lamp color changes that correspond to prohibited operations, and it is possible to improve the accuracy of signal information.

(6) In the information processing device of the present embodiment, the time consistency check includes a second check to check whether the light color change of the signal lamp device is simultaneous with a pre-calculated estimated time table. It's okay to stay.
In addition, when the operation manager of the signal lamp has disclosed a time table in which light color changes regarding the signal lamp are disclosed, the time table may be used instead of the estimated time table.

In this way, it is possible to prevent signal information from being generated based on inaccurate lamp color changes that are not simultaneous with the estimated timetable, and it is possible to improve the accuracy of signal information.

(7) In the information processing device of this embodiment, the control unit may execute a spatial consistency check that checks spatial consistency of the operation of the signal lamp device.

In this way, it is possible to prevent signal information from being generated based on the operation of a signal lamp device that lacks spatial consistency, and it is possible to improve the accuracy of signal information.

(8) In the present embodiment, the spatial consistency check may include a third check for checking the simultaneity of light color changes of the pair of signal lamps facing each other.

In this way, it is possible to prevent signal information from being generated based on inaccurate light color changes regarding a pair of opposing signal lamps that are not simultaneous, and it is possible to improve the accuracy of signal information.

(9) In the information processing device of this embodiment, the spatial consistency check is a fourth check that checks whether the light color changes of a pair of signal lamps oriented in mutually intersecting directions correspond to a prohibited relationship. May contain.

In this way, it is possible to prevent signal information from being generated based on inaccurate light color changes regarding a pair of signal lamps that fall under a prohibited relationship, and it is possible to improve the accuracy of signal information.

(10) In the information processing device of the present embodiment, the signal light device includes a vehicle light device and a pedestrian light device, and the spatial consistency check is performed based on a light color change of the vehicle light device and the pedestrian light device. A fifth check may be included to check whether the relationship between the light color changes of the lamp device violates the transition rule.

In this way, it is possible to prevent signal information from being generated based on inaccurate light color changes regarding vehicle lamps and pedestrian lamps that violate the transition rules, and improve the accuracy of signal information. I can do it.

(11) In the information processing device of the present embodiment, the spatial consistency check is performed when the light color change of the signal light device is located upstream of the stop line in the direction in which the signal light device provides the right of way, and the stop A sixth check may be included to check whether the behavior is consistent with the behavior of the vehicle closest to the line.

In this way, it is possible to check whether the light color determination of the signal lamp device based on the image data is correct based on the behavior of the actual vehicle (the leading vehicle) attempting to pass through the intersection.

(12) In the information processing device of the present embodiment, the communication unit is capable of transmitting a communication packet including the signal information, and the communication packet includes an identification indicating whether or not the signal information is to be corrected and the type of the correction. May contain information.

In this way, the vehicle that has received the communication packet will be able to determine whether the signal information is information that involves correction of lamp color determination, and if it involves correction, the type of correction.

(13) In the information processing device of the present embodiment, the control unit determines that there is no consistency in the time consistency check, and performs a spatial consistency check that checks the spatial consistency of the operation of the signal lamp device. If it is not possible, the light color state of the signal lamp device may be determined to be undefined.

In this way, it is possible to prevent signal information from being generated based on an inaccurate lamp color state, and it is possible to improve the accuracy of signal information.

(14) A device according to another aspect of the present embodiment includes a communication unit that receives data indicating a change in light color of a signal light device at an intersection, and generates signal information representing a light color state of the signal light device based on the data. and a control unit that causes the communication unit to transmit the signal information, wherein the communication unit is capable of transmitting a communication packet including the signal information, and the communication packet is configured to transmit the signal information. Contains additional information related to the information generation method.

According to the information processing device of this embodiment, for example, by defining additional information representing the information source of image data as a signal information generation method, a vehicle that has received a communication packet can generate signal information from the above additional information. It can be determined that the information is based on image data from roadside cameras. Therefore, it is possible to notify the vehicle that the signal information is highly reliable information.

(15) A computer program according to one aspect of the present embodiment is a computer program that causes a computer to function as the information processing apparatus of (1) to (13) above. Therefore, the computer program of this embodiment has the same effects as the information processing apparatuses (1) to (13) described above.

(16) A method according to one aspect of the present embodiment is an information processing method executed by the information processing apparatus of (1) to (13) above. Therefore, the information processing method of this embodiment has the same effects as the information processing apparatuses (1) to (13) described above.

<Details of embodiments of the present disclosure>
Hereinafter, details of embodiments of the present disclosure will be described with reference to the drawings. Note that at least some of the embodiments described below may be combined arbitrarily.

[Overall system configuration]
FIG. 1 is a road plan view showing an example of the configuration of an information providing system 1 according to the present embodiment. FIG. 2 is a block diagram showing an example of the hardware configuration of the edge computer 3. As shown in FIG.
As shown in FIGS. 1 and 2, the information providing system 1 includes a roadside camera 2 installed at a suitable location at an intersection J, an edge computer (information processing device) 3, and a terminal device 4 capable of wireless communication with the edge computer 3. , 5, etc.

A signal light device 6 is installed at the intersection J to indicate whether there is a right of way for the inflow road. The signal light device 6 is installed on a support located on the side of the road at the intersection J.
The signal light device 6 illustrated in FIG. 1 is a vehicle light device that includes at least a blue light, a yellow light, and a red light. The vehicle lamp may include an arrow signal light such as a right turn arrow light. Although not shown in FIG. 1, the intersection J may include a pedestrian light device.

The roadside camera 2 is installed on a pillar near the intersection J. The roadside camera 2 is a digital camera having an image sensor capable of capturing moving images. The roadside camera 2 can be adjusted to enlarge or reduce the angle of view, and is installed so that the signal lamp device 6 fits within the angle of view.
Therefore, the image data output by the roadside camera 2 includes at least one signal lamp 2 installed at the intersection J as a scenery reflected in the image data.

The roadside camera 2 may be installed so that one signal light device 6 fits within the field of view of one roadside camera 2, or two or more signal lights may fit within the field of view of one roadside camera 2. It may be installed so that the container 6 can be accommodated therein.
Further, in FIG. 1, two roadside cameras 2 are installed, but the number of roadside cameras 2 installed may be one or three or more.

The edge computer 3 is an information processing device that is communicably connected to the roadside camera 2, and is installed on a pillar near the intersection J. The connection method between the roadside camera 2 and the edge computer 3 may be wired or wireless.
The edge computer 3 has a setting function for the roadside camera 2, a function of generating predetermined downlink information from image data input from the roadside camera 2, and a function of distributing the generated downlink information to the terminal devices 4 and 5. have

The terminal device 4 is an in-vehicle terminal permanently or temporarily mounted on the vehicle 7. The vehicle 7 refers to all vehicles that travel on the road. Therefore, in addition to cars, light vehicles, and trolleybuses, motorcycles also fall under the category of vehicles.
The drive system of the vehicle 7 is not limited to an internal combustion engine, and includes electric vehicles and hybrid cars. The vehicle 7 may be a normal vehicle driven by the passenger himself or may be a level 3 or higher automated driving vehicle.

The terminal device 5 is a user terminal carried by a pedestrian 8 (see FIG. 2). The user terminal is, for example, a smartphone, a tablet computer, or a notebook computer.
The terminal devices 4 and 5 are capable of wireless communication with a wireless base station 9 (for example, a mobile base station). The wireless base station 9 can communicate with the edge computer 3 via a public communication network 10 including a mobile communication core network and the Internet.

The edge computer 3 transmits communication packets addressed to the terminal devices 4 and 5 containing downlink information to the public communication network 10. Therefore, communication packets containing downlink information are distributed to the terminal devices 4, 5, etc. via the wireless base station 9.
Here, instead of using the public wireless network 10, a DSRC (Dedicated Short Range Communication) type communication device (not shown) may be used as the means for transmitting the communication packet. Note that the DSRC method is also referred to as a short range communication method.
The downlink information includes signal information S1 of intersection J. The signal information S1 is information representing the light color state of the signal lamp device 6. In this embodiment, the signal information S1 represents the light color state for a predetermined period (for example, two cycles) up to the near future in addition to the current light color state. Specifically, the signal information S1 of the intersection J includes the following information 1 to information 5.

Information 1: Location and name of the intersection J to be provided Information 2: Location and name of the inflow road to which the traffic light device 6 to be provided provides the right of passage Information 3: Contents of the current light color of the relevant inflow road (which light device is lit)
Information 4: The lighting order of the relevant inflow route after the current point in time and the directions that can be passed when the light equipment is turned on. Information 5: Timing for turning on or turning off each light equipment (either the absolute time or the number of seconds remaining from the reference time).

However, the signal information S1 of the intersection J may be information representing only the current light color state of the signal lamp device 6, and in this case, the above information 1 to 3 are provided to the terminal devices 4 and 5.

[Hardware configuration of edge computer]
As shown in FIG. 2, the edge computer 3 includes a control section 31, a storage section 32, a communication section 33, a synchronization processing section 34, and a plurality of types of databases 35 to 37.
The databases 35 to 37 are electronic data constructed in the storage unit 32 in a predetermined data arrangement. However, part or all of the databases 35 to 37 may be constructed in an external storage device (not shown) connected to the edge computer 3.

The control unit 31 is an arithmetic processing device including a CPU (Central Processing Unit), a RAM (Random Access Memory), and the like. The control unit 31 may include an integrated circuit such as an FPGA (Field-Programmable Gate Array).
The control unit 31 reads the computer program 38 stored in the storage unit 32 into the main memory (RAM), and executes various information processing according to the program 38. The information processing includes processing for generating signal information S1 from image data of the roadside camera 2, and the like.

The storage unit 32 is an auxiliary storage device including nonvolatile memory such as an HDD (Hard Disk Drive) and an SSD (Solid State Drive).
The storage unit 32 may include a flash ROM (Read Only Memory), a USB (Universal Serial Bus) memory, an SD card, or the like.

The communication unit 33 is a communication interface that performs public communication via the public communication network 10 and dedicated communication with the roadside camera 2. Note that the communication section 33 may be composed of two communication modules.
Upon receiving the image data from the roadside camera 2, the communication unit 33 outputs the received image data to the control unit 31. Therefore, the communication unit 33 functions as an acquisition unit that imports image data whose information source is the roadside camera 2 into the edge computer 3.

The control unit 31 generates signal information S1 based on the input image data and outputs it to the communication unit 33. The communication unit 33 generates communication packets addressed to the terminal devices 4 and 5, including the signal information S1 input from the control unit 31, and transmits the generated communication packets to the wireless base station 9.
Therefore, the communication unit 33 functions as a distribution unit that distributes the signal information S1 to external devices such as the terminal devices 4 and 5. Note that the external device to which the information is distributed may be not only the terminal devices 4 and 5 but also a server (for example, a server of an insurance company that conducts an accident investigation).

The synchronization processing section 34 is a processing section for achieving time synchronization with other communication nodes such as the terminal devices 4 and 5 using a predetermined synchronization method.
The synchronization method of the synchronization processing unit 34 is, for example, a synchronization method based on the output of a GNSS (Global Navigation Satellite System) receiver, or a synchronization method using communication frames such as NTP (Network Time Protocol) and PTP (Precision Time Protocol). etc. can be adopted.

The plurality of types of databases 35 to 37 include a kin database (kin DB) 35, a time table database (time table DB) 36, and a transition rule database (transition rule DB) 37.
In the prohibition database 35, prohibited operations and prohibited relationships that must not occur in the signal lamp device 6 are recorded. The prohibited operation is a light color change that is prohibited for a single signal lamp 6, and includes, for example, an operation that changes from blue to red (without passing through yellow).

A prohibited relationship is a combination of light color states that is prohibited for a plurality of signal lamps 6. For example, a pair of signal lamps 6, 6 facing in mutually intersecting directions (for example, northbound direction and eastbound direction), This includes combinations of light color states that simultaneously turn green.
The estimated timetable is recorded in the timetable database 36. The estimated time table is defined as a table representing the transition order of light colors for each inflow route for one cycle, which is estimated from the light color changes for a predetermined time (for example, one hour) regarding all the signal lights 6 included in the intersection J. This is the data that will be used.

The transition rule database 37 records the transition rules of the signal lamp device 6. The transition rule is a rule for light color transition defined in advance between a vehicle lamp and a pedestrian lamp. The transition rules include, for example, the following rules 1 and 2.

Rule 1: In the case of intersection J where pedestrians and vehicles are separated, the right of way is not granted to both vehicle and pedestrian lamps at the same time.
Rule 2: If it is not an intersection J where pedestrians and vehicles are separated, the right of way for the vehicle light device and the pedestrian light device in the same direction of travel must be established within a predetermined time from the start of the pedestrian flashing. disappear (i.e. transition to yellow or red).

[Functional configuration of edge computer]
FIG. 3 is a block diagram showing an example of the functional configuration of the edge computer 3. As shown in FIG.
As shown in FIG. 3, the control unit 31 of the edge computer 3 includes a data input unit 41, an area setting unit 42, a lamp extraction unit 43, a light color determination unit 44, a light color check unit 45, an information generation unit 46, a sensor It includes an information input section 47 and a behavior determination section 48. These functional units 41 to 48 are realized by execution of the computer program 38 by the control unit 31.

Image data received by the communication unit 33 from the roadside camera 2 is input to the data input unit 41 . The data input unit 41 outputs the input image data to the lamp extraction unit 43 and the behavior determination unit 48.
The area setting unit 42 performs a process of setting a reference point for an area (hereinafter referred to as a "lamp area") where the signal lamp 6 exists among all the pixels forming the image data. This process is executed, for example, by the user's initial settings for the edge computer 3.

FIG. 4 is an explanatory diagram illustrating an example of a method for setting a reference point for a lamp equipment area. As shown in FIG. 4, any of the following methods 1 to 4 can be adopted as a method for setting the reference point.
Method 1: Specify the center point P1 of the signal light device 6 (the center point of the yellow light).
Method 2: Specify the center points P1 to P3 of the yellow light/blue light/red light.
Method 3: Specify the four vertices Pa to Pd when the signal lamp device 6 is approximated to a rectangle.
Method 4: Specify two diagonal vertices Pa and Pd when the signal lamp device 6 is approximated to a rectangle.

In method 2, in the case of the signal light device 6 having an arrow light, the center point of the arrow light may be specified. Furthermore, in method 4, the two diagonal vertices may be Pb and Pc.
Note that the method of inputting the reference point of the lamp area is not particularly limited, but for example, the coordinate value of the plane coordinate given to each pixel of the image data may be transmitted from an input device such as a keyboard connected to the edge computer 3. This can be achieved by

The lamp extraction unit 43 extracts the lights included in the signal lamp 6 (red lights, blue lights, yellow lights, and arrow lights in the case of vehicle lamps) based on the reference point of the designated lamp area. A pixel range (hereinafter referred to as "extraction range") is determined, and each light is extracted within the determined extraction range.
For example, when only one center point P1 is specified, the lamp extraction unit 43 sets a rectangular range of a predetermined length centered on center point P1 as the extraction range, and when three center points P1 to P3 are specified, , a circular range of a predetermined radius centered on each center point P1 to P3 is defined as an extraction range.

Furthermore, when four vertices Pa to Pd are specified, the lamp extraction unit 43 sets a rectangular range surrounded by four vertices Pa to Pd as an extraction range, and when two vertices Pa and Pc are specified, two vertices The extraction range is a rectangular range with Pa and Pd as diagonal lines.
In the extraction range determined as described above, the lamp extractor 43 extracts the coordinate values of pixels corresponding to the signal lamps included in the signal lamp 6 (hereinafter referred to as "light coordinates") based on the feature amount representing the signal lamp. ) and outputs the extracted light coordinates to the light color determination section 44.

If the difference between the lamp coordinates in the previous frame and the lamp coordinates in the current frame is within a predetermined range, the lamp extraction unit 43 considers the lamps to be the same lamp.
Thereby, even if the signal lamp device 6 vibrates due to strong winds, for example, the extracted lamp can be tracked. Therefore, an increase in processing load due to repeated extraction of light coordinates can be suppressed.

The light color determination section 44 determines the light color of the signal lamp device 6 based on the light coordinates input from the lamp device extraction section 43.
Specifically, the light color determination unit 44 determines the lit light and its light color from the RGB values included in the pixel data corresponding to the light coordinates. The light color determining section 44 outputs the determined current light color to the light color checking section 45.

The sensor information input unit 47 is an input unit that receives sensor information output from a traffic-related external sensor. The external sensor is, for example, a millimeter-wave or sub-millimeter-wave radar sensor installed near the intersection J, or a LiDAR (Light Detection And Ranging) sensor.
Further, the external sensor may be a radio device that transfers probe information received from the vehicle 7 to the edge computer 3. Note that the sensor information input unit 47 is an optional functional unit of the edge computer 3 and can be omitted.

The behavior determination unit 48 determines the behavior of vehicles 7, pedestrians 8, etc. on the road based on at least one of the image data input from the data input unit 41 and the sensor information input from the sensor information input unit 47. Determine behavior.
Specifically, the behavior determining unit 48 determines whether the vehicle 7 is stopped or passing near the stop line of the intersection J, and outputs the determination result to the light color checking unit 45.

The light color checking unit 45 performs a predetermined test on the current light color (the current light color of the signal lamp unit 6) determined by the light color determining unit 44, and then sends the light color that has cleared the test to the information generation unit 46. Output. Note that the processing contents of the light color check section 45 will be described later.
The information generation section 46 generates the above-mentioned signal information S1 using the input current lamp color, and outputs the generated signal information S1 to the communication section 33. Further, when the information generation unit 46 creates an estimated time table from the signal information S1, it stores the created estimated time table in the database 36.

[Processing details of the light color check section]
As shown in FIG. 3, the processing executed by the light color check unit 45 includes a "temporal consistency check" and a "spatial consistency check."
The time consistency check is a process of checking whether or not the operation of the signal lamp device 6 has time consistency. The time consistency check includes a first check and a second check.

The spatial consistency check is a process of checking whether the operation of the signal lamp 6 has spatial consistency with the operation of other signal lamps 6. The spatial consistency check includes the third to sixth checks. The contents of the first to sixth checks will be described below with reference to FIGS. 5 to 10.

(First check (time consistency check): Figure 5)
The first check is a process of checking whether the light color change regarding one signal lamp device 6 corresponds to a predetermined prohibited operation recorded in the prohibited rule database 35.
As described above, the prohibited motion is, for example, a motion that changes from blue to red (without passing through yellow). As shown in FIG. 5, the first check performed by the light color check section 45 includes the following steps S11 to S13 depending on whether the prohibited motion is applicable and whether the spatial consistency check is possible. included.

Step S11: If the light color change regarding one signal lamp 6 does not correspond to a prohibited operation, and one signal lamp 6 is a signal lamp 6 that cannot be checked for spatial consistency with other signal lamps, etc., The current light color inputted from the light color determination section 44 is output to the information generation section 46.

Step S12: If the light color change regarding one signal lamp device 6 corresponds to a prohibited operation, and if one signal lamp device 6 is a signal lamp device 6 for which spatial consistency check with other signal lamp devices etc. cannot be performed, the corresponding signal lamp device 6 The current light color of the signal lamp device 6 is determined to be undefined, and the determination result is output to the information generation section 46.

Step S13: If the light color change regarding one signal lamp 6 does not correspond to a prohibited operation, and one signal lamp 6 is a signal lamp 6 that can be checked for spatial consistency with other signal lamps, etc., The current light color input from the light color determination unit 44 is subjected to a spatial consistency check.

(Second check (time consistency check): Figure 6)
The second check is a process of checking whether the light color change regarding one signal lamp device 6 is simultaneous with the estimated timetable recorded in the timetable database 36. "Simultaneousness" means that two comparison targets change at substantially the same time (the time difference is less than or equal to a threshold value).
As described above, the estimated time table is data defined as a table representing the transition order of light colors for each inflow path for one cycle. As shown in FIG. 6, the second check performed by the light color check unit 45 includes the next step depending on whether or not there is simultaneity with the estimated timetable and whether or not a spatial consistency check is possible. Includes S21 to S23.

Step S21: The time difference between the light color change for one signal light device 6 and the light color change in the estimated time table is less than or equal to a predetermined time threshold (for example, 0.1 seconds), and one signal light device 6 is different from the other signal light device 6. If the signal lamp 6 cannot be checked for spatial consistency with a signal lamp or the like, the current lamp color inputted from the lamp color determination unit 44 is output to the information generation unit 46 .

Step S22: The time difference between the light color change for one signal light device 6 and the light color change in the estimated time table exceeds a predetermined time threshold, and one signal light device 6 is spatially consistent with other signal light devices, etc. If the signal lamp 6 cannot be checked, the current color of the signal lamp 6 is determined to be undefined, and the determination result is output to the information generation section 46.

Step S23: The time difference between the light color change for one signal light device 6 and the light color change in the estimated time table is less than or equal to a predetermined time threshold, and one signal light device 6 is spatially consistent with other signal light devices, etc. If the signal lamp device 6 is capable of performing a consistency check, the current lamp color input from the lamp color determination unit 44 is subjected to the spatial consistency check.

(Third check (spatial consistency check): Figure 7)
The third check is a process of checking the simultaneity of light color changes regarding a pair of signal lamps 6, 6 (hereinafter referred to as "opposing lamps") facing in directions facing each other (for example, northbound direction and southbound direction). be. As shown in FIG. 7, the third check performed by the light color check section 45 includes the following steps S31 to S34 depending on whether there is a temporal difference between the opposing lamps.

Step S31: If the time difference between the light color changes regarding the oncoming light equipment is less than or equal to a predetermined time threshold (for example, 0.1 seconds), information is generated based on the current light color of the oncoming light equipment input from the light color determination unit 44. It is output to section 46.
Step S32: If the time difference of the light color change regarding the oncoming light device exceeds a predetermined time threshold, the signal light device 6 (hereinafter referred to as "accurate light device") with the smaller error from the time of light color change in the estimated time table ) and the signal lamp device 6 with the larger error (hereinafter referred to as "inaccurate lamp device").

Step S33: Output the current light color of the accurate lamp as is to the information generation section 46.
Step S34: The current lighting color of the inaccurate lighting device is output to the information generation unit 46 after correcting the time difference with the current lighting color of the accurate lighting device.

(Fourth check (spatial consistency check): Figure 8)
The fourth check is to record in the prohibition database 35 the light color changes regarding a pair of signal lamps 6, 6 (hereinafter referred to as "crossing lamps") directed in directions that intersect with each other (for example, northbound and eastbound directions). This is a process of checking whether or not a predetermined prohibited relationship is met.
As mentioned above, the prohibitive relationship is, for example, a combination of light color states in which the intersection lights simultaneously turn green. As shown in FIG. 8, the fourth check performed by the light color check unit 45 includes the following steps S41 to S42 depending on whether the intersecting light device falls under the prohibited relationship.

Step S41: If the light color change regarding the crossing light device does not fall under the prohibited relationship, the current light color of the crossing light device inputted from the light color determination section 44 is output to the information generation section 46.
Step S42: If the light color change related to the crossing lamp corresponds to a prohibited change, it is determined which of the crossing lamps is an accurate lamp or an inaccurate lamp based on the time of the lamp color change in the estimated time table. identify.

Step S43: Output the current light color of the accurate lamp as is to the information generation section 46.
Step S44: The current lighting color of the inaccurate lighting device is output to the information generation unit 46 after correcting the time difference with the current lighting color of the accurate lighting device.

(Fifth check (spatial consistency check): Figure 9)
The fifth check is that the light color changes regarding vehicle lamps and pedestrian lamps (hereinafter collectively referred to as "pedestrian and pedestrian lamps") are determined according to a predetermined transition rule ( For example, this is a process of checking whether or not the above-mentioned rules 1 and 2) are violated. As shown in FIG. 9, the fifth check performed by the light color check unit 45 includes the following steps S51 to S54 depending on whether the change in light color of the pedestrian/vehicle light device violates the transition rule. .

Step S51: If the light color change regarding the pedestrian/vehicle light device does not violate the transition rule, the current light color of the pedestrian/vehicle light device inputted from the light color determination section 44 is output to the information generation section 46.
Step S52: If the light color change regarding the pedestrian/vehicle light device violates the transition rule, based on the time of light color change in the estimated time table, which of the pedestrian/vehicle light devices is an accurate light device or an inaccurate one is determined. identify whether

Step S53: Output the current light color of the accurate lamp as is to the information generation section 46.
Step S54: The current lighting color of the inaccurate lighting device is output to the information generation unit 46 after correcting the time difference with the current lighting color of the accurate lighting device.

(Sixth check (spatial consistency check): Figure 10)
The sixth check indicates that the light color change for one signal lamp 6 indicates that the vehicle 7 located upstream of the stop line and closest to the stop line (hereinafter referred to as "the leading vehicle") in the direction in which the signal lamp 6 gives the right of way This is a process to check whether the behavior matches the behavior of
In the sixth check, for example, if the leading vehicle is stopped even though it has the right of way, or if the leading vehicle is running even though it does not have the right of way, it is determined that there is a mismatch. As shown in FIG. 10, the sixth check executed by the light color check section 45 includes the following steps S61 to S62 depending on whether the behavior matches the behavior of the leading vehicle.

Step S61: If the light color change of one signal light device 6 matches the behavior of the leading vehicle inputted from the behavior determination section 48, the current light color of the signal light device 6 inputted from the light color determination section 44 is information. It is output to the generation unit 46.

Step S62: If the light color change of one signal lamp device 6 does not match the behavior of the leading vehicle input from the behavior determination unit 48, at least one of the first to fifth checks described above is executed again. Then, perform processing according to the results of the re-executed check.

In the sixth check, the determination of whether the leading vehicle has stopped or passed, which is performed by the behavior determination unit 48, is not limited to the determination based on the image data of the roadside camera 2. Specifically, when sensor information from an external sensor is acquired from the sensor information input section 47, the behavior determination section 48 may determine whether the leading vehicle has stopped or passed based on the sensor information.

[Data structure of communication packet]
FIG. 11 is an explanatory diagram showing an example of the data structure of a communication packet for signal information S1.
As shown in FIG. 11, a communication packet includes a header section, an actual data section, and a frame check sequence (FCS).

In the header section, source and destination address information and the type of information to be stored in the actual data section are written. The actual data section stores information of the type declared in the header section (signal information S1 in this embodiment).
In the communication packet of this embodiment, a storage area for additional information related to the generation method of the signal information S1 is defined in the header section. The additional information storage area includes, for example, an information source code storage area F1 and a correction code storage area F2.

The information source code is a code representing the type of information source of the image data used in the generation process of the signal information S1, as a type of generation method of the signal information S1. For example, when the information source code = 0, it means "roadside camera", and when the information source code = 1, it means "vehicle camera".
Since the edge computer 3 generates the signal information S1 based on the image data of the roadside camera 2, "0" is written in the storage area F1. Therefore, the vehicle 7 or pedestrian 8 that has received the communication packet can sense that the signal information S1 is information based on the image data of the roadside camera 2.

The information source code is not limited to the above, and may be, for example, a code representing what kind of data is used to generate the signal information S1. In that case, the information source code may be defined as follows.
Information source code = 0: Signal information generated from roadside camera image data
Information source code = 1: Signal information generated from image data of an in-vehicle camera installed in one vehicle Information source code = 2: Individual information generated from image data of in-vehicle cameras installed in multiple vehicles Signal information generated by the edge computer through comprehensive judgment based on

Information source code = 3: A signal generated by an edge computer making a comprehensive judgment based on individual information generated from image data of in-vehicle cameras mounted on multiple vehicles and information generated from image data of roadside cameras. information
Information source code = 4: Signal information based on the output from an external traffic signal controller Information source code = 5: Signal information based on the detection result of the lamp line current of the signal by the CT sensor

The application of the information source code may be expanded to the case of a signal information distribution method that does not rely on information generation based on image data, as in the example of information source codes = 4 and 5.
For example, when the information source code = 4, it means "signal information output from a traffic signal controller", and when the information source code = 5, it means a lamp device for current-driving a signal lamp device in a traffic signal controller. It means "signal information detected by a CT sensor" that detects the lighting state of each light color by grasping the line. "Signal information output from a traffic signal controller" is, for example, a timetable held by a traffic signal controller, but it may also be information in which predetermined information has been deleted, updated, or added to the timetable. good. "Signal information detected by the CT sensor" may be, for example, the current value of the lamp line by the CT sensor itself, but it may also be information obtained by adding, deleting, updating, or adding predetermined information to the current value. There may be.

As a result, the edge computer transmits the signal information S1 to the vehicle 7 or pedestrian via the communication unit 33 based on the current light color information (external input system not shown) acquired from an external source such as a traffic signal controller or a CT sensor. 8, the receiver can also detect the source of the signal information.

The correction code is a code representing whether or not the signal information S1 is to be corrected and the type of correction, as a type of generation method of the signal information S1. For example, if the correction code = 0, it means "no correction", if the correction code = 1, it means "correction by the third check", and if the correction code = 2, it means "correction by the fourth check". ", and when correction code = 3, it means "correction by fifth check".
Therefore, the vehicle 7 that has received the communication packet can determine whether the signal information S1 is information that involves correction of lamp color determination, and if it involves correction, the type of correction.

In the communication packet shown in FIG. 11, the storage areas F1 and F2 may be defined as the actual data portion. Further, the information source code and the correction code may be defined by identification values respectively written in a plurality of storage areas.

[Other variations]
The embodiments described above are illustrative in all respects and are not restrictive. The scope of rights of the present invention includes all modifications within the scope of equivalents to the configurations described in the claims.
For example, in the above embodiment, the roadside camera 2 and the edge computer (information processing device) 3 are separate devices, but an integrated information processing device in which both devices are housed in one housing may also be used. .

1 Information provision system 2 Roadside camera 3 Edge computer (information processing device)
4 Terminal device (vehicle terminal)
5 Terminal device (user terminal)
6 Signal lamp 7 Vehicle 8 Pedestrian 9 Wireless base station 10 Public communication network 31 Control unit 32 Storage unit 33 Communication unit (acquisition unit, distribution unit)
34 Synchronization processing unit 35 Inhibition database 36 Time table database 37 Transition rule database 38 Computer program 41 Data input unit 42 Area setting unit 43 Lamp extraction unit 44 Light color determination unit 45 Light color check unit 46 Information generation unit 47 Sensor information input unit 48 Behavior judgment unit J Intersection S1 Signal information P1, P2, P3 Center point Pa, Pb, Pc, Pd Vertex

Claims (16)

1. a communication unit that receives image data of a traffic light device at an intersection included in the scenery;
   An information processing device comprising: a control unit that generates signal information representing a light color state of the signal lamp device based on the image data, and causes the signal information to be transmitted to the communication unit,
   The image data is
   An information processing device that is image data generated by a roadside camera installed so that the angle of view includes the signal light device.
2. The communication department includes:
   capable of transmitting a communication packet including the signal information;
   The communication packet is
   The information processing device according to claim 1, further comprising additional information related to a generation method of the signal information.
3. The control unit includes:
   2. An extraction range of the signal lamp, which is a pixel range including a reference point set in the image data by a user, is determined, and a current light color of the signal lamp is determined based on pixel data within the extraction range. The information processing device described in .
4. The control unit includes:
   The information processing device according to claim 1 or 2, wherein the information processing device executes a time consistency check that checks the temporal consistency of the operation of the signal lamp device.
5. The time consistency check is
   5. The information processing apparatus according to claim 4, further comprising a first check for checking whether a change in light color of the signal lamp device corresponds to a prohibited operation.
6. The time consistency check is
   5. The information processing apparatus according to claim 4, further comprising a second check for checking whether or not the light color change of the signal lamp device is coincident with a pre-calculated estimated time table.
7. The control unit includes:
   The information processing device according to claim 1 or 2, wherein the information processing device executes a spatial consistency check for checking spatial consistency of operation of the signal lamp device.
8. The spatial consistency check is
   8. The information processing apparatus according to claim 7, further comprising a third check for checking the simultaneity of the light color changes of the pair of signal lamps facing each other.
9. The spatial consistency check is
   8. The information processing apparatus according to claim 7, further comprising a fourth check for checking whether a change in light color of a pair of signal lamps oriented in mutually intersecting directions corresponds to a prohibited relationship.
10. The signal lamp device is
    Including vehicle lighting equipment and pedestrian lighting equipment,
    The spatial consistency check is
    8. The information processing apparatus according to claim 7, further comprising a fifth check for checking whether a relationship between a change in light color of the vehicle lamp and a change in light color of the pedestrian lamp violates a transition rule.
11. The spatial consistency check is
    A sixth check to check whether the light color change of the signal light device matches the behavior of a vehicle located upstream of the stop line and closest to the stop line in the direction in which the signal light device gives the right of way. The information processing device according to claim 7, comprising:
12. The communication department includes:
    capable of transmitting a communication packet including the signal information;
    The communication packet is
    The information processing apparatus according to claim 1 or 2, further comprising identification information indicating whether or not the signal information is to be corrected and the type of the correction.
13. The control unit includes:
    If the time consistency check determines that there is no consistency, and the spatial consistency check that checks the spatial consistency of the operation of the signal lamp is impossible, the light color state of the signal lamp is determined to be undefined. The information processing device according to claim 4.
14. a communication unit that receives data indicating a change in light color of a traffic light device at an intersection;
    An information processing device comprising: a control unit that generates signal information representing a light color state of the signal lamp device based on the data and causes the communication unit to transmit the signal information,
    The communication unit is capable of transmitting a communication packet including the signal information,
    The communication packet includes additional information related to a generation method of the signal information.
15. a communication unit for receiving image data included in the scenery of the traffic light device at the intersection; and
    A computer program that causes a computer to function as an information processing device, comprising: a control unit that generates signal information representing a light color state of the signal lamp device based on the image data and transmits the signal information to the communication unit. hand,
    The image data is
    A computer program that is image data generated by a roadside camera installed so that the angle of view includes the signal lamp device.
16. As a step executed by the information processing device,
    a step in which a traffic light device at an intersection receives image data included in a landscape;
    generating signal information representing a light color state of the signal lamp device based on the image data;
    An information processing method comprising the step of transmitting the signal information to the outside,
    The image data is
    An information processing method, wherein the image data is image data generated by a roadside camera installed so that the angle of view includes the signal light device.

Images