JP7438738B2 - information processing equipment - Google Patents

Description

Embodiments of the present invention relate to an information processing device.

Some monitoring devices (information processing devices) used for automatic operation of vehicles such as trains detect fires in front. Such monitoring devices detect fires by identifying high-temperature areas from images taken using an infrared camera or the like.

JP 2017-16249 Publication

The above-mentioned monitoring device may detect a distant fire that does not affect the operation of the vehicle. Therefore, in order to solve such problems, an information processing device is provided that can precisely measure the distance to each part photographed with an infrared camera.

According to an embodiment, an information processing device includes a first visible light camera interface, a second visible light camera interface, an infrared camera interface, and a processor. A first visible light camera interface acquires a first visible light image. A second visible light camera interface obtains a second visible light image corresponding to the first visible light image. The infrared camera interface acquires an infrared image corresponding to the first visible light image and having a lower resolution than the first visible light image and the second visible light image. The processor identifies a distance at each coordinate of the first visible light image based on the first visible light image and the second visible light image, and determines the distance at each coordinate of the first visible light image based on the infrared image. Identify the temperature at each coordinate in the visible light image.

FIG. 1 is a block diagram illustrating a configuration example of a transportation system according to an embodiment. FIG. 2 is a block diagram showing a configuration example of a monitoring device according to an embodiment. FIG. 3 is a diagram illustrating an example of the operation of the monitoring device according to the embodiment. FIG. 4 is a diagram illustrating an example of the operation of the monitoring device according to the embodiment. FIG. 5 is a diagram illustrating an example of the operation of the monitoring device according to the embodiment. FIG. 6 is a diagram illustrating an example of the operation of the monitoring device according to the embodiment. FIG. 7 is a diagram illustrating an example of the operation of the monitoring device according to the embodiment. FIG. 8 is a flowchart illustrating an example of the operation of the monitoring device according to the embodiment. FIG. 9 is a flowchart illustrating an example of the operation of the monitoring device according to the embodiment. FIG. 10 is a flowchart illustrating an example of the operation of the monitoring device according to the embodiment.

Embodiments will be described below with reference to the drawings.
The operation system according to the embodiment controls the operation of trains (vehicles). The operation system uses cameras mounted on trains to detect fires that could disrupt train operation. Note that the operation system may be one that controls the operation of a train that runs in automatic operation, or may be one that controls the operation of a train that runs in manned operation.

FIG. 1 shows a configuration example of a transportation system 100 according to an embodiment. As shown in FIG. 1, the operation system 100 includes a vehicle 1, a command center 2, a traffic signal 3, and the like. Here, the command center 2 communicates with the vehicle 1 and the traffic light 3 via a wireless communication path or a wired communication path. Further, the vehicle 1 is communicably connected to a traffic light 3. The operation system 100 also detects a fire 4.

Note that the operation system 100 may include other configurations as needed in addition to the configuration shown in FIG. 1, or may exclude specific configurations.

The vehicle 1 is an electric train that runs on a predetermined track. The vehicle 1 travels based on commands from a command center 2 or the display state of a traffic light 3. Additionally, the vehicle 1 detects a fire 4. When the vehicle 1 detects the fire 4, it notifies the command room 2 that the fire has been detected.

The vehicle 1 may be one that runs automatically. Further, the vehicle 1 may be driven by a manned driver.

The vehicle 1 will be detailed later.

The command center 2 controls the entire operation system 100. The command center 2 controls the operation of the vehicle 1 based on signals from the vehicle 1, track conditions, weather, and the like. For example, the command center 2 transmits an instruction to the vehicle 1 to proceed or stop. The command center 2 may be manned to control the operation of the vehicle 1. Further, the command center 2 may automatically control the operation of the vehicle 1.

The traffic light 3 displays whether or not the vehicle can proceed in accordance with the control from the command center 2. For example, the traffic light 3 lights up a lamp indicating whether or not the vehicle can proceed. Further, the traffic light 3 may display a speed limit. The traffic light 3 is installed at a position where the driver of the vehicle 1 can see it, for example, near a railroad track.

Further, the traffic light 3 transmits signal information indicating whether or not the vehicle can proceed or the speed limit to the vehicle 1 by wire or wirelessly. For example, the traffic light 3 transmits to the vehicle 1 approaching the traffic light 3 signal information indicating whether or not the vehicle can proceed or the speed limit.

Next, the vehicle 1 will be explained.
As shown in FIG. 1, the vehicle 1 includes a monitoring device 10, a communication device 20, a guide device 30, a driving device 40, a signaling device 50, a control device 60, and the like. The monitoring device 10 is connected to a communication device 20, a guidance device 30, a driving device 40, a control device 60, and the like. Further, the control device 60 is connected to the driving device 40, the signal device 50, and the like.

It should be noted that the vehicle 1 may include other configurations as required in addition to the configuration shown in FIG. 1, or a specific configuration may be excluded.

The monitoring device 10 (information processing device) detects obstacles, fires, etc. that may interfere with the operation of the vehicle 1. The monitoring device 10 will be described in detail later.

The communication device 20 is a wired or wireless interface for communicating with the command center 2.

The guidance device 30 outputs guidance such as a voice announcement, a warning sound, or an image to the passengers of the vehicle 1. The guidance device 30 outputs guidance according to the signal from the monitoring device 10. Further, the guidance device 30 outputs guidance under the control of a conductor, a driver, or the like.

The driving device 40 transmits a signal instructing acceleration, deceleration, coasting, etc. to the control device 60 based on signals from the monitoring device 10, the signal device 50, and the like. For example, the operating device 40 is an ATO (Automatic Train Operation) device.

The signaling device 50 receives signal information from the traffic light 3. The signaling device 50 presents the received signal information to a driver or the like. Further, the signaling device 50 transmits a signal instructing stop or deceleration to the control device 60 based on the signal information. For example, if the speed of the vehicle 1 exceeds the speed limit indicated by the signal information, the signaling device 50 transmits a signal instructing deceleration to the control device 60. For example, the signaling device 50 is an ATC (Automatic Train Control) device or a CBTC (Communication-Based Train Control) device.

The control device 60 accelerates, decelerates, or stops the vehicle 1 based on signals from the monitoring device 10, the driving device 40, the signal device 50, and the like. The control device 60 controls a drive device, a brake device, etc., and accelerates, decelerates, or stops the vehicle 1.

Next, the monitoring device 10 will be explained.
FIG. 2 shows an example of the configuration of the monitoring device 10. As shown in FIG. 2, the monitoring device 10 includes a processor 11, a memory 12, a communication section 13, an operation section 14, a display section 15, a first visible light camera interface 16, a first infrared camera interface 17, a second A visible light camera interface 18, a second infrared camera interface 19, a first visible light camera 101, a first infrared camera 102, a second visible light camera 103, a second infrared camera 104, etc. are provided. Processor 11, memory 12, communication unit 13, operation unit 14, display unit 15, first visible light camera interface 16, first infrared camera interface 17, second visible light camera interface 18, and second infrared camera The interface 19 and are communicably connected to each other. The first visible light camera interface 16 and the first visible light camera 101 are communicably connected to each other. The first infrared camera interface 17 and the first infrared camera 102 are communicably connected to each other. The second visible light camera interface 18 and the second visible light camera 103 are communicably connected to each other. The second infrared camera interface 19 and the second infrared camera 104 are communicably connected to each other.

Note that the monitoring device 10 may include other configurations as needed other than the configuration shown in FIG. 2, or may exclude a specific configuration.

The processor 11 has a function of controlling the overall operation of the monitoring device 10. The processor 11 may include an internal cache, various interfaces, and the like. The processor 11 implements various processes by executing programs stored in advance in the internal memory or the memory 12.

Note that some of the various functions realized by the processor 11 executing programs may be realized by a hardware circuit. In this case, processor 11 controls the functions performed by the hardware circuits.

Memory 12 stores various data. For example, memory 12 functions as ROM, RAM, and NVM.
For example, the memory 12 stores control programs, control data, and the like. The control program and control data are installed in advance according to the specifications of the monitoring device 10. For example, the control program is a program that supports functions realized by the monitoring device 10.

The memory 12 also temporarily stores data being processed by the processor 11. The memory 12 may also store data necessary for executing the application program, results of executing the application program, and the like.

The communication unit 13 is an interface for transmitting and receiving data with other devices. For example, the communication unit 13 is connected to a communication device 20, a guide device 30, a driving device 40, a control device 60, and the like.

The operation unit 14 receives various operation inputs from an operator, and transmits a signal indicating the input operation to the processor 11. The operation unit 14 may be composed of a touch panel.

The display unit 15 displays image data from the processor 11. For example, the display unit 15 is composed of a liquid crystal monitor. When the operation section 14 is composed of a touch panel, the display section 15 may be formed integrally with the operation section 14.

The first visible light camera interface 16 is an interface for transmitting and receiving data to and from the first visible light camera 101. For example, the first visible light camera interface 16 is connected to the first visible light camera 101 by wire. The first visible light camera interface 16 transmits a signal for photographing an image to the first visible light camera 101. Further, the first visible light camera interface 16 transmits the captured image from the first visible light camera 101 to the processor 11. Further, the first visible light camera interface 16 may supply power to the first visible light camera 101.

The first infrared camera interface 17 is an interface for transmitting and receiving data to and from the first infrared camera 102. The configuration of the first infrared camera interface 17 is the same as the configuration of the first visible light camera interface 16, so a description thereof will be omitted.

The second visible light camera interface 18 is an interface for transmitting and receiving data to and from the second visible light camera 103. The configuration of the second visible light camera interface 18 is the same as the configuration of the first visible light camera interface 16, so a description thereof will be omitted.

The second infrared camera interface 19 is an interface for transmitting and receiving data to and from the second infrared camera 104. The configuration of the second infrared camera interface 19 is similar to the configuration of the first visible light camera interface 16, so the explanation will be omitted.

The first visible light camera 101 is a camera that photographs the direction in which the vehicle 1 is traveling. For example, the first visible light camera 101 is installed at the front of the vehicle 1 facing the direction of travel. The first visible light camera 101 captures an image (first visible light image) using visible light. The first visible light camera 101 transmits the captured first visible light image to the processor 11 through the first visible light camera interface 16 . For example, the first visible light camera 101 is a CCD (Charge Coupled Device) camera or a CMOS (Complementary Metal Oxide Semiconductor Sensor) camera.

The first infrared camera 102 is a camera that photographs the direction in which the vehicle 1 is traveling. For example, the first infrared camera 102 is installed near the first visible light camera 101 at the front of the vehicle 1 facing the direction of travel. The first infrared camera 102 takes an image (first infrared image) using infrared light. The first infrared camera 102 transmits the captured first infrared image to the processor 11 through the first infrared camera interface 17 .

The resolution of the first infrared image is lower than the resolution of the first visible light image. That is, the number of pixels of the first infrared image is smaller than the number of pixels of the first visible light image.

The second visible light camera 103 is a camera that photographs the traveling direction of the vehicle 1 from a position different from the position of the first visible light camera 101. For example, the second visible light camera 103 is installed at a different position from the first visible light camera 101, facing the front of the vehicle 1 in the direction of travel. The second visible light camera 103 transmits the captured image (second visible light image) to the processor 11 through the second visible light camera interface 18 . The configuration of the second visible light camera 103 is the same as the configuration of the first visible light camera 101, so a description thereof will be omitted.

The second infrared camera 104 is a camera that photographs the direction in which the vehicle 1 is traveling. For example, the second infrared camera 104 is installed near the second visible light camera 103 at the front of the vehicle 1 facing the direction of travel. The second infrared camera 104 takes an image (second infrared image) using infrared light. The second infrared camera 104 transmits the captured image to the processor 11 through the second infrared camera interface 19 . The configuration of the second infrared camera 104 is the same as the configuration of the first infrared camera 102, so a description thereof will be omitted.

The monitoring device 10 may be composed of a personal computer, a programmable controller, or the like. Further, the monitoring device 10 may be configured from an ASIC (Application Specific Integrated Circuit). Furthermore, the monitoring device 10 may be configured from an FPGA (Field Programmable Gate Array). The configuration of the monitoring device 10 is not limited to a specific configuration.

Next, the functions realized by the monitoring device 10 will be explained. The functions realized by the monitoring device 10 are realized by the processor 11 executing a program stored in the memory 12 or the like.

First, the processor 11 has a function of acquiring a first visible light image, a first infrared image, a second visible light image, and a second infrared image.

For example, at predetermined intervals (e.g., 0.1 seconds), the processor 11 selects the first visible light camera 101, the first infrared camera 102, the second visible light camera 103, and the second infrared camera 104, respectively. A first visible light image, a first infrared image, a second visible light image, and a second infrared image are acquired. That is, the second visible light image, the first infrared image, and the second infrared image correspond to the first visible light image. For example, the second visible light image, the first infrared image, and the second infrared image are photographed at timings corresponding to the timing at which the first visible light image is photographed.

Furthermore, the processor 11 identifies the temperature at each coordinate of the first visible light image based on the first infrared image. That is, based on the first infrared image, the processor 11 identifies the temperature of the object reflected in the pixels reflected in each coordinate of the first visible light image.

FIG. 3 shows an operation example in which the processor 11 identifies the temperature at each coordinate of the first visible light image based on the first infrared image.

Here, temperature (u, v) indicates the temperature at coordinates (u, v) in the first infrared image. The temperature (u, v) is a temperature calculated by the processor 11 based on the intensity or frequency of infrared rays detected at the pixel at the coordinates (u, v).

The processor 11 obtains a coordinate conversion table MAP (X, Y) that converts the coordinates (x, y) of the first visible light image into the coordinates (u, v) of the first infrared image. The coordinate conversion table MAP (x, y) is a function that returns the coordinates (u, v) of the first infrared image corresponding to the coordinates (x, y) of the first visible light image. That is, when the coordinates of the first visible light image are input, the coordinate conversion table MAP outputs the coordinates of the same object as the object at the coordinates in the first infrared image.

The coordinate conversion table MAP is generated based on the positional relationship between the first visible light camera 101 and the first infrared camera 102, the angle of view of both cameras, and the like. The coordinate conversion table MAP may be stored in advance in the memory 12 or the like.

As FIG. 3 shows, the dots in the first infrared image are larger than the dots in the first visible light image. That is, one dot in the first infrared image corresponds to multiple dots in the first visible light image.

The processor 11 inputs the predetermined coordinates of the first visible light image into the coordinate conversion table MAP, thereby acquiring the coordinates of the first infrared image corresponding to the predetermined coordinates. Upon acquiring the coordinates of the first infrared image corresponding to the predetermined coordinates, the processor 11 acquires the temperature at the acquired coordinates based on the first infrared image. Upon acquiring the temperature, the processor 11 specifies the temperature as the temperature at the predetermined coordinates in the first visible light image.
Processor 11 similarly identifies the temperature at each coordinate of the first visible light image. The processor 11 obtains temperatures (x, y) indicating the temperature at each coordinate of the first visible light coordinates.

Further, the processor 11 specifies the temperature at each coordinate of the second visible light image based on the second infrared image, but the operation of specifying the temperature at each pixel of the second visible light image is performed by the first infrared image. Since this operation is the same as the operation of specifying the temperature in each visible light image, the explanation will be omitted.

Furthermore, the processor 11 calculates the distance at each coordinate of the first visible light image (or the second visible light image).

That is, the processor 11 calculates the distance from the reference point or reference plane to the object reflected in the pixel at each coordinate of the first visible light image.

The processor 11 calculates the distance to the object reflected in the pixel at each coordinate based on the parallax between the first visible light image and the second visible light image. The processor 11 obtains a distance (x, y) indicating the distance at each coordinate of the first visible light coordinates.

Further, the processor 11 sets a construction limit in the forward direction of the vehicle 1. The construction limit is an area where there is a risk of collision with the vehicle 1 if an obstacle exists.

4 and 5 show an example of the operation in which the processor 11 sets building limits.

The processor 11 identifies the route along which the vehicle 1 will travel. For example, the processor 11 identifies the switch 70 that branches the track on which the vehicle 1 runs from the first visible light image (or the second visible light image). When the switch 70 is identified, the processor 11 recognizes the open state of the switch 70 based on the image of the switch 70. Upon recognizing the open state of the switch 70, the processor 11 specifies the route along which the vehicle 1 will travel based on the recognition result.

When the route along which the vehicle 1 is to travel is specified, the processor 11 sets building limits at a plurality of positions that are a predetermined distance away from the vehicle 1 in the direction of travel on the identified route.

In the example shown in FIGS. 4 and 5, the processor 11 sets building limits 80 (80a, 80b, . . . ). The building limits 80 (80a, 80b, . . . ) are building limits set at a distance L (La, Lb, . . . ) away from the front of the vehicle 1 in the traveling direction.

Furthermore, the processor 11 has a function of outputting a notification indicating that a fire has been detected based on the scale of the fire 4 and the distance from the vehicle 1 to the fire 4.

4 and 5 show an example of an operation in which the processor 11 outputs a notification indicating that a fire has been detected.

The processor 11 detects the fire 4 from the first visible light image (or second visible light image) associated with the temperature calculated based on the first infrared image (or second infrared image). Detect. For example, the processor 11 selects an area (in which fire 4 is shown) that exceeds a predetermined threshold (temperature for detecting fire 4 (fire temperature)) from the first visible light image (or second visible light image). A fire area 90) is detected. That is, the processor 11 determines whether there is a coordinate (x, y) where temperature (x, y)>fire temperature. The processor 11 detects the fire 4 if there is a coordinate (x, y) where temperature (x, y)>fire temperature.

When a fire 4 is detected, the processor 11 calculates the scale of the fire 4 in the first visible light image. Here, the processor 11 calculates temperature x area (dot size) at each coordinate where the temperature exceeds a predetermined threshold value, and calculates an evaluation value (scale evaluation value) by adding each value.

After calculating the scale evaluation value, the processor 11 determines whether the scale evaluation value is greater than or equal to a predetermined threshold (fire scale regulation value).

If it is determined that the scale evaluation value is equal to or greater than the prescribed fire scale value, the processor 11 sets the building limit as described above.

Once the building limits are set, the processor 11 calculates the center coordinates of each building limit 80 in the three-dimensional space using a predetermined point as the origin.

After calculating the coordinates of the center of each building limit 80, the processor 11 calculates the distance from the center coordinates of each building limit 80 to the fire 4 (fire area 90) based on the distance (x, y) of the first visible light image. The distance K (Ka, Kb...) is calculated.

For example, the processor 11 calculates the difference between the distance (horizontal distance) from the front of the vehicle 1 to each building limit 80 and the distance (horizontal distance) from the front of the vehicle 1 to the fire area 90 for each building. It is calculated as the distance from the center coordinates of the limit 80 to the fire 4.

After calculating each distance, the processor 11 calculates the degree of risk based on one building limit 80 (eg, building limit 80a). For example, the processor 11 calculates temperature x area (dot size)/distance (for example, distance Ka) at each coordinate of the fire area 90, and adds the calculated values to a degree of risk (for example, degree of risk a). Calculate. For example, the processor 11 may calculate a value obtained by subtracting the scale evaluation value by the distance as the degree of risk.

Note that the processor 11 calculates the distance from the center coordinates of each building limit 80 to each coordinate of the fire area 90 in the three-dimensional space, and calculates the temperature x area (dot size)/distance at each coordinate of the fire area 90. The degree of risk may be calculated by adding the calculated values.

Moreover, in the first visible light image, when the building limit 80 and the fire area 90 at least partially overlap, the processor 11 may calculate the degree of danger of the building limit 80. If the building limit 80 and the fire area 90 do not overlap, the processor 11 may calculate the risk level of the building limit 80 as "0".

The processor 11 calculates each risk level (risk level a, risk level b...) based on each building limit 80 (80a, 80b...).

After calculating each degree of risk, the processor 11 calculates a degree of hindrance indicating the extent to which the operation of the vehicle 1 is hindered based on each degree of risk. For example, the processor 11 calculates a value obtained by adding up each degree of risk as the degree of hindrance. Further, the processor 11 may calculate the degree of hindrance by adding a weighted value to each degree of risk. For example, the processor 11 may give greater weight to the degree of danger of the building limit 80 that is set at a short distance from the vehicle 1. The method by which the processor 11 calculates the degree of difficulty is not limited to a specific method.

After calculating the degree of hindrance, the processor 11 outputs a notification indicating that a fire has been detected based on the degree of hindrance. For example, the processor 11 outputs the notification when the degree of hindrance is equal to or higher than a predetermined threshold (default value for determining hindrance). Note that the processor 11 may output the notification when the degree of difficulty is increasing. The conditions under which the processor 11 outputs the notification are not limited to a specific configuration.

Further, the processor 11 transmits the notification to the command room 2 through the communication device 20. The processor 11 also transmits the information to the guide device 30, the driving device 40, and the control device 60. Further, the processor 11 may transmit the notification to other vehicles. The output destination to which the processor 11 outputs the notification is not limited to a specific configuration.

In the example shown in FIGS. 4 and 5, the processor 11 calculates the scale evaluation value of the fire 4 to be relatively high because the fire area 90 is shown large. However, as shown in FIG. 5, the distances (Ka, Kb, . . . ) from each building limit 80 to the fire 4 are relatively long. Therefore, the processor 11 calculates a low degree of hindrance. As a result, the processor 11 does not output a notification indicating that a fire has been detected.

6 and 7 show other examples of the fire 4. In the example shown in FIGS. 6 and 7, the fire area 90 appears small, so the processor 11 calculates the scale evaluation value of the fire 4 to be relatively small. However, as shown in FIG. 7, the distances (Ka, Kb, . . . ) from each building limit 80 to the fire 4 are relatively short. Therefore, the processor 11 calculates the degree of hindrance to be high. As a result, the processor 11 outputs a notification indicating that a fire has been detected.

Further, when the processor 11 detects an obstacle within each building limit 80 in the first visible light image, the processor 11 may transmit a notification indicating that the obstacle has been detected to the command room 2 via the communication device 20. Further, when the monitoring device 10 detects an obstacle, the monitoring device 10 may transmit a signal to the driving device 40 instructing it to stop.

Next, an example of the operation of the monitoring device 10 will be described.
FIG. 8 is a flowchart for explaining an example of the operation of the monitoring device 10.

First, the processor 11 of the monitoring device 10 initializes the memory, timer, input/output, etc. (S11). After initializing the memory, timer, input/output, etc., the processor 11 specifies the temperature at each coordinate of the first visible light image and the second visible light image (S12).

After specifying the temperature at each coordinate of the first visible light image, the processor 11 detects a fire area from the first visible light image or the second visible light image (S13).

When a fire area is detected from the first visible light image (S14, YES), the processor 11 calculates a scale evaluation value (S15). After calculating the scale evaluation value, the processor 11 determines whether the scale evaluation value is greater than or equal to the prescribed fire scale value (S16).

If it is determined that the scale evaluation value is equal to or greater than the prescribed fire scale value (S16, YES), the processor 11 calculates the distance at each coordinate of the first visible light image (S17). After calculating the distance at each coordinate of the first visible light image, the processor 11 sets a building limit (S18).

After setting the building limits, the processor 11 calculates the degree of hindrance based on the building limits and the distance to the fire area (S19). After calculating the degree of hindrance, the processor 11 determines whether the degree of hindrance is greater than or equal to a prescribed value for determining the hindrance (S20).

If it is determined that the degree of hindrance is equal to or higher than the hindrance determination prescribed value (S20, YES), the processor 11 outputs a notification indicating that a fire has been detected (S21).

If there is no fire area in the first visible light image (S14, NO), or if it is determined that the scale evaluation value is not equal to or greater than the fire scale prescribed value (S16, NO), the degree of hindrance must not be equal to or greater than the hindrance determination prescribed value. If it is determined (S20, NO), or if a notification indicating that a fire has been detected is output (S21), the processor 11 returns to S12. Note that the processor 11 may wait for a predetermined time and then return to S12.

The processor 11 repeats the above operations until it receives input of a predetermined operation through the operation unit 14 or the like.

Next, an operation example (S12) in which the processor 11 specifies the temperature at each coordinate of the first visible light image and the second visible light image will be described.

FIG. 9 is a flowchart for explaining an operation example (S12) in which the processor 11 specifies the temperature at each coordinate of the first visible light image and the second visible light image.

First, the processor 11 acquires a first infrared image using the first infrared camera 102 (S31). After acquiring the first infrared image, the processor 11 acquires a first visible light image using the first visible light camera 101 (S32).

After acquiring the first visible light image, the processor 11 identifies the temperature at a predetermined coordinate in the first visible light image based on the first infrared image (S33). After specifying the temperature at the predetermined coordinates in the first visible light image, the processor 11 determines whether the specification of the temperature at each coordinate in the first visible light image is completed (S34).

If it is determined that the temperature identification of each coordinate in the first visible light image has not been completed (S34, NO), the processor 11 returns to S33.

When determining that the temperature at each coordinate in the first visible light image has been identified (S34, YES), the processor 11 acquires a second infrared image using the second infrared camera 104 (S35). After acquiring the second infrared image, the processor 11 acquires a second visible light image using the second visible light camera 103 (S36).

After acquiring the second visible light image, the processor 11 identifies the temperature at a predetermined coordinate in the second visible light image based on the second infrared image (S37). After specifying the temperature at the predetermined coordinates in the second visible light image, the processor 11 determines whether the specification of the temperature at each coordinate in the second visible light image is completed (S38).

If it is determined that the temperature identification of each coordinate in the second visible light image has not been completed (S38, NO), the processor 11 returns to S37. When determining that the temperature at each coordinate in the second visible light image has been determined (S38, YES), the processor 11 ends the operation.

Next, an example of the operation (S18) in which the processor 11 sets a building limit will be described. FIG. 10 is a flowchart for explaining an operation example (S18) in which the processor 11 sets a building limit.

First, the processor 11 recognizes the opening state of the switch 70 from the first visible light image (or the second visible light image) (S41). Upon recognizing the open state of the switch 70, etc., the processor 11 specifies the route along which the vehicle 1 will travel (S42).

After specifying the route, the processor 11 assigns an initial value to L (S43). After substituting the initial value for L, the processor 11 sets a construction limit at a position L away from the vehicle 1 in the traveling direction (S44).

After setting the building limit at a position L away from the vehicle 1 in the traveling direction, the processor 11 adds a predetermined value (ΔL) to L (S45). After adding the predetermined value (ΔL) to L, the processor 11 determines whether L exceeds a predetermined threshold (S46).

If it is determined that L is less than or equal to the predetermined threshold (S46, NO), the processor 11 returns to S44.
If it is determined that L exceeds the predetermined threshold (S46, YES), the processor 11 ends the operation.

Note that the monitoring device 10 may include one infrared camera. In this case, the processor 11 may identify the temperature at each coordinate of the first visible light image and the second visible light image based on one infrared image. Furthermore, the processor 11 may specify the temperature at each coordinate of the first visible light image or the second visible light image based on one infrared image. Further, the monitoring device 10 does not need to specify the temperature at each coordinate of the second visible light image.

Furthermore, the monitoring device 10 may include one visible light camera. In this case, the processor 11 may measure the distance at each coordinate of the visible light image based on one visible light image.

Further, the monitoring device 10 may measure the distance to each part using a distance sensor or the like. In this case, the monitoring device 10 may measure the temperature of each part whose distance was measured based on the infrared image.

Further, the monitoring device 10 may be installed in a car, an airplane, a ship, or the like. Furthermore, the monitoring device 10 may be installed at a predetermined location. The location where the monitoring device 10 is installed is not limited to a specific configuration.

The monitoring device configured as described above uses two visible light cameras to measure the distance to the object at each coordinate of the visible light image. Furthermore, the monitoring device uses an infrared camera to measure the temperature of the object at each coordinate in the infrared image. The monitoring device identifies the temperature at each coordinate of the visible light image by calibrating the visible light image and the infrared image. As a result, the monitoring device can determine the distance and temperature at each coordinate in the visible light image.

Typically, the resolution of an infrared camera is lower than that of a visible light camera. Therefore, distance measurement using two infrared cameras is less accurate than distance measurement using two visible light cameras.

With the above configuration, the monitoring device can measure the temperature at each coordinate while measuring the distance at each coordinate with high accuracy by using visible light images.

The monitoring device also outputs a notification indicating that a fire has been detected based on the distance to the fire. As a result, when the monitoring device detects a fire that may impede train operation, it can output the notification.

Although several embodiments of the invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included within the scope and gist of the invention, as well as within the scope of the invention described in the claims and its equivalents.

DESCRIPTION OF SYMBOLS 1...Vehicle, 2...Command room, 3...Traffic light, 4...Fire, 10...Monitoring device, 11...Processor, 12...Memory, 13...Communication section, 14...Operation section, 15...Display section, 16...First Visible light camera interface, 17...First infrared camera interface, 18...Second visible light camera interface, 19...Second infrared camera interface, 20...Communication device, 30...Guidance device, 40...Driving device, 50... Signal device, 60... Control device, 70... Switch, 80... Building limit, 80a... Building limit, 90... Fire area, 100... Traffic system, 101... First visible light camera, 102... First infrared camera, 103 ...Second visible light camera, 104...Second infrared camera, L...Distance.

Claims (5)

a first visible light camera interface that acquires a first visible light image;
a second visible light camera interface that acquires a second visible light image corresponding to the first visible light image;
an infrared camera interface that acquires an infrared image corresponding to the first visible light image, the resolution being lower than the resolution of the first visible light image and the second visible light image;
Here, the first visible light image, the second visible light image, and the infrared image are images taken in the traveling direction from a traveling vehicle,
Identifying the distance at each coordinate of the first visible light image based on the first visible light image and the second visible light image,
Identifying the temperature at each coordinate of the first visible light image based on the infrared image ,
Detecting a fire based on the temperature of each coordinate of the first visible light image,
setting a building limit for the vehicle;
outputting a notification indicating that a fire has been detected based on the distance to the fire and the building limit ;
a processor;
An information processing device comprising: The processor identifies the temperature at each coordinate of the first visible light image using a table that converts the coordinates of the first visible light image into the coordinates of the infrared image.
The information processing device according to claim 1. the processor outputs the notification further based on the scale of the fire;
The information processing device according to claim 1 . The processor includes:
Identifying a fire area with a temperature exceeding a predetermined fire temperature from the first visible light image,
outputting the notification based on the temperature of the fire area, the area of the fire area, and the distance between the building limit and the fire;
The information processing device according to claim 3 . a first visible light camera that captures the first visible light image;
a second visible light camera that captures the second visible light image;
an infrared camera that captures the infrared image;
Equipped with
The information processing device according to any one of claims 1 to 4 .