JP2020023285A - Monitoring device - Google Patents

Abstract

To enable detection of abnormality of a railway facility installed on the ground.SOLUTION: The monitoring device includes: a storage unit storing a plurality of images including images of a railway facility installed on the ground, which are photographed in advance by an imaging unit installed to face a front part of a railway vehicle, in association with positional information of the railway vehicle as a plurality of reference images; a monitoring image acquisition unit for acquiring images photographed by the imaging unit as monitoring images; a positional information acquisition unit for acquiring positional information of a present position of the railway vehicle; a reference image selection unit for selecting a reference image corresponding to the positional information acquired by the positional information acquisition unit from the plurality of reference images stored in the storage unit; a difference area detection unit for detecting a difference area that is an area in which a difference in luminance between the monitoring image and the reference image selected by the reference image selection unit is a predetermined value or more; and a detection image generation unit for generating a detection image by embedding the difference area into the monitoring image to be displayed.SELECTED DRAWING: Figure 2

Description

  Embodiments of the present invention relate to a monitoring device.

  2. Description of the Related Art Conventionally, a railroad administrator has to promptly know any abnormality in railway equipment installed on the ground. For this purpose, for example, there is a method in which an image including railway equipment is captured and stored by a camera installed forward in a railway vehicle, and an observer looks at the image to check for an abnormality in the railway equipment.

JP 2001-23091 A

  However, in the above-mentioned method, there is a problem that it takes time and effort since the observer checks the abnormality of the railway equipment by looking at the image.

  Therefore, an object of the present embodiment is to provide a monitoring device capable of detecting an abnormality of railway equipment installed on the ground.

  The monitoring device of the embodiment is, in advance, a plurality of images including the railway equipment installed on the ground, which is captured by the imaging unit installed forward in the railway vehicle, in association with the position information of the railway vehicle A storage unit that stores a plurality of reference images, an image captured by the imaging unit, a monitoring image obtaining unit that obtains a monitoring image, and a position information obtaining unit that obtains position information of the current position of the railway vehicle. A reference image selection unit that selects a reference image corresponding to the position information acquired by the position information acquisition unit from the plurality of reference images stored in the storage unit, and the monitoring image and the reference image selection unit A difference area detecting unit that detects a difference area in which the difference in luminance between the reference images selected by the method is equal to or more than a predetermined value, and enables the difference area to be displayed on the monitoring image. And a detection image generation unit for generating a detection image crowded because.

FIG. 1 is a diagram illustrating a schematic configuration of the railway vehicle according to the first embodiment. FIG. 2 is a block diagram illustrating a functional configuration of the monitoring device according to the first embodiment. FIG. 3 is a flowchart illustrating a process performed by the monitoring device according to the first embodiment. FIG. 4 is an explanatory diagram of the operation of the difference area detection unit in the first embodiment. FIG. 5 is a block diagram illustrating a functional configuration of the monitoring device according to the second embodiment. FIG. 6 is a diagram illustrating the operation of the reference image selection unit according to the second embodiment. FIG. 7 is a block diagram illustrating a functional configuration of the monitoring device according to the third embodiment. FIG. 8 is a block diagram illustrating a functional configuration of the monitoring device according to the fourth embodiment. FIG. 9 is a flowchart illustrating a process performed by the monitoring device according to the fourth embodiment.

  Hereinafter, the monitoring devices according to the first to fourth embodiments will be described with reference to the accompanying drawings. In the description of the second embodiment and thereafter, the description of the same items as those of the embodiments described above will be omitted as appropriate.

(1st Embodiment)
FIG. 1 is a diagram illustrating a schematic configuration of the railway vehicle RV of the first embodiment. As illustrated in FIG. 1, the railway vehicle RV includes a camera 10 (an imaging unit), a monitoring device 20, a position measurement device 30, and a display device 40. The camera 10 is installed in the traveling direction (forward) of the railway vehicle RV, and captures and stores an image including railway equipment installed on the ground when the railway vehicle RV runs on the track R. The monitoring device 20 is a computer device capable of detecting an abnormality in the railway equipment (details will be described later). The type of the camera 10 is not limited, but is preferably a high-resolution camera.

  The position measuring device 30 is a unit that measures the current position of the railway vehicle RV and transmits the position information to the monitoring device 20. The position measurement device 30 is, for example, a GNSS (Global Navigation Satellite System) device. The GNSS device receives a radio wave from a plurality of artificial satellites via an antenna and measures the current position of the railway vehicle RV. Note that, for example, when the railway vehicle RV is in a tunnel, the GNSS device cannot receive radio waves from artificial satellites. Therefore, a device that measures the current position of the railway vehicle RV based on the output of a speed generator (tach generator) provided on the axle of the railway vehicle RV may be used as the position measurement device 30. Further, as the position measuring device 30, an apparatus that detects the acceleration in the traveling direction of the railway vehicle RV using an acceleration sensor or a gyro sensor, calculates the position by integrating the acceleration two-dimensionally, and measures the current position of the railway vehicle RV. May be used together or substituted. The display device 40 is a unit for displaying various information, and is, for example, an LCD (Liquid Crystal Display).

  FIG. 2 is a block diagram illustrating a functional configuration of the monitoring device 20 according to the first embodiment. As illustrated in FIG. 2, the monitoring device 20 includes a reference image storage unit 201, a detected image storage unit 202, a position information acquisition unit 203, a monitoring image acquisition unit 204, an image conversion unit 205, a reference image selection unit 206, a difference area detection The storage unit 207 includes a unit 207, a detected image generation unit 208, a display control unit 209, and a storage unit 220. Part or all of the units 203 to 209 are realized by a processor such as a CPU (Central Processing Unit) of the railway vehicle RV executing software stored in the storage unit 220. Further, some or all of the units 203 to 209 may be realized by hardware which is a circuit board such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), and an FPGA (Field Programmable Gate Array). Good.

  The storage unit 220 is realized by, for example, a random access memory (RAM), a read only memory (ROM), a flash memory, or the like. The reference image storage unit 201 and the detected image storage unit 202 are realized by, for example, an HDD (Hard Disk Drive), a flash memory, or the like.

  The storage unit 220 stores various information such as a program executed by a processor of the railway vehicle RV.

  The reference image storage unit 201 previously stores a plurality of images including the railway equipment captured by the camera 10 when the railroad vehicle RV is running as a plurality of reference images in association with the position information and the time information of the railroad vehicle RV. . The reference image is a reference image used for comparison with the monitoring image. 1 shows that reference images 1, 2,..., N are stored in the reference image storage unit 201. The position information is measured by the position measuring device 30. The position information is used for matching between the monitoring image and the reference image (details will be described later).

  The time information is, for example, information indicating the time of the day. The time information is used as auxiliary information when comparing the monitoring image with the reference image. For example, if the monitoring image is a daytime image and the reference image is a nighttime image, the difference in luminance is large even if the shooting target is the same, but the difference in the shooting time zone of both images can be grasped by using time information. And can respond. Further, in addition to the position information and the time information, information such as the weather at the time of shooting (clear, cloudy, rain, snow, etc.) may be associated with the reference image. Note that essential information associated with the reference image is position information, and information such as time information and weather is not essential.

  The detected image storage unit 202 stores the detected image generated by the detected image generation unit 208. It is preferable that the monitoring device 20 be configured so that the detection image stored in the detection image storage unit 202 can be transferred to an external device. Then, the detected image can be viewed on the external device.

  The position information acquisition unit 203 acquires, from the position measurement device 30, position information and time information of the current position of the running railway vehicle RV.

  The monitoring image obtaining unit 204 obtains, as a monitoring image, an image captured by the camera 10 when the railway vehicle RV runs.

  The image conversion unit 205 converts the monitoring image into the same format as the reference image (image conversion). Note that if the format of the monitoring image is originally the same as the format of the reference image, conversion (image conversion) by the image conversion unit 205 is unnecessary.

  The reference image selection unit 206 selects a reference image corresponding to the position information acquired by the position information acquisition unit 203 from the plurality of reference images stored in the reference image storage unit 201 (for example, a reference image whose position information is closest to each other). ). Note that the reference image selection unit 206 may also use time information when selecting a reference image.

  The difference area detection unit 207 detects a difference area in which the difference in luminance between the monitoring image and the reference image selected by the reference image selection unit 206 is equal to or more than a predetermined value. Here, FIG. 4 is an explanatory diagram of the operation of the difference area detection unit 207 in the first embodiment. The difference area detection unit 207 performs the following processes (1) to (6) on the monitoring image (FIG. 4A) and the reference image (FIG. 4B).

(1) The monitoring image and the reference image are divided into the same lattice. (2) The average luminance value of each lattice is calculated for each of the monitoring image and the reference image. (3) The average luminance value of the monitoring image and the reference are calculated for each lattice. Calculate the difference between the average brightness values of the images. (4) Determine whether the difference between the average brightness values is equal to or greater than a predetermined value (threshold). (6) Output the area information (for example, the coordinates of the upper left corner, the vertical width, and the horizontal width of the grid in the case of a rectangle) of the detected difference area (grid or a combination of a plurality of grids).

  In the difference image shown in FIG. 4C, a portion having no difference is black. That is, the grid portion that is not black is the difference area. For example, if a foreign matter such as a plastic bag adheres to a railway facility (for example, an overhead line) in the monitoring image and the foreign matter does not exist in that part in the reference image, the difference in the average luminance value becomes a predetermined value or more and the difference area May be detected as In addition, as the foreign substance, for example, a tree or the like which is tilted or fallen can be considered.

  Returning to FIG. 2, the detected image generation unit 208 generates a detection image by embedding the difference area detected by the difference area detection unit 207 in the monitor image after the image conversion so that it can be displayed, and stores the detection image in the detection image storage unit 202. Remember. When storing the detected image in the detected image storage unit 202, the detected image generation unit 208 also stores the position information and the time information in association with each other. The format of the detected image is arbitrary.

  The display control unit 209 causes the display device 40 to display the detected image stored in the detected image storage unit 202 based on a user operation or the like. As a result, the detection image in which the difference area is clearly displayed is displayed on the display device 40, and the user can easily recognize the difference area. Then, the user can easily recognize, for example, that a foreign object is attached to the railway facility, that a tree is approaching or falls down, and the like.

  FIG. 3 is a flowchart illustrating a process performed by the monitoring device 20 according to the first embodiment. In step S <b> 1, the monitoring image acquisition unit 204 acquires an image captured by the camera 10 during traveling of the railway vehicle RV as a monitoring image, and the position information acquisition unit 203 transmits the position of the current position of the railway vehicle RV from the position measurement device 30. Get information and time information.

  Next, in step S2, the reference image selection unit 206 selects a reference image based on the position information and the time information. Specifically, the reference image selection unit 206 selects a reference image corresponding to the position information and the time information acquired in step S1, from a plurality of reference images stored in the reference image storage unit 201.

  Next, in step S3, the image conversion unit 205 converts the monitoring image into the same format as the reference image.

  Next, in step S4, the difference area detection unit 207 calculates an average luminance value of the monitoring image and the reference image in grid units (the number of grids N).

  Next, the difference area detection unit 207 performs the processing of steps S5 to S7 for all the lattices. In step S5, the difference area detection unit 207 calculates a difference between the average luminance value of the monitoring image and the average luminance value of the reference image for one grid.

  Next, in step S6, it is determined whether or not the difference between the average luminance values of the grid is equal to or larger than a predetermined value. If Yes, the process proceeds to step S7, and if No, the process returns to step S5.

  In step S7, the difference area detection unit 207 stores the grid in the storage unit 220 as a difference area.

  After Steps S5 to S7 are completed, in Step S8, the detected image generation unit 208 determines whether or not a difference area exists. If Yes, the process proceeds to Step S9, and if No, the process ends.

  In step S <b> 9, the detection image generation unit 208 generates a detection image by embedding the difference area in the monitoring image so as to be displayed, and stores the detection image in the detection image storage unit 202. After step S9, the process ends.

  Thus, according to the monitoring device 20 of the first embodiment, it is possible to detect an abnormality of the railway equipment installed on the ground. That is, the detected image storage unit 202 stores the detected image in which the difference area is embedded so as to be displayed. Therefore, the user can easily recognize abnormality of the railway equipment (for example, attachment of foreign matter to the railway equipment, approach of a tree, or the like) based on the difference area by looking at the detection image clearly indicating the difference area.

(2nd Embodiment)
Next, a monitoring device 20 according to a second embodiment will be described. FIG. 5 is a block diagram illustrating a functional configuration of the monitoring device 20 according to the second embodiment. The monitoring device 20 of FIG. 5 is different from the monitoring device 20 of FIG. 2 in that a reference image candidate selection unit 210 is provided and that the operation content of the reference image selection unit 206 is different.

  The reference image candidate selection unit 210 selects a plurality of reference images corresponding to the position information (or time information) acquired by the position information acquisition unit from the plurality of reference images stored in the reference image storage unit 201. . The reference image selection unit 206 also provides a normalized cross-correlation (NCC: -1.0 to 1.0) between the monitoring image and each of the plurality of reference images selected by the reference image candidate selection unit 210. Is calculated, and the reference image having the largest normalized cross-correlation is selected.

  Here, FIG. 6 is an explanatory diagram of the operation of the reference image selection unit 206 in the second embodiment. First, the plurality of reference images C1 to C3 illustrated in FIG. 6A are acquired by the reference image candidate selection unit 210 from the plurality of reference images stored in the reference image storage unit 201 by the position information acquisition unit. This is selected according to the position information.

  Then, for example, the reference image selection unit 206 calculates the normalized cross-correlation of the monitoring image M1 shown in FIG. 6B with each of the plurality of reference images C1 to C3 shown in FIG. Select the reference image with the largest cross-correlation. The same applies to the monitoring images M2 to M5.

  As described above, according to the monitoring device 20 of the second embodiment, the reference image selecting unit 206 has the maximum normalized cross-correlation with the monitoring image among the plurality of reference images selected by the reference image candidate selecting unit 210. By selecting the reference image, the matching between the monitoring image and the reference image can be made more accurate. Further, when calculating the normalized cross-correlation, since the image is calculated as a vector, there is an advantage that it is strong against an error in luminance due to the environment.

(Third embodiment)
Next, a monitoring device 20 according to a third embodiment will be described. FIG. 7 is a block diagram illustrating a functional configuration of the monitoring device according to the third embodiment. The monitoring device 20 of FIG. 7 is different from the monitoring device 20 of FIG. 5 in that a kilometer information storage unit 211 is provided and that the operation content of the reference image candidate selection unit 210 is different. The kilometer is information owned by a railway manager or the like, and is information indicating a position of a station, a railway facility, or the like based on a certain position of the track R (FIG. 1).

  The kilometer information storage unit 211 stores kilometer information. The reference image storage unit 201 stores the reference image in association with the position information, the time information, and the kilometer information of the railway vehicle RV. Further, the reference image candidate selection unit 210 is configured to obtain a position information acquisition unit from the plurality of reference images stored in the reference image storage unit 201 based on the position information, time information, and kilometer information of each reference image. A plurality of reference images corresponding to the position information acquired by 203 are selected.

  As described above, according to the monitoring device 20 of the third embodiment, the reference image candidate selection unit 210 stores the reference image storage unit 201 by using the kilometer information in addition to the position information and the time information. More appropriate plural reference images can be selected from the plural reference images.

(Fourth embodiment)
Next, a monitoring device 20 according to a fourth embodiment will be described. FIG. 8 is a block diagram illustrating a functional configuration of the monitoring device 20 according to the fourth embodiment. The monitoring device 20 of FIG. 8 is different from the monitoring device 20 of FIG. 2 in that a moving image data generation unit 212 is provided and that the detected image storage unit 202 is replaced with a video data storage unit 213. I do.

  The moving image data storage unit 213 encodes the plurality of detected images generated by the detected image generating unit 208 to generate moving image data. For example, H.264 is used for image coding. A standard such as H.264 may be used. The moving image data storage unit 213 stores the moving image data generated by the moving image data storage unit 213.

  FIG. 9 is a flowchart illustrating a process performed by the monitoring device 20 according to the fourth embodiment. Steps S1 to S8 are the same as in FIG. In the case of Yes in step S8, in step S10, the detection image generation unit 208 generates a detection image by embedding the difference area in the monitoring image so as to be displayable. Hereinafter, it is assumed that a plurality of detected images are accumulated at the time of step S11.

  Next, in step S11, the moving image data storage unit 213 generates moving image data by encoding a plurality of detected images, and stores the generated moving image data in the moving image data storage unit 213. After step S11, the process ends.

  As described above, according to the monitoring device 20 of the fourth embodiment, the moving image data storage unit 213 stores the moving image data generated from the plurality of detected images in which the difference area is embedded so as to be displayed. Therefore, the user can easily recognize the abnormality of the railway equipment (for example, the attachment of a foreign object to the railway equipment or the approach of a tree, etc.) based on the difference area by viewing the moving image data indicating the difference area. Since the data is moving image data, by reproducing the data, the user can check how the difference area changes over time. Note that a dedicated player capable of decoding corresponding to the encoding may be used for reproducing the moving image data.

  Although several embodiments of the present invention have been described, these embodiments are provided by way of example and are not intended to limit the scope of the invention. These new embodiments can be implemented in other various forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and their equivalents.

  For example, if the image captured by the camera 10 is color, the brightness and color of the image may change depending on the shooting time zone, weather, and the like. Or the difference in color may be reduced.

  In addition, a GPS (Global Positioning System) device may be used as the position measurement device 30 instead of the GNSS device.

  DESCRIPTION OF SYMBOLS 10 ... Camera, 20 ... Monitoring device, 30 ... Position measuring device, 40 ... Display device, 201 ... Reference image storage part, 202 ... Detection image storage part, 203 ... Position information acquisition part, 204 ... Monitoring image acquisition part, 205 ... Image conversion unit, 206: reference image selection unit, 207: difference area detection unit, 208: detected image generation unit, 209: display control unit, 210: reference image candidate selection unit, 211: kilometer information storage unit, 212: video Data generation unit, 213: moving image data storage unit, R: track, RV: railway vehicle

Claims (4)

In advance, a plurality of images including the railway equipment installed on the ground, which are captured by the imaging unit installed forward in the railway vehicle, are stored as a plurality of reference images in association with the position information of the railway vehicle. Storage unit to
A monitoring image acquisition unit that acquires an image photographed by the imaging unit as a monitoring image,
A position information acquisition unit that acquires position information of a current position of the railway vehicle,
From the plurality of reference images stored in the storage unit, a reference image selection unit that selects a reference image corresponding to the position information acquired by the position information acquisition unit,
A difference region detection unit that detects a difference region in which the difference in luminance between the monitoring image and the reference image selected by the reference image selection unit is a region having a predetermined value or more,
A detection image generation unit that embeds the difference area in the monitoring image so as to be displayable and generates a detection image;
A monitoring device comprising: In advance, a plurality of images including the railway equipment installed on the ground, which are captured by the imaging unit installed forward in the railway vehicle, are stored as a plurality of reference images in association with the position information of the railway vehicle. Storage unit to
A monitoring image acquisition unit that acquires an image photographed by the imaging unit as a monitoring image,
A position information acquisition unit that acquires position information of a current position of the railway vehicle,
From the plurality of reference images stored in the storage unit, a reference image candidate selection unit that selects a plurality of the reference images corresponding to the position information acquired by the position information acquisition unit,
A reference image selection unit that calculates a normalized cross-correlation between the monitoring image and each of the plurality of reference images selected by the reference image candidate selection unit, and selects the reference image having the largest normalized cross-correlation. When,
A difference region detection unit that detects a difference region in which the difference in luminance between the monitoring image and the reference image selected by the reference image selection unit is a region having a predetermined value or more,
A detection image generation unit that embeds the difference area in the monitoring image so as to be displayable and generates a detection image;
A monitoring device comprising: The storage unit stores the reference image in association with position information and kilometer information of the railway vehicle,
The reference image candidate selection unit is configured to obtain, from a plurality of reference images stored in the storage unit, the position acquired by the position information acquisition unit based on the position information and the kilometer information of each of the reference images. The monitoring device according to claim 2, wherein a plurality of reference images corresponding to the information are selected.   The monitoring device according to any one of claims 1 to 3, further comprising a moving image data generation unit that generates moving image data by encoding a plurality of detection images generated by the detection image generation unit.

Images