JP6471541B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus.

  Since railroad vehicles used by a large number of passengers require safe operation, those involved in the railcars are making various attempts to operate the railroad vehicles safely. For example, there are a number of attempts to detect passengers' fall quickly, as passengers may fall into a track from a platform, and some of them are actually used at railway vehicle stations. Yes.

As a specific example, a camera system is well known in which a plurality of cameras are installed under the platform ceiling and a plurality of cameras monitor a station premises such as on a platform or a track.
As such a camera system, for example, Patent Document 1 discloses a camera system in which a plurality of cameras are connected in a daisy chain, and monitoring information acquired by each camera is transmitted to a server via another camera. ing.

JP 2012-11989

  By the way, in order to monitor using a camera system, it is necessary to adjust the angle at which the camera is mounted and the angle of view with high accuracy, and a great deal of time is required. In particular, in order to monitor the entire platform, it is necessary to install a large number of cameras, and each camera requires a specific adjustment, which requires a lot of time and impairs passenger convenience. Since installation work needs to be performed at night, technology to reduce the installation work has been demanded.

  The present invention has been made in view of such problems, and an object of the present invention is to provide an image processing apparatus that can shorten the time required for installing a camera for monitoring a platform.

  According to a first aspect of the present invention, there is provided an image processing apparatus for processing a photographed image photographed by a stereo camera installed on an upper part of a platform extending in a first direction, which is photographed by a first photographing unit of the stereo camera. Based on the parallax between the first image and the second image captured by the second imaging unit, the first direction among the X coordinate axis and the Y coordinate axis of the generated distance image and the X coordinate axis and the Y coordinate axis of the generated distance image A counting unit that counts the number of pixels with zero parallax for each coordinate of the first coordinate axis along the second direction orthogonal to the X coordinate axis and the Y coordinate axis based on the counting result for each coordinate of the first coordinate axis An image control device is provided that includes an image control unit that yaw-rotates the distance image with a Z coordinate axis that is orthogonal as a rotation axis.

  The image control further includes a specifying unit that specifies a rail coordinate that is a position in the first coordinate axis of a rail installed along the platform based on the counting result for each coordinate of the first coordinate axis. The unit may determine a rotation angle at which the distance image is yaw-rotated based on the number of pixels with zero parallax in the rail coordinates.

  In addition, based on the counting result for each coordinate of the first coordinate axis, further comprising a specifying unit that specifies a home end coordinate that is a position of the end of the platform in the first coordinate axis, the counting unit, Of the X coordinate axis and the Y coordinate axis of the distance image, the number of pixels with zero parallax is counted for each coordinate of the second coordinate axis along the first direction, and the image control unit counts the result for each coordinate of the second coordinate axis The distance image may be roll-rotated based on the home end coordinates.

  In addition, the specifying unit may determine a sleeper coordinate that is a position on the second coordinate axis of the sleeper that is installed together with a rail extending along the platform, based on a counting result for each coordinate of the second coordinate axis, and a plurality of sleepers. The image control unit may roll-rotate the distance image based on a counting result of the number of pixels with zero parallax in the plurality of identified sleeper coordinates.

  In addition, the counting result of the counting unit in the first distance image generated from the captured image captured by the stereo camera at the first timing, and the captured image captured at the second timing different from the first timing by the stereo camera. It is good also as providing the detection part which detects the change of the installation state of the said stereo camera by comparing with the count result of the said count part in the 2nd distance image produced | generated from.

  Further, the monitoring control unit that performs platform monitoring control based on the distance image that is rotationally controlled by the image control unit, the distribution state of the number of pixels with zero parallax in each coordinate of the coordinate axis that counted the number of pixels with zero parallax Based on this, a monitoring control unit that stops the monitoring control may be further provided.

  ADVANTAGE OF THE INVENTION According to this invention, the time which the installation operation | work of the camera which monitors a platform requires can be shortened.

It is a figure which shows the structural example of the imaging | photography system which concerns on one Embodiment. It is a figure which shows the example of installation of the imaging device in a station platform. It is a figure which shows the example of installation of the imaging device in a station platform. It is a figure which shows an example of the various picked-up images image | photographed with the imaging device. It is a figure which shows the outline | summary of an imaging | photography system. It is a block diagram which shows the function structure of a data management apparatus. It is a figure which shows the operation example in case an image control part carries out the yaw rotation of the distance image. It is a figure which shows the operation example in case an image control part translates a distance image. It is a figure which shows the operation example in case an image control part rolls a distance image. It is a figure for demonstrating the roll rotation method of the distance image by an image control part. It is a figure which shows an example of the monitoring area | region set to a distance image. It is a figure which shows typically the histogram which shows the pixel number of the parallax zero according to a weather condition. It is a flowchart which shows the flow of a process of an imaging | photography system. It is a flowchart which shows the flow of a process of an imaging | photography system. It is a flowchart which shows the flow of a process of an imaging | photography system.

[Overall Configuration of Shooting System 10]
FIG. 1 is a diagram illustrating an overall configuration of a photographing system 10 according to the present embodiment, and FIG. 2 is a diagram illustrating an installation example of a photographing device 101 on a platform P of a station. 2A is a diagram of the station platform P viewed from the traveling direction of the railway vehicle, and FIG. 2B is a diagram of the station platform P viewed from a direction orthogonal to the traveling direction of the railway vehicle.

As shown in FIG. 1, the imaging system 10 includes an imaging device group 100 and a data management device 200 connected to the imaging device group 100 so as to be communicable.
Note that the imaging system 10 may include the hub 300 between the data management device 200 and the imaging device group 100, or may include a plurality of imaging device groups 100 connected to the hub 300. In this case, for example, each imaging device group 100 may be installed in each of a plurality of platforms P at a station. The imaging system 10 may also include a monitor 400 that displays an image captured by the imaging device group 100.

  In the imaging device group 100, a plurality of imaging devices 101 (imaging device 101-1, imaging device 101-2... Imaging device 101-n, where n is a natural number of 3 or more) are connected in a daisy chain along a predetermined direction. Configured. As shown in FIG. 2B, in the present embodiment, the plurality of imaging devices 101 are daisy chain connected along the length direction of the platform P of the station.

  The photographing apparatus 101 is a stereo camera in which a plurality of photographing units are horizontally arranged along the length direction of the platform P of the station. As shown in FIG. 2A, the photographing apparatus 101 is referred to as an end portion of the platform P (hereinafter referred to as “home end Pe”) in the upper part (ceiling) of the platform P of the station so as not to affect the operation of the railway vehicle. ) And set back a predetermined amount inside the platform P from directly above. Further, as illustrated in FIG. 2B, the image capturing apparatuses 101 are installed so that the image capturing ranges S of the respective image capturing apparatuses 101 overlap with the image capturing ranges S of the adjacent image capturing apparatuses 101.

  The photographing apparatus 101 photographs the periphery of the platform P, that is, the platform P and the track R from a bird's-eye view. Here, FIG. 3 shows an example of the first image 51 and the second image 52 photographed by the photographing apparatus 101, and the distance image 53 generated from the first image 51 and the second image 52.

The first image 51 is a photographed image photographed by the first photographing unit of the photographing apparatus 101, and the second image 52 is a photographed image photographed by the second photographing unit of the photographing apparatus 101.
As shown in the first image 51 in FIG. 3, the photographing apparatus 101 photographs the platform P and the track R. In the present embodiment, the track R is a ballast track composed of a rail Ra, a plurality of sleepers Rt, and gravel Rb.

The distance image 53 is an image generated based on the parallax between the first image 51 and the second image 52.
Here, the parallax calculation method is already known, and the parallax is calculated by searching corresponding points corresponding to the reference point of the reference image (first image 51) from the comparison image (second image 52). The Specifically, a pixel block having a luminance pattern similar to the pixel block extracted from the reference image among the pixel blocks in the parallax search area of the comparison image is extracted from the reference image centering on the reference pixel. Is identified by a known method such as SAD (Sum of Absolute Differences) method. Then, the parallax is calculated from the difference between the coordinates of the pixel block of the reference image and the coordinates of the pixel block of the comparison image.

  Since the parallax is calculated for a region where the texture is clear, the parallax is not calculated for an object with a small surface texture such as the platform P, the rail Ra, and the sleeper Rt. Therefore, when the ballast trajectory is imaged by the imaging device 101, as shown in the distance image 53 in FIG. 3, the parallax cannot be calculated for the platform P, the rail Ra, and the sleeper Rt, and the pattern is characterized as gravel Rb. In a region where the texture is clear, the parallax can be calculated.

  In each image, the horizontal direction is the X direction (first direction), and the vertical direction is the Y direction (second direction). Since the plurality of imaging units of the imaging apparatus 101 are horizontally arranged along the length direction of the platform P of the station, the X direction corresponds to the length direction of the platform P. The Y direction corresponds to the width direction of the platform P and the track R. In the following, for convenience of explanation, the upper part of the image is Y coordinate “small” and the lower part of the screen is Y coordinate “large”.

  The first image 51 and the second image 52 photographed by the photographing device 101 and the distance image 53 generated from the first image 51 and the second image 52 are transmitted to the data management device 200 via a predetermined communication line. And stored in the storage unit 202 of the data management device 200. The distance image 53 may be generated by the data management apparatus 200 without being generated by the imaging apparatus 101.

[Outline of Shooting System 10]
Next, an overview of the imaging system 10 according to the present embodiment will be described. In the imaging system 10, the time required for the installation work of the imaging apparatus 101 is shortened by rotating the generated distance image 53. As shown in FIG. 4A, the distance image 53 can be rotated using any of the XYZ axes as a rotation axis. Hereinafter, the rotation about the X axis as a rotation axis is “roll rotation”. The rotation with the Y axis as the rotation axis is called “pitch rotation”, and the rotation with the Z axis as the rotation axis is called “yaw rotation”. In the following description, the distance image before rotation is referred to as “distance image 53”, the distance image after rotation is referred to as “distance image 53A”, and “distance image 53” is used when it is not distinguished before and after rotation.
In the present embodiment, the distance image 53 is rotated with the X axis or the Z axis among the XYZ axes as the rotation axis. However, the distance image 53 may be rotated with the Y axis as the rotation axis.

As shown in FIG. 4B, when the distance image 53 is yaw-rotated around the Z axis, the inclination of the XY axes in the distance image 53 is corrected. Such yaw rotation can be performed using a technique already known as affine transformation.
Here, in the monitoring of the platform P, monitoring that a passenger or a foreign object has fallen on the track R and monitoring that a passenger approaches the home end Pe are required. For these monitoring, for example, a fall monitoring area for monitoring the line fall is set on the line R side at the home end Pe in the distance image 53, and an approach monitoring area for monitoring the approach to the home end Pe is set on the platform P side. This can be done by setting (see FIG. 10 described later).

In this regard, in the distance image 53, when the track R and the platform P extend horizontally in the X direction, although both monitoring areas can be easily set, the track R and the platform P are inclined with respect to the X direction. In this case, the setting work for both monitoring areas becomes complicated.
The yaw rotation is preferably used when the line R or the platform P is inclined with respect to the X direction, and the distance image 53 in which the line R or the platform P is inclined with respect to the X direction is yaw rotated. A distance image 53A in which R and the platform P extend horizontally in the X direction is generated.

  Thereby, in the imaging | photography system 10, it is not necessary to adjust an installation angle strictly at the time of installation of the imaging device 101, and the time which installation work requires can be shortened.

As shown in FIG. 4C, when the distance image 53 is rotated around the X axis, the inclination of the YZ axis, particularly the Z axis, in the distance image 53 is corrected.
Here, the distance image 53 is generated based on the parallax between the first image 51 and the second image 52. If the parallax is known, since the distance from the photographing apparatus 101 to the subject can be measured by the principle of triangulation, the Z coordinate can be obtained from the distance image 53 in addition to the XY coordinates of the subject. The roll rotation of the distance image 53 is already known as coordinate transformation, and can be performed by calculating how the XYZ coordinates of the subject expressed in local coordinates are arranged in the global coordinate system.

  In the monitoring of the platform P, since the photographing apparatus 101 is set back from the end of the platform P, the photographing apparatus 101 looks down at the home end Pe and the track R obliquely. Here, as shown in FIG. 2A, when the passenger is standing at the end of the platform P, the distance image 53 photographed by the photographing apparatus 101 appears as if the upper body of the passenger exists in the track. There is a risk that the fall of the track will be erroneously detected.

The roll rotation is used to correct a state where the home end Pe is looked down obliquely due to setback.
Specifically, in the photographing system 10, as shown in FIG. 4C, the distance image 53 is rotated around the X axis (same as the length direction of the platform P) to generate the distance image 53A. In monitoring the platform P, drop monitoring and approach monitoring are performed with the home end Pe as a boundary. In such a monitoring, it is preferable to monitor the home end Pe from directly above. Therefore, in this embodiment, the distance image 53 is rotated around the home end Pe.

  As a result of the roll rotation of the distance image 53, the inclination of the Z-axis is corrected. As a result, as shown in FIG. 4D, the home end Pe was photographed from directly above the distance image 53 with the home end Pe looked down obliquely. A distance image 53A like that can be obtained. Thereby, even when the photographing apparatus 101 is set back from the home end Pe and installed, the boundary portion between the platform P and the track R can be monitored with high accuracy.

Further, since the monitoring accuracy of the photographing apparatus 101 can be adjusted by the rotation processing after the installation, it is not necessary to precisely adjust the mounting angle and the angle of view when the photographing apparatus 101 is installed. Therefore, in the imaging system 10, it is not necessary to spend an enormous amount of time for installing the imaging apparatus 101, and the time required for the installation operation of the imaging apparatus 101 can be shortened.
The details of the data management apparatus 200 that enables such control will be described below.

[Functional Configuration of Data Management Device 200]
FIG. 5 is a block diagram showing a functional configuration of the data management apparatus 200. The data management apparatus 200 includes a control unit 201 and a storage unit 202.

  The storage unit 202 is configured by, for example, a ROM and a RAM. The storage unit 202 stores various programs for causing the data management apparatus 200 to function. The storage unit 202 also includes image data of the first image 51 captured by the first imaging unit of the imaging apparatus 101 (stereo camera) and the second image 52 captured by the second imaging unit, and the first image 51 and the second image. The image data of the distance image 53 generated from the image 52 and the image data of the distance image 53A generated by rotating the distance image 53 are stored.

  The control unit 201 includes, for example, a CPU, and executes various programs stored in the storage unit 202, thereby generating a generation unit 203, a counting unit 204, a specifying unit 205, an image control unit 206, a detection unit 207, and a monitoring unit. It functions as the control unit 208.

  The generation unit 203 generates a distance image 53 from the first image 51 photographed by the first photographing unit of the photographing apparatus 101 and the second image 52 photographed by the second photographing unit. Specifically, the generation unit 203 specifies the parallax between the first image 51 and the second image 52 by a known method such as the SAD method, and generates the distance image 53 based on the specified parallax.

  The counting unit 204 scans the distance image 53 in the X direction and counts the number of pixels with zero parallax for each Y coordinate, and scans the distance image 53 in the Y direction for the number of pixels with zero parallax for each X coordinate. Count.

  The specifying unit 205 specifies the positions (Y coordinates) of the platform P, the home end Pe, and the rail Ra based on the counting result regarding the Y coordinate of the counting unit 204. Further, the specifying unit 205 specifies each position (X coordinate) of the plurality of sleepers Rt installed orthogonal to the rail Ra, based on the counting result regarding the X coordinate of the counting unit 204.

  As will be described later with reference to FIG. 6, when a ballast trajectory is imaged as in the present embodiment, the number of pixels with zero parallax increases at the positions (Y coordinates) of the platform P and the rail Ra. On the other hand, since the parallax is calculated at the gravel Rb position (Y coordinate), the number of pixels with zero parallax is reduced.

The identifying unit 205 uses the Y coordinate having the largest Y coordinate among the Y coordinates in which the number of pixels with zero parallax exceeds a predetermined threshold as the platform P, and sets the next largest Y coordinate to the rail on the near side of the two rails Ra. Ra (hereinafter referred to as “lower rail Rad”), and the rail having the next largest Y coordinate is referred to as the rear rail Ra (hereinafter referred to as “upper rail Rau”) of the two rails Ra. And the specific | specification part 205 pinpoints the boundary part of the platform P and gravel Rb with the home end Pe.
Further, the specifying unit 205 sets the position (Y coordinate) of the upper rail Rau as the coordinate R1, the position (Y coordinate) of the lower rail Rad as the coordinate R2, and the position (Y coordinate) of the home end Pe as the coordinate R3.

Further, as will be described later with reference to FIG. 8, since the parallax cannot be calculated on the sleeper Rt on the track R of the ballast track, the number of pixels with zero parallax increases at the position (X coordinate) of the sleeper Rt. On the other hand, since the parallax is calculated for the gravel Rb, the number of pixels with zero parallax is reduced at the position of the gravel Rb (X coordinate).
The specifying unit 205 specifies the X coordinate where the number of pixels with zero parallax exceeds a predetermined threshold as the position of the sleeper Rt (X coordinate).

Returning to FIG. 5, the image control unit 206 rotates the distance image 53 based on the counting result of the counting unit 204. As described above, in the present embodiment, the distance image 53 is rotated by yaw rotation with the Z axis as the rotation axis and roll rotation with the X axis as the rotation axis.
Hereinafter, rotation control of the distance image 53 by the image control unit 206 will be described.

[Yaw rotation around the Z axis]
The image control unit 206 yaw-rotates the distance image 53 based on the counting result of the number of pixels with zero parallax for each Y coordinate. Specifically, the image control unit 206 yaw-rotates the distance image 53 based on the number of pixels with zero parallax at the coordinates R1 and R2 that are the coordinates of the rail Ra.
As described above, the yaw rotation is performed when the track R and the platform P are inclined with respect to the X direction. Here, FIG. 6 is a diagram illustrating an operation example when the image control unit 206 performs yaw rotation of the distance image 53.

  When the line R or the like extends horizontally in the X direction as shown in FIG. 6A, the count result of the number of pixels with zero parallax for each Y coordinate is as shown in the histogram shown in FIG. On the other hand, when the line R or the like is inclined with respect to the X direction as shown in FIG. 6C, the count result of the number of pixels with zero parallax for each Y coordinate is as shown in the histogram shown in FIG. Become.

Referring to FIG. 6B, when the line R or the like extends horizontally in the X direction, the number of pixels with zero parallax at the position of the rail Ra (coordinates R1, R2) and the pixel with zero parallax at the position of the gravel Rb. The difference with the number appears significantly, and the peak of the number of pixels with zero parallax (the maximum value, the 90th percentile value, etc.) is also large. On the other hand, referring to FIG. 6D, when the line R or the like is inclined with respect to the X direction, the difference between the number of pixels with zero parallax at the position of the rail Ra and the number of pixels with zero parallax at the position of the gravel Rb. And the peak of the number of pixels with zero parallax is also small.
Therefore, in the counting result of the number of pixels with zero parallax for each Y coordinate, it is possible to grasp the inclination degree of the line R or the like, in other words, the inclination degree of the photographing apparatus 101, from the steepness of the peak of the pixel number with zero parallax.

  The image control unit 206 determines the rotation angle for yaw rotation based on the number of pixels with zero parallax at the position of the rail Ra, and yaw rotates the distance image 53 by the rotation angle. Specifically, the image control unit 206 determines the rotation angle at which the peak of the number of pixels with zero parallax is the steepest, and rotates the distance image 53 by yaw. As a result, a distance image 53A (FIG. 6A) in which the lines R and the like extend horizontally in the X direction from a distance image 53 (FIG. 6C) in which the lines R and the like are inclined with respect to the X direction. Generated.

  Note that the image control unit 206 may slide the distance image 53 so that the home end Pe is at the center as shown in FIG. 7 after the yaw rotation so as to be horizontal to the track R or the like. Specifically, the image control unit 206 translates the distance image 53 so that the coordinate R3 of the home end Pe specified by the specifying unit 205 matches the center coordinate C of the Y axis, and generates the distance image 53A. It is good as well.

[Roll rotation around X axis]
The image control unit 206 roll-rotates the distance image 53 about the coordinate R3 of the home end Pe as a rotation axis based on the counting result of the number of pixels with zero parallax for each X coordinate. Specifically, the image control unit 206 roll-rotates the distance image 53 based on the sleeper Rt specified from the count result of the number of pixels with zero parallax for each X coordinate.
As described above, the roll rotation is performed in order to obtain the distance image 53A as if the home end Pe, which is looking down obliquely by the setback, was photographed from directly above. Here, FIG. 8 is a diagram illustrating an operation example when the image control unit 206 rotates the distance image 53 in a roll manner.

Since the sleepers Rt are installed perpendicular to the rail Ra on the track R, each sleeper Rt is parallel in the Y direction in the real space. On the other hand, in the state where the photographing apparatus 101 is photographing while looking down at the track R or the like, the distance from the photographing apparatus 101 is different between the platform P side of the sleeper Rt and the far side of the track R. Each sleeper Rt is not parallel.
In this regard, when the home end Pe is photographed from directly above, the distance from the photographing device 101 is equal on the platform P side of the sleeper Rt and the far side of the track R (more specifically, than when viewed obliquely). Approximate).

Therefore, in the present embodiment, as shown in FIG. 8A, a state where each of the plurality of sleepers Rt constituting the line R is parallel along the Y direction is photographed from directly above the end of the platform P. In this state, the image control unit 206 acquires the amount of rotation necessary to reach the state based on the count result of the number of pixels with zero parallax for each X coordinate.
Here, when each sleeper Rt is parallel to the Y direction as shown in FIG. 8A, the count result of the number of pixels with zero parallax for each X coordinate is as shown in the histogram shown in FIG. 8B. . On the other hand, as shown in FIG. 8C, when each of the sleepers Rt is not parallel to the Y direction, the counting result of the number of pixels with zero parallax for each X coordinate is like the histogram shown in FIG. become.

Similar to the number of pixels with zero parallax for each Y coordinate described above in FIG. 6, the number of pixels with zero parallax and gravel Rb at the position of the sleeper Rt are determined depending on whether the sleepers Rt are parallel to the Y direction or not. And the peak of the number of pixels with zero parallax (maximum value, 90th percentile value, etc.) are different.
The image control unit 206 determines a rotation angle for rotating the distance image 53 based on these tendencies.

  Hereinafter, a method for determining the rotation angle of the image control unit 206 will be specifically described. FIG. 9 is a diagram schematically illustrating a histogram indicating the number of pixels with zero parallax for each X coordinate when the sleeper Rt is parallel to the Y direction.

To simplify the understanding, at positions other than the sleeper Rt (X coordinate), the number of pixels with zero parallax is counted for the platform P and the rail Ra, and at the position of the sleeper Rt, the platform P and the rail Ra are counted. In addition, the number of pixels with zero parallax is counted for the sleeper Rt. Since the platform P and the rail Ra have a substantially constant width (height), and the length of the sleepers Rt also substantially coincide with each other in the plurality of sleepers Rt, the peak of the number of pixels with zero parallax is in each of the plurality of sleepers Rt. In general.
Therefore, the image control unit 206 determines whether or not the peaks of the number of pixels with zero parallax for each X coordinate substantially coincide with each other so that each of the sleepers Rt becomes parallel along the Y direction, In other words, the rotation angle for obtaining the distance image 53A as if the home end Pe was photographed from directly above can be determined.

Moreover, when each of the plurality of sleepers Rt is parallel, the width of the sleepers Rt and the interval between the sleepers Rt are also substantially the same. Therefore, the image control unit 206 can also determine the rotation angle based on the peak width of the number of pixels with zero parallax for each X coordinate and the interval at which the peaks appear.
Note that the image control unit 206 may determine the rotation angle by using any one of these conditions alone, or may determine the rotation angle by using any combination of conditions. Also good.

By the way, even if the home end Pe is photographed from directly above, the distance from the photographing device 101 to the platform P side of the sleeper Rt and the distance from the photographing device 101 to the back side of the track R of the sleeper Rt are completely the same. Not to do. Further, since the rotation is performed based only on the XYZ coordinates obtained based on the photographing result from one direction, that is, the distance image 53, there is a predetermined limit in the matching accuracy between the rotated shape and the actual shape.
Therefore, the state in which each of the sleepers Rt is parallel along the Y direction does not require that each of the sleepers Rt be completely parallel, and it is sufficient if the parallelism satisfies a predetermined threshold value. The predetermined threshold is set empirically by experiments or the like.

  Returning to FIG. 5, the detection unit 207 detects a change in the installation state of the imaging apparatus 101 that has been installed and is being monitored. Specifically, the detection unit 207 includes the count result of the counting unit 204 in the first distance image 53 generated from the captured image captured by the imaging apparatus 101 at the first timing (for example, at the time of installation), and the imaging apparatus. 101 is compared with the counting result of the counting unit 204 in the second distance image 53 generated from the captured image captured at the second timing (for example, during monitoring operation) different from the first timing, and a predetermined value is set between the two. When there is a difference as described above, a change in the installation state of the photographing apparatus 101 is detected.

  The detection of the change in the installation state of the photographing apparatus 101 by the detection unit 207 may be performed by comparing histograms (counting results) in the distance image 53 before the image control unit 206 performs rotation control. This may be performed by comparing histograms in the distance image 53A after the rotation of the unit 206 is controlled. The detection by the detection unit 207 may be performed by comparing the histograms for each Y coordinate (FIG. 6B), or by comparing the histograms for each X coordinate (FIG. 8B). It is good as well.

  Here, since the histogram in the distance image 53A after the rotation control has a certain tendency (for example, peak steepness, sleeper width, interval, etc.), the detection unit 207 uses the first timing histogram ( The change in the installation state of the imaging device 101 may be detected by determining whether or not the histogram in the distance image 53A at the second timing has the certain tendency without using the counting result.

  When the detection unit 207 detects a change in the installation state of the imaging apparatus 101, the installation state of the imaging apparatus 101 is adjusted automatically or manually. The manual adjustment is, for example, that the administrator physically adjusts the installation state of the photographing apparatus 101, and the automatic adjustment is that the installation state is virtually adjusted by rotation control by the image control unit 206. It is.

  In the automatic adjustment, the image control unit 206 controls rotation of the second distance image 53 based on the counting result of the counting unit 204 in the second distance image 53 at the second timing. Since the rotation control by the image control unit 206 has already been described above, a detailed description thereof will be omitted.

  It should be noted that once the photographing apparatus 101 is installed, there is little deviation in the Y-axis direction. Therefore, in the rotation control when adjusting the change in the installation state, the translation in the Y-axis direction (see FIG. 7) may not be performed. The detection by the detection unit 207 is the same, and the determination as to whether or not the coordinate R3 of the home end Pe matches the center coordinate C of the Y axis may be omitted.

  The monitoring control unit 208 performs monitoring control of the platform P and the track R based on the distance image 53A that is rotationally controlled by the image control unit 206. Specifically, the monitoring control unit 208 sets a monitoring area for the distance image 53A and performs monitoring control.

  Here, an example of the monitoring area set for the distance image 53A is shown in FIG. The monitoring control unit 208 sets a fall monitoring area for monitoring the fall on the track R and an approach monitoring area for monitoring the approach to the home end Pe for the rotated distance image 53A. Specifically, the monitoring control unit 208 divides the entire region of the distance image 53A into a region on the track side and a region on the platform side with the home end Pe as a boundary, and any region of the region on the track side is divided. A fall monitoring area is set, and an arbitrary area of the platform area is set as an approach monitoring area.

When one track and another track are adjacent, depending on the route, materials may be placed between the tracks. In the fall monitoring area, it is necessary to set an appropriate range in the Y direction so that this material is not detected as a foreign object in the track.
In this regard, in the present embodiment, the monitoring control unit 208 sets the Y coordinate of the fall monitoring area in the range from the coordinate Y1 to the coordinate Y2, as shown in FIG. The coordinate Y1 is a position 50 cm behind the coordinate R1 of the upper rail Rau, and the coordinate Y2 is the coordinate R2 of the home end Pe. The depth 50 cm can be calculated based on the relationship between the size of one pixel of the image element and the distance from the photographing apparatus 101, and the photographing apparatus 101 is installed 4.3m above the track surface. In this case, 50 cm becomes 47 pixels.

As the approach monitoring area, it is necessary to set from the home end Pe to a predetermined inner area. Since many stations have Braille blocks installed on the platform P, the area from the platform edge Pe to the Braille blocks may be set as an approach monitoring area. Since they are not unified, in this embodiment, the approach monitoring area is set in the following range.
Specifically, as shown in FIG. 10, the monitoring control unit 208 sets the Y coordinate of the approach monitoring area within the range from the coordinate Y2 to the coordinate Y3. The coordinate Y3 is a position 90 cm in front of the coordinate Y2, and when the photographing apparatus 101 is installed 2.2 m above the platform surface, the coordinate Y2 + 163 (pixel) becomes the coordinate Y3.

  In addition, the range of the X direction of a fall monitoring area and an approach monitoring area can be set arbitrarily arbitrarily. As an example, the monitoring control unit 208 sets a range obtained by subtracting the margins on both sides and the parallax search area from the entire range in the X direction as the X direction range of the fall monitoring area and the approach monitoring area.

  The monitoring control unit 208 searches the distance image 53A and determines whether or not abnormal parallax exists in the fall monitoring area and the approach monitoring area. Then, when there is an abnormal parallax in the fall monitoring area and the approach monitoring area, the monitoring control unit 208 detects the fall on the track R or the approach to the home end Pe.

Further, the monitoring control unit 208 periodically monitors the histogram for each X coordinate and the histogram for each Y coordinate of the distance image 53A, and performs monitoring control based on the distribution state of the number of pixels with zero parallax in the X coordinate and the Y coordinate. Whether to continue or cancel.
Monitoring by the monitoring control unit 208 is performed by paying attention to the fact that the line R and the platform P become zero parallax in the distance image 53A. However, during snowfall or snowfall, the distribution of the number of pixels with zero parallax is significantly different from the normal time, and the monitoring accuracy is reduced. Therefore, the monitoring control unit 208 has a histogram or Y for each X coordinate of the distance image 53A. The surrounding weather conditions (snowfall and snowfall) are ascertained from the histogram for each coordinate, and monitoring control is stopped when the weather is not suitable for monitoring.

  FIG. 11 is a diagram schematically illustrating a histogram indicating the number of pixels with zero parallax according to the surrounding weather conditions. FIGS. 11A and 11C are diagrams in normal time (for example, clear sky). FIG. 11B is a histogram showing the number of pixels with zero parallax, FIG. 11B is a histogram showing the number of pixels with zero parallax during snowfall, and FIG. 11D shows the number of pixels with zero parallax during snowfall. It is a histogram. In FIG. 11, a histogram for each X coordinate of the distance image 53A is shown, but the histogram for each Y coordinate has the same tendency.

Since the number of pixels with zero parallax is large at the position of the sleeper Rt, as shown in FIG. 11A, the peak of the number of pixels with zero parallax (maximum value, 90th percentile value, etc.) Appears prominently.
On the other hand, at the time of snowfall, as a result of photographing the falling snow and falling raindrops in the first photographing unit and the second photographing unit, the parallax is calculated in the area where the snow and raindrops are present. As a result, as shown in FIG. 11B, in the histogram for each X coordinate during snowfall, the peak of the number of pixels with zero parallax is low, and the bottom of the number of pixels with zero parallax (minimum value, 10th percentile value, etc.) ) Becomes smaller.

Further, since the parallax is calculated at the position of the gravel Rb, as shown in FIG. 11C, the bottom of the pixel number with zero parallax appears prominently in the histogram for each X coordinate in the normal state.
On the other hand, at the time of snow accumulation, the gravel Rb pattern is obscured by the snow piled up on the track R, and the track surface is in a state of less texture like white. As a result, as shown in FIG. 11D, in the histogram for each X coordinate at the time of snowfall, the bottom of the number of pixels with zero parallax increases and the difference from the peak decreases.

  The monitoring control unit 208 pays attention to such a tendency, and stops the monitoring control when the distribution state of the number of pixels with zero parallax is not suitable for monitoring. The distribution state of the number of pixels with zero parallax may be, for example, the difference between the peak and the bottom of the number of pixels with zero parallax, and the average value, variance, standard of all the coordinates of the number of pixels with zero parallax It may be a deviation or the like, or may be a peak, bottom, median, etc. of the number of pixels with zero parallax.

[Processing of Imaging System 10]
Next, with reference to FIG. 12 to FIG. 14, a processing flow of the photographing system 10 of the present invention will be described. FIG. 12 is a flowchart illustrating a flow of installation setting processing for performing various settings when the imaging apparatus 101 is installed.

  When the installation worker installs the imaging device 101 on the platform P, in step S1, the generation unit 203 of the data management device 200 determines the parallax between the first image 51 and the second image 52 captured by the installed imaging device 101. A distance image 53 is generated. Subsequently, in step S2, the counting unit 204 of the data management device 200 scans the distance image 53, counts the number of pixels with zero parallax for each Y coordinate, and generates a histogram (see FIG. 6B).

Subsequently, in step S <b> 3, the image control unit 206 of the data management device 200 determines whether or not the platform P and the track R are photographed so as to be horizontal. Specifically, the image control unit 206 determines whether or not the peak of the number of pixels with zero parallax is steep by referring to the histogram for each Y coordinate. If the peak is steep, the image control unit 206 can take a picture horizontally. It is determined that
When this determination is NO, in step S4, the image control unit 206 of the data management device 200 yaw rotates the distance image 53 so that the platform P and the track R are horizontal.

  Subsequently, in step S5, the counting unit 204 of the data management apparatus 200 scans the distance image 53, counts the number of pixels with zero parallax for each X coordinate, and generates a histogram (see FIG. 8B).

Subsequently, in step S6, the image control unit 206 of the data management apparatus 200 determines whether or not the home end Pe has been photographed from directly above. Specifically, the image control unit 206 identifies the position of the sleeper Rt with reference to the histogram for each X coordinate, and shoots the home end Pe from directly above based on the interval, width, length, etc. of the sleeper Rt. Determine if it is done.
If this determination is NO, in step S7, the image control unit 206 of the data management device 200 rolls the distance image 53 so that the home end Pe is photographed from directly above.

  When the rotation control is performed so that the determinations in step S3 and step S6 are YES, in step S8, the data management apparatus 200 sets the rotated angle in association with the imaging apparatus 101, and the installation setting for the imaging apparatus 101 is performed. The process is completed, and the process proceeds to the installation setting process for the next photographing apparatus 101.

Next, FIG. 13 is a flowchart illustrating a monitoring area setting process for setting a monitoring area for the distance image 53.
First, in step S11, the counting unit 204 of the data management device 200 scans the distance image 53, counts the number of pixels with zero parallax for each Y coordinate, and generates a histogram. Subsequently, in step S12, the monitoring control unit 208 of the data management device 200 sets a fall monitoring area on the line R side with the home end Pe as a boundary, and approaches the platform P side with the home end Pe in step S13. Set the monitoring area.

Specifically, the monitoring control unit 208 sets an area (Y coordinate) from a predetermined position on the back side of the upper rail Rau to the home end Pe as a fall monitoring area. In addition, the monitoring control unit 208 sets an area (Y coordinate) from the home end Pe to a predetermined inner position as an approach monitoring area. In addition, the monitoring control unit 208 sets ranges in the X direction of the fall monitoring area and the approach monitoring area in consideration of the margins on both sides of the distance image 53, the parallax search area, and the like.
Thereby, a fall monitoring area and an approach monitoring area shown in FIG. 10 are set in the distance image 53.

  Next, FIG. 14 is a flowchart showing the flow of the monitoring setting process for making various settings for the photographing apparatus 101 during the monitoring operation. Note that the imaging system 10 monitors the platform P and the track R based on the distance image 53A obtained by performing rotation control on the distance image 53 captured by each imaging device 101 based on the rotation angle set in the installation setting process. .

  First, in step S21, the counting unit 204 of the data management device 200 scans the distance image 53, counts the number of pixels with zero parallax for each Y coordinate, and generates a histogram. Subsequently, in step S22, the detection unit 207 of the data management apparatus 200 determines whether there is a change in the installation state with respect to the horizontal setting. Specifically, the detection unit 207 compares the histogram of the installation setting process with the histogram generated in step S21, or the histogram generated in step S21 has a predetermined tendency (for example, zero parallax based on the rail Ra). It is determined whether or not there is a change in the installation state with respect to the horizontal setting.

If it is determined in step S22 that the installation state has changed, in step S23, the image control unit 206 of the data management device 200 rotates the distance image 53 by a predetermined angle. Subsequently, in step S24, the image control unit 206 of the data management apparatus 200 determines whether the rotation is within the rotation limit range. The rotation restriction range refers to a range in which the home end Pe, the fall monitoring area, and the approach monitoring area can be properly recognized in the rotated distance image 53A.
In step S24, when it is within the rotation limit range, the data management apparatus 200 moves the process to step S22, and repeats the processes of step S22 to step S24 until the horizontal setting shift is eliminated.

  When the horizontal setting shift is eliminated, in step S25, the counting unit 204 of the data management apparatus 200 scans the distance image 53, counts the number of pixels with zero parallax for each X coordinate, and generates a histogram. To do. Subsequently, in step S26, the detection unit 207 of the data management apparatus 200 determines whether there is a change in the installation state with respect to the vertical setting. Specifically, the detection unit 207 compares the histogram of the installation setting process with the histogram generated in step S25, or the histogram generated in step S25 has a predetermined tendency (for example, the interval between the sleepers Rt, the width). In other words, it is determined whether or not there is a change in the installation state with respect to the vertical setting.

If it is determined in step S26 that the installation state has changed, in step S27, the image control unit 206 of the data management apparatus 200 rolls the distance image 53 by a predetermined angle. Subsequently, in step S28, the image control unit 206 of the data management apparatus 200 determines whether the rotation is within the rotation limit range.
In step S28, if it is within the rotation limit range, the data management apparatus 200 moves the process to step S26, and repeats the processes of step S26 to step S28 until the vertical setting shift is eliminated.

  When the difference between the horizontal setting and the vertical setting is resolved, in step S29, the monitoring control unit 208 of the data management device 200 continues the monitoring control and ends the monitoring setting process.

  On the other hand, if the rotation control within the rotation limit range cannot eliminate the deviation of the horizontal setting or the vertical setting, in step S30, the monitoring control unit 208 of the data management apparatus 200 stops the monitoring control and ends the monitoring setting process. Note that, in the process of step S30, for example, in addition to the case where the installation angle of the photographing apparatus 101 is greatly shifted due to a strong impact or the like, the distribution state of the number of pixels with zero parallax is a predetermined tendency due to snowfall or snowfall as described above. This is done when there is a large deviation from

[Effect in the photographing system 10]
The embodiment of the present invention has been described above. Next, effects in the imaging system 10 will be described.

In the imaging system 10, when the rail Ra or the platform P is inclined in the distance image 53, the distance image 53 is yaw-rotated around the Z axis so that the rail Ra or the platform P is horizontal in the X direction. Specifically, in the imaging system 10, the distance image 53 is yaw-rotated so that the peak of the number of pixels with zero parallax becomes steep based on the histogram with zero parallax for each Y coordinate.
Thereby, when installing the photographing apparatus 101, it is not necessary to precisely adjust the mounting angle and the angle of view. Therefore, in the imaging system 10, it is not necessary to spend an enormous amount of time for installing the imaging apparatus 101, and the time required for the installation operation of the imaging apparatus 101 can be shortened.

Such an effect that the installation time can be shortened is more noticeable in the imaging system 10 that performs monitoring over the entire platform.
That is, in the imaging system 10, the platform is monitored using a large number of imaging apparatuses 101, but the installation work time can be shortened in each of the imaging apparatuses 101, so that the installation time is greatly reduced in the entire imaging system 10. be able to. In addition, although the installation of the photographing apparatus 101 is limited to nighttime so as not to affect the operation of the railway vehicle, the installation time can be greatly reduced. Can be constructed, which can contribute to the safe operation of railway vehicles.

Further, in the imaging system 10, paying attention to the fact that the three-dimensional data (XYZ coordinates) of the subject can be calculated from the distance image 53 captured by the imaging apparatus 101, the home end Pe is photographed from directly above. The distance image 53 is rotated around the axis. Specifically, in the imaging system 10, the distance image 53 is rotated based on the zero parallax histogram for each X coordinate so that the peak interval of the number of pixels with zero parallax is constant.
Thereby, in the imaging system 10, it is not necessary to spend an enormous amount of time for installing the imaging apparatus 101, and the time required for the installation operation of the imaging apparatus 101 can be shortened. Further, since the distance image 53A as if the home end Pe was photographed from right above can be obtained by the roll rotation, the monitoring accuracy of the boundary portion between the platform P and the track R is improved, and it exists on the platform. Regardless of this, it is possible to prevent erroneous detection that the track is falling.

In addition, the imaging system 10 detects a change in the installation state of the imaging apparatus 101 during monitoring operation. Specifically, when the zero parallax histogram during the monitoring operation is different from the zero parallax histogram at the time of installation by a predetermined amount or when it no longer has a certain tendency, the change in the installation state of the photographing apparatus 101 Is detected.
A stable operation of the imaging system 10 can be expected by making an appropriate adjustment according to the change in the installation state. The adjustment can be performed manually by the administrator by notifying the administrator of a change in the installation state, or can be automatically performed based on the rotation control of the image control unit 206, for example.

On the other hand, the imaging system 10 grasps the surrounding weather conditions (snowfall and snowfall) and stops monitoring control when the weather is not suitable for monitoring. Specifically, the imaging system 10 grasps the surrounding weather condition based on the distribution state of the number of pixels with zero parallax, and determines whether monitoring control is continued or stopped.
As described above, the tendency of the number of pixels with zero parallax is significantly different for snowfall and snowfall compared to normal times, so in this case, monitoring errors can be prevented and false detections and omissions can be prevented. Can do. In addition, when monitoring control is stopped, it is more preferable to notify the manager (station staff) appropriately.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above embodiment. It is apparent from the description of the scope of claims that embodiments with such changes or improvements can be included in the technical scope of the present invention.

DESCRIPTION OF SYMBOLS 10 ... Shooting system 100 ... Shooting device group 101 ... Shooting device 200 ... Data management device (image processing device)
201 ... Control unit 202 ... Storage unit 203 ... Generation unit 204 ... Counting unit 205 ... Specification unit 206 ... Image control unit 207 ... Detection unit 208 ... Monitoring control unit 300 ... Hub 400 ... Monitor

Claims (6)

An image processing apparatus for processing a captured image captured by a stereo camera installed on an upper part of a platform extending in a first direction,
A generating unit that generates a distance image based on the parallax between the first image captured by the first imaging unit of the stereo camera and the second image captured by the second imaging unit;
A counting unit that counts the number of pixels with zero parallax for each coordinate of the first coordinate axis along the second direction orthogonal to the first direction among the X coordinate axis and the Y coordinate axis of the generated distance image;
Based on the counting result for each coordinate of the first coordinate axis, an image control unit that yaw rotates the distance image with the Z coordinate axis orthogonal to the X coordinate axis and the Y coordinate axis as a rotation axis;
An image processing apparatus comprising: Based on the counting result for each coordinate of the first coordinate axis, further comprising a specifying unit that specifies a rail coordinate that is a position in the first coordinate axis of a rail installed along the platform;
The image control unit determines a rotation angle for yaw rotation of the distance image based on the number of pixels with zero parallax in the rail coordinates.
The image processing apparatus according to claim 1. A specifying unit that specifies a home end coordinate that is a position of the end of the platform in the first coordinate axis based on the counting result for each coordinate of the first coordinate axis;
The counting unit counts the number of pixels with zero parallax for each coordinate of the second coordinate axis along the first direction among the X coordinate axis and the Y coordinate axis of the distance image,
The image control unit roll-rotates the distance image around the home end coordinate based on a counting result for each coordinate of the second coordinate axis.
The image processing apparatus according to claim 1. The specifying unit specifies sleeper coordinates, which are positions on the second coordinate axes of the sleepers installed together with rails extending along the platform, for each of the plurality of sleepers based on the counting result for each coordinate of the second coordinate axes. And
The image control unit roll-rotates the distance image based on a count result of the number of pixels with zero parallax in the identified plurality of sleeper coordinates.
The image processing apparatus according to claim 3. Generated from the counting result of the counting unit in the first distance image generated from the captured image captured at the first timing by the stereo camera and the captured image captured at the second timing different from the first timing by the stereo camera. A detection unit that detects a change in the installation state of the stereo camera by comparing the counting result of the counting unit in the second distance image
The image processing apparatus according to claim 1, further comprising: A monitoring control unit that performs platform monitoring control based on a distance image that is rotationally controlled by the image control unit, based on a distribution state of the number of pixels with zero parallax in each coordinate of the coordinate axis that counts the number of pixels with zero parallax A monitoring control unit for stopping the monitoring control,
The image processing apparatus according to claim 1, further comprising:

Images