JP6476945B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus.

  As railway vehicles used by a large number of passengers are required to operate safely, railway vehicle personnel have made various attempts to operate the railway vehicles safely. For example, if a passenger falls from the platform to the track, it may lead to a serious accident, so many attempts have been made to detect the fall of the passenger quickly, and some of them are actually adopted at the railway station. ing.

As a specific example, a camera system is well known in which a plurality of cameras are installed under the ceiling of a platform and the plurality of cameras monitor a station yard such as a platform or a track top.
As such a camera system, for example, Patent Document 1 discloses a camera system in which a plurality of cameras are daisy-chained and monitoring information acquired by each camera is transmitted to a server via another camera. ing.

JP, 2012-11989, A

By the way, in the camera system installed in the upper part of the platform, it does not directly above the boundary between the platform and the track (that is, the platform edge) but by a predetermined amount on the platform side The camera must be installed at the set back position. In such an installation, the camera system monitors the station yard with the platform edge and the track obliquely looking down, so at the installation site, it is necessary to adjust the angle at which the camera is attached and the angle of view with high accuracy. It took a lot of time.
In particular, in order to monitor all over the platform, it is necessary to install a large number of cameras, and each camera requires its own adjustment, which requires a great deal of time and impairs the convenience of the passengers. As adjustment work needs to be performed at night so that there is no need for technology to reduce adjustment work.

  The present invention has been made in view of such a problem, and an object of the present invention is to provide an image processing apparatus capable of shortening the time required for the installation operation of a camera monitoring a platform.

  A first aspect of the present invention is an image processing apparatus for processing a first captured image captured by a stereo camera installed on a platform extending in a first direction, the first imaging unit of the stereo camera In the first distance image based on the parallax, and a first generation unit that generates a first distance image based on the parallax between the first image photographed in step h and the second image photographed in the second photographing unit An image processing apparatus, comprising: a calculation unit that calculates three-dimensional data of the object; and a second generation unit that corrects the first distance image based on the three-dimensional data and generates a corrected second distance image I will provide a.

  The image processing apparatus further includes a rotation control unit that calculates three-dimensional data after rotation of the object by rotating the three-dimensional data about an arbitrary position extending in the first direction as an axis, the second generation unit The second distance image may be generated based on the calculated three-dimensional data after rotation.

  Further, from the first distance image, a detection unit that detects a crosstie of a track extending along the first direction on the side surface of the platform, and each of a plurality of crosslinks that constitute the track with the rotation are the first The image processing apparatus may further include an acquisition unit for acquiring a rotation amount parallel to a second direction orthogonal to one direction, and the rotation control unit may rotate the three-dimensional data by the acquired rotation amount. .

  Further, the acquisition unit is configured such that each of the sleepers is parallel along a second direction based on coordinates of the first direction at a plurality of monitoring points different in the second direction of the detected one tie. It may also be determined.

  Further, the second distance image is divided into a first area and a second area bordering on the axis, and an arbitrary area of the first area is set as a first monitoring area, and the second area is selected. A monitoring setting unit that sets an arbitrary region as a second monitoring region; and a searching unit that determines which of the first monitoring region and the second monitoring region is a foreign object present in the second distance image; May be further provided.

  In addition, when the foreign substance is detected in the first distance image, the search unit rotates three-dimensional data of the foreign substance around the axis to calculate three-dimensional data of the foreign substance after rotation, and It may be determined whether the foreign matter is present in the first monitoring area or the second monitoring area based on three-dimensional data after rotation.

  ADVANTAGE OF THE INVENTION According to this invention, while being able to shorten the time required for the installation operation | work of the camera which monitors a platform, the danger in a platform and a track can be detected precisely.

It is a figure showing an example of composition of an imaging system concerning one embodiment. It is a figure which shows the example of installation of the imaging device in a station platform. It is a figure which shows the example of installation of the imaging device in a station platform. It is a figure which shows an example of the various picked-up image which the imaging device image | photographed. It is a figure which shows an example of the various picked-up image which the imaging device image | photographed. It is a figure showing an outline of a photography system. It is a figure showing an outline of a photography system. It is a block diagram which shows the function structure of a data management apparatus. It is a figure for demonstrating the rotation method of the distance image by a rotation control part. It is a figure for demonstrating the rotation method of the distance image by a rotation control part. It is a figure which shows an example of the monitoring area | region set to the distance image after rotation. It is a figure for demonstrating the flow of the searching method of the foreign material by the searching part. It is a flow chart which shows a flow of processing of an imaging system. It is a flow chart which shows a flow of processing of an imaging system. It is a figure for demonstrating the conventional monitoring operation method.

[Overall Configuration of Shooting System 10]
FIG. 1 is a view showing an entire configuration of a photographing system 10 according to the present embodiment, and FIG. 2 is a view showing an installation example of a photographing apparatus 101 on a platform P of a station. 2A is a view of the platform P of the station from the direction of travel of the railcar, and FIG. 2B is a view of the platform P of the station from a direction orthogonal to the direction of travel of the railcar.

As shown in FIG. 1, the imaging system 10 includes an imaging device group 100 and a data management device 200 communicably connected to the imaging device group 100.
The imaging system 10 may include the hub 300 between the data management device 200 and the imaging device group 100, and may include a plurality of imaging device groups 100 connected to the hub 300. In this case, for example, each imaging device group 100 may be installed on each of the plurality of platforms P in the station. The imaging system 10 may also include a monitor 400 that displays an image captured by the imaging device group 100.

  In the imaging device group 100, a plurality of imaging devices 101 (imaging device 101-1, imaging device 101-2, ... imaging device 101-n, where n is a natural number of 3 or more) are daisy-chained along a predetermined direction Is configured. As shown in FIG. 2B, in the present embodiment, the plurality of photographing devices 101 are daisy-chained connected along the length direction of the platform P of the station.

  The photographing device 101 is a stereo camera in which a plurality of photographing units are arranged horizontally along the length direction of the platform P of the station. As shown in FIG. 2A, the imaging apparatus 101 does not affect the operation of the railway vehicle, in the upper portion (ceiling) of the station platform P, the boundary between the platform P and the track R (ie, the track of the platform P). It is installed by setting back a predetermined amount to the inside of the platform P from directly above the end on the R side. Further, as shown in FIG. 2B, the photographing devices 101 are installed so that the photographing ranges S of the respective photographing devices 101 overlap the photographing ranges S of the adjacent photographing devices 101.

  The imaging device 101 captures an image of the periphery of the platform P, that is, the platform P and the track R from above. Here, FIG. 3A shows an example of a first image 51 and a second image 52 photographed by the photographing device 101, and a distance image 53 generated from the first image 51 and the second image 52.

The first image 51 is a photographed image photographed by the first photographing unit of the photographing device 101, and the second image 52 is a photographed image photographed by the second photographing unit of the photographing device 101.
As shown in the first image 51 of FIG. 3A, the imaging device 101 captures the platform P and the track R. In the present embodiment, the track R is a ballast track configured of a rail Ra, a plurality of sleepers Rt, and a gravel Rb.

The distance image 53 is an image generated based on the parallax between the first image 51 and the second image 52.
Here, the method of calculating the parallax is already known, and the parallax is calculated by searching for a corresponding point corresponding to the reference point of the reference image (first image 51) from the comparison image (second image 52). Specifically, a pixel block is extracted from the reference image centering on the reference pixel, and among pixel blocks in the parallax search area of the comparison image, a pixel block whose luminance pattern is similar to the pixel block extracted from the reference image Is identified by a known method such as SAD (Sum of Absolute Differences) method. Then, the parallax is calculated from the difference between the coordinates of the pixel block of the reference image and the coordinates of the pixel block of the comparison image.

  Since the parallax is calculated for the region where the texture is clear, the surface pattern with less texture such as platform P, rail Ra and tie Rt is not calculated for the uniform object. Therefore, when photographing the ballast track with the photographing device 101, as shown in the distance image 53 of FIG. 3A, the parallax can not be calculated for the platform P, the rails Ra and the sleepers Rt, and the pattern is characteristic like gravel Rb The parallax can be calculated in the area where the texture is clear.

  In each image, the horizontal direction is the X direction (first direction), and the vertical direction is the Y direction (second direction). Since the plurality of imaging units of the imaging device 101 are horizontally arranged along the length direction of the platform P of the station, the X direction corresponds to the length direction of the platform P. Also, the Y direction corresponds to the width direction of the platform P and the line R. In the following, for the convenience of description, the upper part of the image is taken as the “Y” coordinate, and the lower part of the screen is taken as the “Y” coordinate.

  The first image 51 and the second image 52 captured by the imaging device 101 and the distance image 53 generated from the first image 51 and the second image 52 are transmitted to the data management device 200 via a predetermined communication line. And stored in the storage unit 202 of the data management apparatus 200. Note that the distance image 53 may be generated by the data management apparatus 200 without being generated by the imaging device 101.

[Overview of shooting system 10]
Subsequently, an outline of the imaging system 10 according to the present embodiment will be described. Since the photographing device 101 is installed by setting back from the end of the platform P, the photographing device 101 looks down at the end of the platform P and the track R at an angle. Here, as shown in FIG. 2A, when the passenger stands at the end of the platform P, in the distance image 53 captured by the imaging device 101, the upper body of the passenger exists in the track as shown in FIG. 3B. It appears as if it were, and there is a risk of false detection of a line fall.

  Therefore, in the imaging system 10, as shown in FIG. 4A, the distance image 53 is rotated about an arbitrary axis extending in the longitudinal direction (X direction) of the platform P, and a new distance image 54 is generated. In addition, in the station yard monitoring, it is necessary to monitor with high accuracy whether the hazard is occurring on the platform or in the track. In such a monitoring, it is preferable to monitor the boundary between the platform P and the line R from directly above, and in this embodiment, in particular, the distance image 53 around the end of the platform P extending in the longitudinal direction of the platform P. Rotate.

  As a result of this rotation, as shown in FIG. 4B, it is possible to obtain a distance image 54 as if the platform end was photographed from directly above, from the imaging device 101 installed by setting back from the platform end. Thereby, even in the case of setting back from the end of the platform P, the danger in the platform and the track can be accurately monitored.

Moreover, since the monitoring accuracy of the imaging device 101 can be adjusted by the rotation processing after installation, there is no need to precisely adjust the attachment angle and the angle of view when the imaging device 101 is installed. Therefore, in the imaging system 10, the installation of the imaging device 101 does not need to take a long time, and the time required for the installation operation of the imaging device 101 can be shortened.
Hereinafter, details of the data management apparatus 200 that enables such control will be described.

[Functional Configuration of Data Management Device 200]
FIG. 5 is a block diagram showing a functional configuration of the data management device 200. As shown in FIG. The data management device 200 is configured to include a control unit 201 and a storage unit 202.

  The storage unit 202 is configured of, for example, a ROM, a RAM, and the like. The storage unit 202 stores various programs for causing the data management apparatus 200 to function. In addition, the storage unit 202 includes the first image 51 captured by the first imaging unit of the imaging apparatus 101 (stereo camera) and the image data of the second image 52 captured by the second imaging unit, and the first image 51 and the second image. The image data of the distance image 53 generated from the image 52 and the image data of the distance image 54 generated by rotating the distance image 53 are stored.

  The control unit 201 is constituted by, for example, a CPU, and executes various programs stored in the storage unit 202 to thereby execute a first generation unit 203, a calculation unit 204, a rotation control unit 205, a second generation unit 206, and monitoring. It functions as the setting unit 207 and the search unit 208.

  The first generation unit 203 generates a distance image 53 (first distance image) from the first image 51 captured by the first imaging unit of the imaging device 101 and the second image 52 captured by the second imaging unit. Specifically, the first generation unit 203 specifies the parallax between the first image 51 and the second image 52 by a known method such as the SAD method, and generates the distance image 53 based on the specified parallax.

  The calculation unit 204 calculates three-dimensional data of an object in the distance image 53 based on the parallax between the first image 51 and the second image 52. When an object is present in the imaging range of the imaging device 101, the object has a pattern different from that of the other area, so parallax occurs between the first image 51 and the second image 52, and the object in the distance image 53 Appears. If the parallax is known, the distance from the imaging device 101 to the object can be measured according to the principle of triangulation, so the calculation unit 204 calculates the Z coordinate of the object in addition to the XY coordinates of the object in the distance image 53. Can. The calculation unit 204 calculates these XYZ coordinates as three-dimensional data of the object.

The second generation unit 206 corrects the distance image 53 (first distance image) by performing a predetermined operation on the three-dimensional data (XYZ coordinates) calculated by the calculation unit 204, and the corrected distance image 54. (2nd distance image) is generated. In the present embodiment, as an example, the second generation unit 206 generates the distance image 54 using the three-dimensional data rotated by the rotation control unit 205.
The details will be described below.

The rotation control unit 205 rotates the three-dimensional data of the object about an arbitrary axis extending in the X direction of the distance image 53 to calculate three-dimensional data after the rotation of the object. Such rotation control is already known as coordinate conversion, and is performed by calculating how an object represented by local coordinates is arranged in the global coordinate system.
Specifically, since the distance image 53 is generated based on the parallax between the first image 51 and the second image 52, the object in the distance image 53 is given three-dimensional data (local coordinates) of XYZ coordinates. Be The rotation control unit 205 arranges the three-dimensional data (local coordinates) in the world coordinate system based on the rotation axis extending in the X direction, and rotates around the rotation axis to obtain the three-dimensional object after rotation. Calculate dimensional data.

  The second generation unit 206 generates a distance image 54 based on the calculated three-dimensional data after rotation. Since the distance (distance before rotation) of the object from the photographing device 101 is known in the distance image 53, the second generation unit 206 determines the distance before rotation as three-dimensional data of the object before rotation and the distance after rotation. The corrected distance image 54 can be generated by performing correction based on the difference from the three-dimensional data of the object.

Here, as described above, the rotation axis extending in the X direction is the end of the platform P, and such rotation processing is a distance image 54 as if the end of the platform P was photographed from directly above. Do to get
Therefore, the rotation control unit 205 specifies a specific unit 209 for specifying the end of the platform P as the rotation axis from the distance image 53, and the amount of rotation necessary for photographing the end of the platform P from directly above. A detection unit 210 and an acquisition unit 211 for acquisition may be included.

The identifying unit 209 identifies the position (Y coordinate) of the end of the platform in the distance image 53 based on the parallax between the first image 51 and the second image 52.
Specifically, as shown in FIG. 6A, the identifying unit 209 scans the distance image 53 in the Y direction, and counts the number of pixels with zero parallax for each Y coordinate. Then, the specifying unit 209 detects the position (Y coordinate) of the platform P and the rail Ra based on the number of pixels with zero parallax.

  FIG. 6 (B) is a histogram showing the relationship between the Y coordinate and the number of pixels with zero parallax. When the ballast trajectory is photographed as in the present embodiment, the number of pixels with zero parallax increases at the position (Y coordinate) of the platform P and the rail Ra extending along the X direction of the photographed image. On the other hand, since the parallax is calculated at the position of the gravel Rb (Y coordinate), the number of pixels with zero parallax decreases.

Among the Y coordinates having a large number of pixels with zero parallax, the identifying unit 209 sets the platform P having the largest Y coordinate as the platform P and the next largest one of the Y coordinates is the rail Ra on the front side of the two rails Ra (hereinafter, It is referred to as "lower rail Rad", and the one with the next largest Y coordinate is referred to as the rear rail Ra of the two rails Ra (hereinafter referred to as "upper rail Rau").
Then, the specifying unit 209 specifies the Y coordinate “R3” of the boundary portion between the platform P and the gravel Rb as the end of the platform P. Further, the specifying unit 209 sets the position (Y coordinate) of the upper rail Rau as a coordinate R1, and sets the position (Y coordinate) of the lower rail Rad as a coordinate R2.

The detection unit 210 detects, from the distance image 53, the position of a tie Rt of the track R extending along the X direction on the side surface of the platform P.
Specifically, the detection unit 210 scans an arbitrary position in the region (Y coordinate) in which the upper rail Rau and the lower rail Rad exist in the distance image 53 in the X direction, and the position of the sleeper Rt (X coordinate ) To detect. In the example shown in FIG. 7, the section between the upper rail Rau and the lower rail Rad is divided into three, and the coordinates M1 and the coordinate M2 are set at positions where the section between the upper rail Rau and the lower rail Rad is divided into three. Further, since the sleeper Rt is installed extending to the front side and the rear side of the rail Ra, the detection unit 210 sets the coordinate M3 at a position on the front side of the lower rail Rad, and is deeper than the upper rail Rau. Set the coordinate M4 to the side position.
The detection unit 210 sets “coordinate R2 + α1 of the lower rail Rad” as the coordinate M3 and sets “coordinate R1-α2 of the upper rail Rau” as the coordinate M4. At this time, α1 and α2 are values set empirically by experiment or the like.

Subsequently, the detection unit 210 scans the distance image 53 in the X direction for each of the coordinate M1 to the coordinate M4, and counts the number of pixels with zero parallax for each X coordinate. FIG. 7B is a histogram showing, for example, the relationship between the X coordinate and the number of pixels with zero parallax when the position of the coordinate M1 is scanned in the X direction.
Since the parallax can not be calculated in the sleeper Rt, the number of pixels with zero parallax increases at the position (X coordinate) of the sleeper Rt. On the other hand, in the gravel Rb, since the parallax is calculated, the number of pixels with zero parallax decreases at the position (X coordinate) of the gravel Rb. The detection unit 210 detects the position (X coordinate) of the crosstie Rt based on the number of pixels with zero parallax.

The acquisition unit 211 acquires the amount of rotation required to obtain the distance image 54 as if the end of the platform P was photographed from directly above. Since the sleepers Rt are installed perpendicularly to the upper rail Rau and the lower rail Rad in the track R, the respective sleepers Rt are installed in parallel in the Y direction.
In this respect, when the platform P and the track R are viewed obliquely from below, the distance from the photographing device 101 differs between the platform P side of the sleeper Rt and the back side of the track R, as shown in FIG. 7C. , Each sleeper Rt not parallel. On the other hand, when the end of the platform P is photographed from directly above, the distance from the photographing device 101 becomes equal between the platform P side of the tie Rt and the back side of the track R (more specifically, looking down obliquely) Approximates more than the case).

  Therefore, in the present embodiment, as shown in FIG. 7D, the end of the platform P is photographed from directly above the state in which each of a plurality of sleepers Rt constituting the line R is parallel along the Y direction In the state, the acquisition unit 211 acquires the amount of rotation required to reach the state as a amount of rotation necessary to obtain the distance image 54 as if the end of the platform P was photographed from directly above.

  Specifically, the acquiring unit 211 determines that each of the sleepers Rt is parallel along the Y direction based on the coordinates in the X direction at a plurality of different monitoring points in the Y direction among the detected one sleepers Rt. Determine In the case of the example shown in FIG. 7, the coordinates M1 to the coordinates M4 are positions at which Y coordinates are different. The acquisition unit 211 compares the X coordinates of the sleepers Rt at each of the coordinates M1 to the coordinates M4, and determines that each of the sleepers Rt is parallel along the Y direction when the X coordinates substantially match.

Even when the platform end is photographed from directly above, the distance from the imaging device 101 to the platform P side of the sleeper Rt completely matches the distance from the imaging device 101 to the back side of the track R of the sleeper Rt. Do not mean. In addition, since the rotation is performed based only on the imaging result from one direction, which is the distance image 53, the matching accuracy between the shape after rotation and the actual shape also has a predetermined limit.
Therefore, the state in which the respective sleepers Rt are parallel along the Y direction does not require that the respective sleepers Rt completely coincide with each other, and it is sufficient if the degree of parallelism satisfies a predetermined threshold. The predetermined threshold is set empirically by experiment or the like.

  The amount of rotation acquired by the acquisition unit 211 is stored in the storage unit 202 in association with the imaging device 101 that has captured the distance image 53 of interest. The rotation control unit 205 reads the amount of rotation set corresponding to the imaging device 101 from the storage unit 202, and rotates the distance image 53 by the amount of rotation to calculate three-dimensional data after rotation.

  In this way, it is necessary to precisely adjust the angle and angle of view to be attached when the imaging device 101 is installed by acquiring and setting the amount of rotation in which the platform end is imaged from directly above for each imaging device 101 There is no As a result, in the imaging system 10, it is not necessary to spend a great deal of time installing the imaging device 101, and the time required for the installation operation of the imaging device 101 can be shortened.

  Returning to FIG. 5, the monitoring setting unit 207 sets, for the distance image 54 after rotation, a fall detection area for monitoring a fall on the track R and an approach detection area for monitoring an approach to the platform end. Specifically, the monitoring setting unit 207 divides the distance image 54 into the area on the line side and the area on the platform side with the rotation axis (platform end) as a boundary, and any area in the line area is A drop detection area is set, and an arbitrary one of the areas on the platform side is set as an approach detection area.

When one track and another track are adjacent to each other, materials may be placed between the tracks depending on the line. In the fall detection area, it is necessary to set an appropriate range in the Y direction so as not to detect this material as foreign matter in the track.
In this respect, in the present embodiment, as shown in FIG. 8, the monitor setting unit 207 sets the Y coordinate of the fall detection area in the range from the coordinate Y1 to the coordinate Y2. The coordinate Y1 is 50 cm behind the coordinate R1 of the upper rail Rau, and the coordinate Y2 is a coordinate R2 of the platform end. The depth 50 cm can be calculated based on the relationship between the size of one pixel of the image element and the distance from the photographing device 101, and the photographing device 101 is installed 4.3 m above the track surface In this case, 50 cm is 47 pixels.

As the approach detection area, it is necessary to set from the platform end to the inner predetermined area. In many stations, the platform P has braille blocks installed, so the area from the platform end to the braille blocks may be set as an approach detection area, but the installation pattern of braille blocks is uniform at all stations. Because this is not the case, in the present embodiment, the proximity detection area is set in the following range.
Specifically, as shown in FIG. 8, the monitoring setting unit 207 sets the Y coordinate of the approach detection area in the range from the coordinate Y2 to the coordinate Y3. The coordinate Y3 is a position 90 cm on the near side of the coordinate Y2. When the imaging device 101 is installed 2.2 m above the platform surface, the coordinate Y2 + 163 (pixel) is the coordinate Y3.

  The ranges of the fall detection area and the approach detection area in the X direction can be arbitrarily set as appropriate. As an example, the monitoring setting unit 207 sets a range obtained by subtracting the margin on both sides and the parallax search area from the entire range in the X direction as the range of the fall detection area and the approach detection area in the X direction.

Referring back to FIG. 5, the searching unit 208 detects and detects a foreign object (for example, a person or a thing existing in the track or a person or a thing near the platform end) in the monitoring area of the imaging device 101 It is determined whether the foreign matter is present in the fall detection area or the approach detection area.
Specifically, when the searching unit 208 detects a foreign object (abnormal parallax) in the distance image 53 captured by the imaging device 101, the acquisition unit 211 acquires three-dimensional data of the foreign object with the platform end as the rotation axis. The volume is rotated, and three-dimensional data of the foreign matter after rotation is calculated. Then, the search unit 208 compares the calculated three-dimensional data after rotation with the fall detection area and the approach detection area set for the distance image 54, and the foreign matter is either in the fall detection area or the approach detection area. Determine if it exists.

Here, with reference to FIG. 9, a method of searching for foreign matter by the searching unit 208 will be specifically described. In order to facilitate the description, in the present embodiment, as shown in FIG. 9A, a method of searching for a foreign object by the searching unit 208 will be described by taking an example where there is a rectangular solid object at the platform end.
When foreign matter is present at the platform end, the foreign matter appears as anomalous parallax in the distance image 53, as shown in FIG. 9B. Since the imaging device 101 is installed so as to look down at the platform end and the track obliquely, an anomalous parallax exists in the area in the track in the distance image 53 even when foreign matter is present in the platform, It is determined as if foreign matter is present in the track.

  Since the three-dimensional data (XYZ coordinates) of the foreign object can be calculated from the parallax between the first imaging unit and the second imaging unit for the foreign object, the searching unit 208 rotates the foreign object detected with the platform end as the rotation axis Do. As a result, as shown in FIG. 9C, it is possible to obtain a distance image 54 as if the platform end was photographed from directly above. Whether the foreign substance is present in the platform by determining whether the predetermined area is present in the fall detection area or the proximity detection area, focusing on the predetermined area at the center of the foreign substance in the distance image 54 , Determine whether it exists in the track.

  In addition, since the imaging by the imaging device 101 is from only one direction, the shape of the foreign object after rotation is distorted with respect to the actual shape without matching the actual shape when the foreign object is photographed from directly above. I will. However, for monitoring the platform P and the line R, it is sufficient if it is possible to detect the line falling and the approach to the platform end, and it is not necessary to accurately determine the actual shape of the foreign matter. Therefore, even if the shape of the foreign object after rotation is distorted, the station yard can be appropriately monitored.

[Process of imaging system 10]
Subsequently, the flow of processing of the imaging system 10 of the present invention will be described with reference to FIGS. 10 and 11. FIG. 10 is a flowchart showing a flow of setting processing for performing various settings on the photographing apparatus 101 installed on the platform P.

  As shown in FIG. 10, first, in step S1, the imaging device 101 captures an image of the track R to obtain a first image 51 and a second image 52. Subsequently, in step S2, the first generation unit 203 of the data management device 200 generates a distance image 53 from the parallax of the captured first image 51 and second image 52.

  Subsequently, in step S <b> 3, the calculation unit 204 calculates three-dimensional data (XYZ coordinates) of the object included in the distance image 53 based on the parallax of the first image 51 and the second image 52. Subsequently, in step S4, the identifying unit 209 identifies the platform end of the distance image 53. For example, the identifying unit 209 scans the distance image 53 in the Y direction to generate a histogram indicating the relationship between the Y coordinate and the number of pixels with zero parallax, and the position of the platform end based on the number of pixels with zero parallax. Identify (Y coordinate).

  Subsequently, in step S5, the rotation control unit 205 rotates the three-dimensional data of the object calculated in step S3 around the platform end identified in step S4. This rotation is performed until it is determined in step S6 that the sleepers Rt have become parallel. In the determination in step S6, the rotation control unit 205 (acquisition unit 211) determines that the sleepers Rt are parallel when the X direction coordinates at a plurality of different monitoring points in the Y direction of the sleepers Rt substantially match. It is determined that

  When the sleepers Rt become parallel due to the rotation, subsequently, in step S7, the rotation control unit 205 (acquisition unit 211) associates the amount of rotation required until the state becomes the imaging device 101 photographed in step S1. To register. In step S8, the monitoring setting unit 207 sets the fall detection area and the approach detection area on the rotated distance image 54, registers the fall detection area and the approach detection area in association with the imaging apparatus 101, and ends the setting process.

Subsequently, with reference to FIG. 11, a foreign matter detection process in the case where a foreign matter is detected in the monitoring area of the imaging device 101 will be described.
As shown in FIG. 11, first, in step S11, the imaging device 101 captures the line R and acquires a first image 51 and a second image 52. Subsequently, in step S12, the first generation unit 203 of the data management device 200 generates a distance image 53 from the parallax of the captured first image 51 and second image 52.

  Subsequently, in step S13, the search unit 208 determines whether a foreign object is present in the distance image 53. As an example, when there is anomalous parallax at the platform end or in the track, the search unit 208 determines that a foreign object is present.

  If there is a foreign object in the distance image 53, subsequently, in step S14, the search unit 208 sets the platform end of the distance image 53 as the rotation axis by the amount of rotation set in step S7 of FIG. The distance image 53 is rotated. Subsequently, in step S14, the search unit 208 compares the position where the foreign matter is present in the rotated distance image 54 with the fall detection area and the approach detection area set in step S8 of FIG. And it is determined in which of the proximity detection areas the foreign matter is present. Subsequently, in step S15, a predetermined warning is issued according to the area in which the foreign matter is present, and the foreign matter detection processing is ended.

[Effect in shooting system 10]
The embodiments of the present invention have been described above. Then, the effect in photography system 10 is explained.

In the imaging system 10, as a first effect, the time required for the installation work of the imaging device 101 can be shortened.
When photographing the platform end and the track obliquely, it is usually necessary to adjust the angle at which the photographing device 101 is attached and the angle of view with high accuracy in order to prevent false detection of the track fall. In this respect, the imaging system 10 obtains an image as if the platform edge was photographed from directly above, focusing on the fact that three-dimensional data (XYZ coordinates) of the subject can be calculated from the distance image 53 captured by the imaging device 101 The amount of rotation required for the image pickup apparatus 101 is set.

As a result, since it is possible to adjust the monitoring accuracy of the imaging device 101 by the rotation process after installation, it is not necessary to precisely adjust the attachment angle and the angle of view when the imaging device 101 is installed. Therefore, in the imaging system 10, the installation of the imaging device 101 does not need to take a long time, and the time required for the installation operation of the imaging device 101 can be shortened.
This first effect is more pronounced in the imaging system 10 monitoring across the platform. That is, although the imaging system 10 monitors the platform using a large number of imaging devices 101, the installation work time can be shortened in each of the imaging devices 101, so the installation time is greatly shortened in the entire imaging system 10. be able to.

In addition, since many inspection operations are performed on the platform, the imaging device 101 may be shifted due to some event even after the imaging device 101 is attached. In the conventional operation, when the imaging apparatus 101 is shifted, the installation of the imaging apparatus 101 has to be manually adjusted again. This readjustment also needs to be performed at night so as not to give to the operation of the railway vehicle, which not only imposes a heavy burden on the workers, but also can not properly monitor until the adjustment work is completed.
In this respect, in the imaging system 10, by performing setting processing for the imaging device 101 periodically or manually by the administrator, even if the imaging device 101 is shifted, an appropriate amount of rotation can be set again. . Thus, the number of adjustment operations can be shortened, and continuous monitoring can be performed, which is preferable.

Further, in the imaging system 10, as a second effect, improvement in the monitoring accuracy by the imaging device 101 can be expected.
When photographing the platform end at an angle, the upper body of the passenger present at the platform end may be reflected on the track, which may cause false detection as a track drop. In this regard, in the past, false detection was prevented by narrowing the search range in the height direction (Z direction) of the drop detection. However, in the imaging system 10, the platform P is photographed with the platform end photographed from directly above Since the line R is monitored, it is possible to prevent the erroneous detection of the line fall without adjusting the search range in the height direction.

  Here, with reference to FIG. 12, the conventional monitoring operation method in the imaging system 10 will be described. As shown in FIG. 12 (A), the passenger Ua exists in the track and the passenger Ub exists at the platform end. Considering the average height of the passengers, setting the range of 190 cm in height from the track surface as the search range makes it possible to search the entire body of the passenger Ua present in the track, while the platform in the range of 190 cm from the track surface The upper body of the passenger Ub present above is also included in the search range, which is not preferable. Therefore, in the conventional operation, a range of 130 cm in height from the track surface is set as the search range.

As shown in FIG. 12B, when the search range in the height direction is 130 cm from the R plane of the track, the search range is the foot portion of the passenger Ub on the platform. If the search range is up to the foot portion, the detection size is small, and the passenger Ub is not erroneously detected as a track drop.
On the other hand, when the search range in the height direction is 130 cm from the track surface, the distance search ends at the chest portion of the passenger Ua on the track. If it is cut at the chest part, the texture may be small and the passenger Ua may be lost, and the fall of the track may be missed.

On the other hand, as shown in FIG. 12C, when the search range in the height direction is up to 190 cm from the track surface, the whole body of the passenger Ua becomes the search range, so the passenger Ua on the track is stabilized. Can detect and never miss a track fall.
On the other hand, when the search range in the height direction is 190 cm from the track surface, the distance search is performed to the chest portion of the passenger Ub on the platform. If the chest portion of the passenger Ub on the platform is searched, the texture may be small but it may be detected, but it may be detected, and if it is detected, it leads to a false detection of the line fall .

  As described above, in the conventional monitoring method, the search range in the height direction is adjusted in consideration of erroneous detection and omission of detection of the line fall. In this respect, in the imaging system 10, since the platform end can be monitored in a state of being photographed from directly above, it is possible to accurately detect a danger such as a line fall without adjusting the search range.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It is apparent to those skilled in the art that various changes or modifications can be added to the above embodiment. It is also apparent from the scope of the claims that the embodiments added with such alterations or improvements can be included in the technical scope of the present invention.

In the above embodiment, the distance image 53 and an object included in the distance image 53 are rotated about an axis extending in the longitudinal direction (X direction) of the platform P, but the rotation direction is limited to this. Instead, it may rotate around an axis extending in a direction (Y direction) orthogonal to the longitudinal direction of the platform P, or may rotate around an axis extending in the height direction (Z direction).
Depending on the installation state of the imaging device 101, the length direction of the platform P may not coincide with the X direction of the distance image 53. Even in this case, the distance image 53 in the height direction (Z direction) By rotating around the extending axis, the longitudinal direction of the platform P and the X direction of the distance image 53 can be aligned. Therefore, the effect that the time which installation work requires can be shortened can be anticipated.

10: shooting system 100: shooting device group 101: shooting device 200: data management device (image processing device)
201: control unit 202: storage unit 203: first generation unit 204: calculation unit 205: rotation control unit 206: second generation unit 207: monitoring setting unit 208 · · Search unit 209 · · · identification unit 210 · · · detection unit 211 · · · acquisition unit 300 · · · hub 400 · · · monitor

Claims (5)

An image processing apparatus that processes an image captured by a stereo camera installed on a platform extending in a first direction, the image processing system comprising:
A first generation unit that generates a first distance image based on a parallax between a first image captured by a first imaging unit of the stereo camera and a second image captured by a second imaging unit;
A calculator configured to calculate three-dimensional data of an object in the first distance image based on the parallax;
A second generation unit configured to correct the first distance image based on the three-dimensional data and generate a corrected second distance image;
A rotation control unit that calculates three-dimensional data after rotation of the object by rotating the three-dimensional data about an arbitrary position extending in the first direction as an axis;
Equipped with
The second generation unit generates a second distance image based on the calculated three-dimensional data after rotation.
Image processing device. A detection unit that detects a crosstie of a track extending along the first direction from the first distance image to the side surface of the platform;
An acquisition unit configured to acquire an amount of rotation parallel to a second direction orthogonal to the first direction, with each of a plurality of sleepers constituting the track as the rotation occurs;
And further
The rotation control unit rotates the three-dimensional data by the acquired rotation amount.
The image processing apparatus according to claim 1 . The acquisition unit determines that each of the sleepers is parallel along a second direction based on coordinates of the first direction at a plurality of monitoring points different in the second direction of the detected one sleeper. judge,
The image processing apparatus according to claim 2 . The second distance image is divided into a first area and a second area bordering on the axis, an arbitrary area of the first area is set as a first monitoring area, and an arbitrary area of the second area is set. A monitoring setting unit configured to set the area as a second monitoring area;
A search unit that determines whether the foreign matter present in the second distance image is present in either the first monitoring area or the second monitoring area;
The image processing apparatus according to any one of claims 1 to 3 , further comprising: When the search unit detects foreign matter in the first distance image, the search unit rotates three-dimensional data of the foreign matter around the axis to calculate three-dimensional data of the foreign matter after rotation, and also after the rotation. Determining whether the foreign matter is present in the first monitoring area or the second monitoring area based on three-dimensional data of
The image processing apparatus according to claim 4 .

Images