JP6299317B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus.

  Since railroad vehicles used by a large number of passengers are required to be operated safely, the related parties of the railcars have conventionally made various attempts to operate the railroad vehicles safely. For example, if a passenger falls from a platform to a track, there is a risk of a serious accident. ing.

As a specific example, a camera system is well known in which a plurality of cameras are installed under the platform ceiling and a plurality of cameras monitor a station premises such as on a platform or a track.
As such a camera system, for example, Patent Document 1 discloses a camera system in which a plurality of cameras are connected in a daisy chain, and monitoring information acquired by each camera is transmitted to a server via another camera. ing.

JP 2012-11989

  In such a camera system, a parallax of a captured image of a stereo camera is usually used to detect a passenger or a foreign object dropped on a track. Therefore, in the conventional camera system, in order to detect passengers and foreign objects that have fallen on the track, each camera is adjusted so that the parallax is zero on the rail surface of the track, and is installed on the platform. By installing the camera so that the rail surface has zero parallax, when a foreign object falls on the track (rail), the parallax is generated where the foreign object is reflected. Foreign matter was detected.

  However, in order to install the camera so that the rail surface has zero parallax, it is necessary to adjust the shooting direction of each shooting unit with high accuracy, which takes a lot of time. In order to monitor the entire platform, it is necessary to install a large number of cameras. Adjustment of a large number of cameras requires an enormous amount of time, and it is necessary to perform adjustment work at night so as not to impair passenger convenience. Therefore, there is a need for technology that eliminates the need for adjustment work. .

  The present invention has been made in view of such problems, and an object of the present invention is to provide an image processing apparatus that eliminates the need for adjusting a camera for monitoring a platform.

  In order to solve the above-described problems, an image processing apparatus according to the present invention includes a first image captured by a first imaging unit of a stereo camera in which a plurality of imaging units are arranged along a first direction, and an adjacent pixel. The coordinates of at least the second direction orthogonal to the first direction are different from the extraction unit that extracts a plurality of pixel blocks composed of pixels having a luminance difference of a predetermined value or more and the plurality of pixel blocks extracted by the extraction unit. A selection unit that selects two or more pixel blocks, and an acquisition unit that acquires the parallax between the second image captured by the second imaging unit of the stereo camera for each of the two or more pixel blocks selected by the selection unit. And a generation unit that generates disparity information indicating a relationship between the parallax and the coordinates in the second direction based on the parallax acquired by the acquisition unit and the coordinates in the second direction of each of the two or more pixel blocks. When Equipped with a.

  In the above image processing apparatus, the stereo camera is a stereo camera that is installed on a platform extending along the first direction and photographs the periphery of the platform from a bird's-eye view, and is arranged on the side surface of the platform in the first direction. A detection unit that detects coordinates in the second direction indicating a position in the captured image of the rail of the track extending along the rail, and the selection unit is configured such that the coordinates in the second direction are more than coordinates in which the rail is positioned. At least one pixel block located in a direction away from the stereo camera may be selected.

  In the above image processing device, the detection unit may detect coordinates where the rail is located based on parallax in the plurality of pixel blocks.

  In the above-described image processing device, the detection unit detects a position of a sleeper of the track, and the selection unit is configured to detect the two or more pixel blocks from a plurality of the pixel blocks that are located at positions other than the position of the sleeper detected by the detection unit. The pixel block may be selected.

  In the above-described image processing device, the selection unit selects the two or more pixel blocks in each of a first region whose coordinates in the first direction are within a predetermined range and a second region outside the predetermined range, The generation unit may generate the parallax information in association with each of the first area and the second area.

  In the above image processing device, a search unit that searches for an abnormal pixel block in which a parallax at a predetermined coordinate between the first image and the second image differs by a predetermined amount or more with respect to the parallax of the coordinate specified by the parallax information; And an output unit that outputs warning information on condition that the search unit detects the abnormal pixel block.

  According to the present invention, it is possible to install a camera for monitoring the platform without performing adjustment work.

It is a figure which shows the structural example of the imaging | photography system which concerns on one Embodiment. It is a figure which shows the example of installation of the imaging device in a station platform. It is a figure which shows the example of installation of the imaging device in a station platform. It is a figure which shows an example of the various picked-up images image | photographed with the imaging device. It is a block diagram which shows the function structure of a data management apparatus. It is a figure which shows an example of the parallax information which shows the relationship between parallax and a Y coordinate. It is a figure which shows an example of the flow of the parallax information generation function of a data management apparatus. It is a figure which shows an example of the flow of the parallax information generation function of a data management apparatus. It is a figure which shows an example of the flow of the parallax information generation function of a data management apparatus. It is a figure which shows an example of the flow of the foreign material detection function of a data management apparatus. It is a flowchart which shows the flow of a process of an imaging | photography system. It is a flowchart which shows the flow of a process of an imaging | photography system. It is a figure which shows another example of the foreign material detection function of a data management apparatus.

[Overall Configuration of Shooting System 10]
FIG. 1 is a diagram illustrating an overall configuration of a photographing system 10 according to the present embodiment, and FIG. 2 is a diagram illustrating an installation example of a photographing apparatus 101 on a station platform. 2A is a diagram of the station platform as seen from the traveling direction of the railway vehicle, and FIG. 2B is a diagram of the station platform as viewed from a direction orthogonal to the traveling direction of the railway vehicle.

As shown in FIG. 1, the imaging system 10 includes an imaging device group 100 and a data management device 200 connected to the imaging device group 100 so as to be communicable.
Note that the imaging system 10 may include the hub 300 between the data management device 200 and the imaging device group 100, or may include a plurality of imaging device groups 100 connected to the hub 300. In this case, for example, each imaging device group 100 may be installed on each of a plurality of platforms at a station. The imaging system 10 may also include a monitor 400 that displays an image captured by the imaging device group 100.

  In the imaging device group 100, a plurality of imaging devices 101 (imaging device 101-1, imaging device 101-2... Imaging device 101-n, where n is a natural number of 3 or more) are connected in a daisy chain along a predetermined direction. Configured. As shown in FIG. 2B, in the present embodiment, the plurality of imaging devices 101 are daisy chain connected along the length direction of the platform P of the station.

  The photographing apparatus 101 is a stereo camera in which a plurality of photographing units are horizontally arranged along the length direction of the platform P of the station. As shown in FIG. 2A, the photographing device 101 is installed at a predetermined angle from the upper part (ceiling) of the platform P of the station toward the track R, and as shown in FIG. 2B, each photographing device 101 101 is set so that the shooting range S of 101 overlaps the shooting range S of the adjacent shooting device 101.

  The photographing apparatus 101 photographs the periphery of the platform P, that is, the platform P and the track R from a bird's-eye view. Here, FIG. 3 shows an example of the first image 51 and the second image 52 photographed by the photographing apparatus 101, and the distance image 53 generated from the first image 51 and the second image 52.

The first image 51 is a photographed image photographed by the first photographing unit of the photographing apparatus 101, and the second image 52 is a photographed image photographed by the second photographing unit of the photographing apparatus 101.
As shown in the first image 51 in FIG. 3, the photographing apparatus 101 photographs the platform P and the track R. In the present embodiment, the track R is a ballast track composed of a rail Ra, a plurality of sleepers Rt, and gravel Rb.

The distance image 53 is an image generated based on the parallax between the first image 51 and the second image 52.
Here, the parallax calculation method is already known, and the parallax is calculated by searching corresponding points corresponding to the reference point of the reference image (first image 51) from the comparison image (second image 52). Specifically, a pixel block is extracted from the reference image around the reference pixel, and a pixel block whose luminance pattern is similar to the pixel block extracted from the reference image among the pixel blocks in the search area of the comparison image. It specifies by well-known methods, such as SAD (Sum of Absolute Differences) method. Then, the parallax is calculated from the difference between the coordinates of the pixel block of the reference image and the coordinates of the pixel block of the comparison image. In the present embodiment, the distance image 53 is generated with the parallax set to zero for pixels in which the luminance difference between adjacent pixels of the plurality of pixels constituting the pixel block is not equal to or greater than a predetermined value.

  When the ballast trajectory is imaged by the imaging device 101 in which the first imaging unit and the second imaging unit are horizontally arranged along the length direction of the platform P, as shown in the distance image 53 of FIG. The parallax cannot be calculated for the rail Ra and the sleeper Rt, and the parallax can be calculated for the gravel Rb because the luminance pattern is characteristic.

In each image, the horizontal direction is the X direction (first direction), and the vertical direction is the Y direction (second direction). Since the plurality of imaging units of the imaging apparatus 101 are horizontally arranged along the length direction of the platform P of the station, the X direction corresponds to the length direction of the platform P. Further, the Y direction corresponds to the width direction of the platform P and the track R (distance from the photographing apparatus 101).
In the following, for convenience of explanation, the upper part of the image is Y coordinate “small” and the lower part of the screen is Y coordinate “large”. In other words, the Y coordinate is set to “small” as the distance from the image capturing apparatus 101 is increased, and the Y coordinate is set to “large” as it is closer.

  The first image 51 and the second image 52 photographed by the photographing device 101 and the distance image 53 generated from the first image 51 and the second image 52 are transmitted to the data management device 200 via a predetermined communication line. And stored in the storage unit 202 of the data management device 200. The distance image 53 may be generated by the data management apparatus 200 without being generated by the imaging apparatus 101.

[Functional Configuration of Data Management Device 200]
FIG. 4 is a block diagram illustrating a functional configuration of the data management apparatus 200. The data management apparatus 200 includes a control unit 201 and a storage unit 202.

  The storage unit 202 is configured by, for example, a ROM and a RAM. The storage unit 202 stores various programs for causing the data management apparatus 200 to function. The storage unit 202 is generated from the first image 51 taken by the first photographing unit of the stereo camera and the image data of the second image 52 taken by the second photographing unit, and the first image 51 and the second image 52. Image data of the distance image 53 to be stored.

  The control unit 201 is configured by, for example, a CPU, and functions as a parallax information generation function and a foreign object detection function by executing various programs stored in the storage unit 202.

[Parallax information generation function]
As shown in FIG. 2A, since the photographing apparatus 101 is installed on the platform P at a predetermined angle, the track R appears to be inclined when viewed from the photographing apparatus 101. Therefore, the distance from the imaging device 101 differs between the platform P side and the opposite side in the track R. In the following, the platform P side may be referred to as “front side” and the opposite side as “back side”.
As shown in FIG. 5, the parallax varies depending on the distance from the photographing apparatus 101, the parallax when photographing an object close to the photographing apparatus 101 is large, and the parallax when photographing a distant object is small. Even parallax is different.

The disparity information generation function generates disparity information (Yd function shown in FIG. 5) indicating the relationship between such disparity and distance.
In order to realize such a parallax information generation function, the control unit 201 includes an extraction unit 203, a selection unit 205, an acquisition unit 206, and a generation unit 207.

  The extraction unit 203 extracts, from the first image 51 photographed by the first photographing unit, a plurality of pixel blocks composed of pixels having a luminance difference with an adjacent pixel equal to or greater than a predetermined value (for example, “3”). For example, when a pixel block of 4 × 4 pixels is used, the extraction unit 203 selects a pixel block having a predetermined number or more (for example, 10 pixels) of pixels having a luminance difference from adjacent pixels out of 16 pixels. Extract multiple.

In order to generate the parallax information (Yd function) illustrated in FIG. 5, it is only necessary to have a plurality of parallaxes at positions with different Y coordinates. Here, when calculating the parallax, if the brightness of the pixels in the pixel block is single, there are many candidates in the search area of the comparison image that are similar to the brightness pattern of the reference pixel block. This is not suitable for calculating parallax. In this regard, a pixel block composed of pixels having a luminance difference between adjacent pixels that is greater than or equal to a predetermined value is a region having a characteristic luminance pattern, and is suitable for calculating parallax.
Therefore, the selection unit 205 selects at least two pixel blocks having different coordinates in the Y direction from the plurality of pixel blocks extracted by the extraction unit 203.

  The acquisition unit 206 acquires the parallax between the two or more pixel blocks selected by the selection unit 205 and the second image 52 captured by the second imaging unit of the imaging apparatus 101.

The generation unit 207 generates parallax information indicating the relationship between the parallax and the Y-direction coordinates based on the parallax acquired by the acquisition unit 206 and the Y-direction coordinates of each of the two or more pixel blocks. The generation unit 207 generates disparity information (the following expression (1)) based on the disparity of two or more pixel blocks and the coordinates in the Y direction using, for example, the least square method.
d (y) = αY + β (1)
Here, d (y) indicates parallax, Y indicates the Y coordinate of the pixel block, and α and β indicate parameters calculated by the least square method.

Here, in the distance image 53 schematically shown in FIG. 3, the parallax cannot be calculated for the platform P, the rail Ra, and the sleeper Rt, but the influence of noise cannot be completely removed. Partial parallax is also calculated for Ra and sleepers Rt.
Since it is not preferable to calculate parallax information (Yd function) using parallax calculated as such noise, the control unit 201 may further include a detection unit 204.

The detection unit 204 detects the positions of the platform P, the rail Ra, and the sleepers Rt. More specifically, the Y coordinate where the platform P and the rail Ra are located and the X coordinate where the sleeper Rt is located are detected.
The selection unit 205 selects two or more pixel blocks from positions other than the positions of the platform P, the rail Ra, and the sleepers Rt detected by the detection unit 204. At this time, the selection unit 205 selects parallax information (Y that is calculated by selecting at least one pixel block whose Y coordinate is located in a direction (back side) farther from the imaging device 101 than the coordinate where the rail Ra is located. -D function) can be improved in accuracy.

[Example of disparity information generation function]
Next, an example of a specific processing flow of the disparity information generation function will be described with reference to FIGS.
As shown in FIG. 6A, the first image 51 is composed of a plurality of pixel blocks 510. The pixel block 510 is a pixel block of 4 × 4 pixels, for example, as illustrated in FIG.

  The extraction unit 203 identifies a pixel having a luminance difference of at least a predetermined value (for example, 3 or more) from adjacent pixels among the 16 pixels included in the pixel block 510. For example, the upper left pixel in the pixel block 510 illustrated in FIG. 6B has a luminance difference of 4 (= 9-5) with an adjacent pixel, and the luminance difference with the adjacent pixel is a predetermined pixel or more. is there. On the other hand, the upper right pixel in the pixel block 510 has a luminance difference of 0 (= 8-8) with an adjacent pixel, and the luminance difference with the adjacent pixel is not a predetermined pixel or more.

The extraction unit 203 sets the parallax d for a pixel whose luminance difference with an adjacent pixel is greater than or equal to a predetermined value, and sets zero parallax for a pixel whose luminance difference with an adjacent pixel is less than a predetermined value. Note that the parallax d is the same in the pixels in the pixel block 510 and is calculated at a predetermined timing by a known method such as the SAD method. As described above, the extraction unit 203 fuses the luminance difference information between adjacent pixels and the parallax of the pixel block 510.
Further, the extraction unit 203 counts the number of pixels having a luminance difference with the adjacent pixels equal to or greater than a predetermined value, and extracts a plurality of pixel blocks with a count result equal to or greater than the predetermined number. In the present embodiment, the extraction unit 203 extracts pixel blocks in which 10 or more pixels out of 16 pixels have a luminance difference of a predetermined value or more.

As illustrated in FIG. 6C, the selection unit 205 generates disparity information (Yd function) for at least two or more pixel blocks 510 having different Y coordinates from the pixel block 510 extracted by the extraction unit 203. Select as point data for.
At this time, as shown in FIG. 6D, in order to prevent selecting the pixel block 510 in which the parallax is calculated as noise on the platform P, the rail Ra, and the sleeper Rt, the detection unit 204 includes the platform P, The positions of the rail Ra and the sleepers Rt are detected.

Specifically, as illustrated in FIG. 7E, the detection unit 204 scans the distance image 53 in the X direction, counts the number of pixels with zero parallax, and based on the number of pixels with zero parallax, The positions (Y coordinates) of the platform P and the rail Ra are detected.
FIG. 7F is a histogram showing the relationship between the Y coordinate and the number of pixels with zero parallax. When the ballast trajectory is imaged as in the present embodiment, the number of pixels with zero parallax increases at the position (Y coordinate) of the platform P and the rail Ra extending along the X direction of the captured image. On the other hand, since the parallax is calculated at the gravel Rb position (Y coordinate), the number of pixels with zero parallax is reduced.

The detection unit 204 detects a Y coordinate having the largest Y coordinate among the Y coordinates having a large number of pixels with zero parallax as the platform P, and detects the next largest Y coordinate on the rail Ra (in front of the two rails Ra ( Hereinafter, it is detected as “lower rail Rad”, and the next largest for the Y seat is detected as a rail Ra on the back side of the two rails Ra (hereinafter referred to as “upper rail Rau”).
Further, the detection unit 204 sets the position (Y coordinate) of the upper rail Rau as the coordinate R1, and sets the position (Y coordinate) of the lower rail Rad as the coordinate R2.

When the positions of the upper rail Rau and the lower rail Rad are detected, the detection unit 204 subsequently detects the position (X coordinate) of the sleeper Rt.
Specifically, the detection unit 204 divides the upper rail Rau and the lower rail Rad into a predetermined number. In the present embodiment, as shown in FIG. 7G, the upper rail Rau and the lower rail Rad are divided into three parts. Therefore, the detection unit 204 sets the coordinates M1 and M2 at positions where the space between the upper rail Rau and the lower rail Rad is divided into three.

Further, since the sleepers Rt are installed to extend to the near side and the far side with respect to the rail Ra, the detection unit 204 sets the coordinate M3 at a position on the near side with respect to the lower rail Rad, and further to the far side than the upper rail Rau. The coordinate M4 is set at the side position.
The detecting unit 204 sets “coordinate R2 + α1 of the lower rail Rad” as the coordinate M3, and sets “coordinate R1-α2 of the upper rail Rau” as the coordinate M4. At this time, α1 and α2 are values empirically set by experiments or the like.

Subsequently, the detection unit 204 scans the distance image 53 in the X direction for each of the coordinates M1 to M4, and counts the number of pixels with zero parallax for each pixel block 510 unit. FIG. 8H is a histogram showing the relationship between the X coordinate of the pixel block 510 and the number of pixels with zero parallax in the pixel block 510.
Since the parallax cannot be calculated in the sleeper Rt, the number of pixels with zero parallax increases in the pixel block 510 at the position (X coordinate) of the sleeper Rt. On the other hand, since the parallax is calculated for the gravel Rb, the number of pixels with zero parallax is reduced in the pixel block 510 at the position (X coordinate) of the gravel Rb. The detection unit 204 detects the position (X coordinate) of the sleeper Rt based on the number of pixels with zero parallax in the pixel block 510.

  When the detection unit 204 detects the positions of the rail Ra and the sleeper Rt, the selection unit 205 selects two or more pixel blocks from the plurality of pixel blocks 510 positioned other than the position of the rail Ra and the sleeper Rt. That is, the detection unit 204 selects two or more pixel blocks 510 located at the gravel Rb position. More specifically, the selection unit 205 identifies a portion where n or more pixel blocks 510 in which the number of pixels with zero parallax is less than the threshold value is continuous as the position of the gravel Rb, and determines an arbitrary pixel block from the portion. 510 is selected.

In the present embodiment, the selection unit 205 selects the pixel block 510 located at a position (X coordinate) other than the sleeper Rt for each of the coordinates M1 to M4 (Y coordinate) as shown in FIG. 8 (I). To do.
When the selection unit 205 selects a pixel block from each of the coordinates M1 to M4, the generation unit 207 generates disparity information (Y−) based on the disparity calculated for these pixel blocks and the Y coordinate of these pixel blocks. d function).

Note that the relationship between the parallax and the coordinates in the Y direction may differ between the central region of the captured image and other regions. Therefore, in the present embodiment, as shown in FIG. 8J, the captured image may be divided in the X direction, and disparity information may be generated for each of the divided areas.
When the captured image is divided into three as illustrated in FIG. 8J, the selection unit 205 selects two or more pixel blocks in each of the left region 61, the central region 62, and the right region 63 having different X-direction coordinates. The generation unit 207 generates disparity information (Yd function) in association with each of the left region 61, the central region 62, and the right region 63.

[Foreign matter detection function]
Returning to FIG. 4, the foreign object detection function detects a foreign object (for example, a foreign object dropped on the track R) from the image captured by the imaging apparatus 101 using the parallax information generated by the parallax information generation function.
In order to realize such a foreign object detection function, the control unit 201 includes a search unit 208 and an output unit 209.

The search unit 208 determines a pixel block in which the parallax at a predetermined coordinate between the first image 51 and the second image 52 is different from the parallax of the coordinate specified by the parallax information (Yd function) by a predetermined amount or more. Extract as That is, the search unit 208 searches the distance image 53 and detects an abnormal pixel block from a plurality of pixel blocks constituting the distance image 53.
The output unit 209 outputs warning information on condition that the search unit 208 detects an abnormal pixel block. The output unit 209 can output warning information by an arbitrary method. For example, the output unit 209 may output warning information by displaying predetermined information on the monitor 400, and a speaker (not shown). The warning information may be output by uttering a predetermined sound.

[Example of foreign object detection function]
Next, an example of a specific processing flow of the foreign object detection function will be described with reference to FIG.
As shown in FIG. 9A, the search unit 208 searches a rectangular search area 530 that substantially matches the width direction of the line R, and extracts abnormal pixel blocks. The rectangular search area 530 is an area that covers a predetermined amount (for example, equivalent to 20 cm in the real world) beyond the upper rail Rau from the end of the platform P on the track R side, as shown in FIG. It is composed of a plurality of pixel blocks 510 arranged along the direction.

Here, in FIG. 9, among the pixel blocks 510 included in the rectangular search area 530, the disparity calculated based on the disparity information for the pixel block 510 having the smallest Y coordinate is the disparity d1, and the pixel having the next smallest Y coordinate. The disparity calculated based on the disparity information for the block is referred to as disparity d2,..., And the disparity calculated based on the disparity information for the pixel block having the largest Y coordinate is referred to as disparity dn.
Referring to the parallax information (Yd function) shown in FIG. 5, since the parallax is smaller as the Y coordinate is smaller (back side) and the parallax is larger as the Y coordinate is larger (front side), the parallax d1 is the smallest parallax. Yes, the parallax dn is the largest parallax.

9 (Ca) and FIG. 9 (Da) show the line R viewed from the photographing apparatus 101. FIG. Since the photographing apparatus 101 is installed at an upper portion of the platform P with a predetermined angle, the track R appears to be inclined when viewed from the photographing apparatus 101 as shown in FIG.
9C and 9D show histograms showing the relationship between the parallax and the number of pixels in the rectangular search area 530. FIG. Note that since there are pixels set as zero parallax in the pixel block 510, the number of pixels with zero parallax increases.

  As shown in FIG. 9C-a, in the situation where no foreign matter exists on the track R, the distance from the imaging device 101 to the imaging target (track R) is uniform according to the installation angle of the imaging device 101. To change. Therefore, as shown in FIG. 9C-B, in the rectangular search area 530, the parallax from the parallax d1 to the parallax dn is calculated approximately equally.

  On the other hand, as shown in FIG. 9 (D-a), in the situation where the foreign object 70 exists on the track R, the distance to the object to be photographed (the track R, the foreign object 70) changes due to the presence of the foreign object 70. To do. Therefore, as shown in FIG. 9 (D-b), the distribution situation from the parallax d1 to the parallax dn is different from the situation where no foreign matter is present. That is, abnormal parallax is calculated in the pixel block 510 (abnormal pixel block) corresponding to the foreign object 70.

The search unit 208 extracts the parallax calculated in the rectangular search area 530 as parallax based on the foreign substance 70 as parallax based on the foreign matter 70, which has exceeded the threshold value except for zero parallax and counted the largest number of pixels. . The search unit 208 calculates a Y coordinate corresponding to the abnormal parallax dy from Expression (2) obtained by modifying Expression (1).
Y = (dy−β) / α (2)

  Note that the Y coordinate calculated by the equation (2) is a Y coordinate indicating the starting point of the foreign material 70. Therefore, as shown in FIG. 9E, the output unit 209 displays a region within a predetermined range (for example, 1 m) from the calculated Y coordinate (start point) as a strip rectangle 71, and indicates the presence of the foreign object 70 to the monitor 400. To display.

  The functional configuration of the data management apparatus 200 has been described above. Note that each of the plurality of imaging devices 101 may have a function of one of the extraction unit 203 to the output unit 209 included in the data management device 200. In other words, the image processing apparatus of the present invention may be realized by the data management apparatus 200, may be realized by each of the plurality of imaging apparatuses 101, or may be realized by the data management apparatus 200 and the plurality of data management apparatuses 200. It may be realized by each of the imaging devices 101.

[Processing of Imaging System 10]
Next, with reference to FIG. 10 and FIG. 11, a processing flow of the photographing system 10 of the present invention will be described.

FIG. 10 is a flowchart showing the flow of the disparity information generation process based on the disparity information generation function.
First, the imaging device 101 captures the track R, and acquires the first image 51, the second image 52, and the distance image 53 (step S1). Subsequently, the extraction unit 203 of the data management apparatus 200 extracts the pixel block 510 with parallax based on the luminance information (step S2). That is, the extraction unit 203 refers to the first image 51 and extracts a plurality of pixel blocks 510 configured by pixels having a luminance difference with an adjacent pixel equal to or greater than a predetermined value.

Subsequently, the detection unit 204 detects the position of the rail Ra (step S3). In this process, the detection unit 204 scans the distance image 53 in the X direction, counts the number of pixels with zero parallax, and detects the position (Y coordinate) of the rail Ra based on the number of pixels with zero parallax. .
Moreover, the detection part 204 detects the position of the sleeper Rt (step S4). In this process, the detection unit 204 detects pixel blocks located at the Y coordinate between the upper rail Rau and the lower rail Rad, the Y coordinate on the far side from the upper rail Rau, and the Y coordinate on the near side from the lower rail Rad. Scanning in the X direction, the position (X coordinate) of the sleeper Rt is detected based on the number of pixels with zero parallax in the pixel block 510 unit.

  Subsequently, the selection unit 205 selects, as point data, a pixel block 510 located in a portion other than the rail Ra and sleepers Rt in the track R, that is, gravel Rb (step S5). Subsequently, the generation unit 207 calculates the disparity information (Yd function) shown in the above formula (1) based on the disparity calculated in the pixel block 510 selected as the point data and the Y coordinate of the pixel block 510. Generate (step S6), and the process ends.

FIG. 11 is a flowchart showing the flow of foreign object detection processing based on the foreign object detection function.
First, the imaging device 101 captures the track R, and acquires the first image 51, the second image 52, and the distance image 53 (step S11). Subsequently, the search unit 208 of the data management device 200 searches for the track R region of the distance image 53 (step S12). That is, the search unit 208 searches for a rectangular search area 530 that substantially matches the width direction of the line R.

  Subsequently, the search unit 208 acquires the disparity distribution state (step S13). That is, the search unit 208 generates a histogram indicating the relationship between the parallax in the rectangular search area 530 and the number of pixels for which the parallax has been calculated. Subsequently, the search unit 208 determines whether there is a parallax in which a predetermined number of pixels or more are counted (step S14). When this determination is YES, the search unit 208 determines that there is a foreign object in the searched rectangular search area 530 (step S15), and moves the process to step S16. On the other hand, when this determination is NO, since there is no foreign object in the searched rectangular search area 530, the search unit 208 ends the process and moves to searching for the next rectangular search area 530.

  In step S16, the search unit 208 identifies the location (Y coordinate) of the foreign object based on the parallax information (Yd function). In addition, the output unit 209 displays an area within a predetermined range from the specified Y coordinate as a strip rectangle 71 and displays the presence of the foreign substance 70 on the monitor 400. When this processing is finished, the search unit 208 moves to search for the next rectangular search area 530.

[Effect in the photographing system 10]
The embodiment of the present invention has been described above. Next, effects in the imaging system 10 will be described.

In the photographing apparatus 101 installed at an upper portion of the platform P with a predetermined angle, the distance from the photographing apparatus 101 to the track R is different between the near side and the far side. For this reason, the parallax between the first image 51 and the second image 52 photographed by the photographing apparatus 101 is also different between the near side and the far side. In this regard, the imaging system 10 generates parallax information (Yd function) indicating the relationship between the distance (Y coordinate) from the imaging apparatus 101 and the parallax.
Thereby, since the foreign object can be easily detected using the generated parallax information, it is not necessary to adjust the rail surface so that the parallax is zero, and the photographing apparatus 101 can be easily installed.

At this time, the imaging system 10 selects point data for generating disparity information from the pixel block 510 configured by pixels having a luminance difference with an adjacent pixel equal to or greater than a predetermined value.
As a result, the pixel block 510 composed only of pixels having similar luminance is not selected as point data, and the pixel block 510 having a characteristic luminance pattern of each pixel can be selected as point data. Since the pixel block 510 having a characteristic luminance pattern is suitable for calculating parallax, it can generate appropriate parallax information, which is preferable.

  In addition, the imaging system 10 detects the positions of the upper rail Rau and the lower rail Rad in the track R, and selects at least one pixel block 510 located on the back side of the upper rail Rau. Thereby, the accuracy of the calculated parallax information can be improved, which is preferable.

At this time, the imaging system 10 detects the coordinates where the rail Ra is located based on the parallax in the plurality of pixel blocks 510. That is, in the imaging system 10, the Y coordinate having a large number of pixels with zero parallax is detected as the coordinate where the rail Ra is located. Moreover, in the imaging | photography system 10, the position of sleeper Rt is detected and the pixel block 510 located in positions other than rail Ra and sleeper Rt is selected as point data.
Thereby, it can prevent generating parallax information based on the parallax computed as noise on rail Ra and sleeper Rt, and can raise the accuracy of parallax information to compute.

  Moreover, in the imaging system 10, since the parallax information is calculated according to the area of the captured image (the left area 61, the center area 62, and the right area 63), the accuracy of the calculated parallax information can be improved, which is preferable.

  Further, in the imaging system 10, when there is an abnormal pixel block in which an abnormal parallax is calculated with respect to the parallax information in the distance image 53, warning information is output, and thus the foreign object 70 that has fallen on the track R can be easily removed. And can contribute to the safe operation of railway vehicles.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above embodiment. It is apparent from the description of the scope of claims that embodiments with such changes or improvements can be included in the technical scope of the present invention.

  For example, in the above-described embodiment, the foreign object detection function detects the foreign object 70 from the relationship between the parallax and the number of pixels for which the parallax is calculated (see FIG. 9D). In this regard, as shown in FIG. 12, the parallax d and Y coordinate of the pixel block 510 included in the rectangular search area 530 and the parallax d at the Y coordinate calculated based on the parallax information (Yd) function. Alternatively, the foreign object 70 may be detected. That is, the pixel block in which the parallax d calculated for the pixel block 510 exceeds the parallax (d (y)) calculated from the parallax information (Yd function) with respect to the Y coordinate of the pixel block 510 by a predetermined amount or more. The foreign object 70 may be detected by extracting 510 as an abnormal pixel block.

DESCRIPTION OF SYMBOLS 10 ... Shooting system, 100 ... Shooting device group, 101 ... Shooting device, 200 ... Data management device (image processing device), 201 ... Control unit, 202 ... Storage unit, 203 ... Extraction unit, 204 ... Detection unit, 205 ... Selection unit, 206 ... Acquisition unit, 207 ... Generation unit, 208 ... Search unit, 209 ... Output unit, 300 ..Hub, 400 ... monitor

Claims (6)

A plurality of pixel blocks configured by pixels having a luminance difference with a neighboring pixel or more from a first image captured by a first imaging unit of a stereo camera in which a plurality of imaging units are arranged along a first direction. An extractor for extracting;
A selection unit that selects at least two pixel blocks having different coordinates in a second direction orthogonal to the first direction from the plurality of pixel blocks extracted by the extraction unit;
An acquisition unit that acquires a parallax with the second image captured by the second imaging unit of the stereo camera for each of the two or more pixel blocks selected by the selection unit;
A generating unit that generates disparity information indicating a relationship between the parallax and the coordinate in the second direction based on the parallax acquired by the acquisition unit and the coordinate in the second direction of each of the two or more pixel blocks;
An image processing apparatus comprising: The stereo camera is a stereo camera that is installed on a platform extending along the first direction and photographs the periphery of the platform from a bird's-eye view,
A detector for detecting coordinates in the second direction indicating a position in a captured image of a rail of a rail extending along the first direction on a side surface of the platform;
The selection unit selects at least one pixel block in which the coordinate in the second direction is located in a direction away from the stereo camera than the coordinate in which the rail is located;
The image processing apparatus according to claim 1. The detection unit detects coordinates where the rail is located based on parallax in the plurality of pixel blocks.
The image processing apparatus according to claim 2. The detection unit detects the position of sleepers on the track,
The selection unit selects the two or more pixel blocks from a plurality of the pixel blocks located other than the position of the sleepers detected by the detection unit;
The image processing apparatus according to claim 2. The selection unit selects the two or more pixel blocks in each of a first region whose coordinates in the first direction are within a predetermined range and a second region outside the predetermined range;
The generation unit generates the disparity information in association with each of the first region and the second region.
The image processing apparatus according to claim 1. A search unit that searches for an abnormal pixel block in which a parallax at a predetermined coordinate between the first image and the second image differs by a predetermined amount or more with respect to the parallax of the coordinate specified by the parallax information;
An output unit that outputs warning information on condition that the search unit detects the abnormal pixel block;
The image processing apparatus according to claim 1, further comprising: