JP7058164B2 - Traffic route judgment system - Google Patents

Description

The present invention relates to a passage determination system for determining the state of a passage.

Conventionally, it has been proposed to identify the vibration of a bridge by analyzing the image of the bridge and to grasp the progress of deterioration of the bridge (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2017-20739

In the method using an image as shown in Patent Document 1, it is necessary to install a camera for taking an image of the bridge near the bridge. Therefore, it takes time and effort for a person to go to the site to install the camera.

The present invention has been made in view of the above, and an object of the present invention is to provide a passage determination system that can easily determine the state of a passage such as a bridge.

In order to achieve the above object, the passage determination system according to the present invention acquires a moving image captured from a moving vehicle including the passage, or a moving image captured from a vehicle operating on the passage. According to the vibration of the passage from the acquisition unit, the vehicle vibration detection unit that detects the vibration of the vehicle during operation, the motion image acquired by the motion image acquisition unit, and the vibration of the vehicle detected by the vehicle vibration detection unit. It is equipped with a passage vibration calculation unit that calculates the information and a determination unit that determines the state of the passage from the information corresponding to the vibration of the passage calculated by the passage vibration calculation unit, and the vibration is time-series. The passage vibration calculation unit calculates the video vibration including the vibration of the passage and the vibration of the vehicle from the moving image acquired by the video acquisition unit, and the calculated video vibration and the vehicle. From the vibration of the vehicle detected by the vibration detection unit, information corresponding to the vibration of the passage is calculated .

In the passage determination system according to the present invention, a moving image captured from a vehicle is used to determine the state of the passage. Therefore, it is not necessary to fixedly install a camera for taking an image of the passage near the passage, and it is possible to determine the state of the passage. Therefore, according to the passage determination system according to the present invention, it is possible to easily determine the state of a passage such as a bridge.

According to the present invention, it is not necessary to fixedly install a camera for taking an image of the passage near the passage, the state of the passage can be determined, and the state of the passage such as a bridge can be easily determined. Can be determined.

It is a figure which shows the structure of the passage determination system which concerns on embodiment of this invention. It is a figure which shows typically the example of the vibration of a vehicle and a bridge which is a passage, and the image pickup from the camera mounted on the vehicle. It is a figure which shows the calculation of the vibration data schematically. It is a figure which shows typically the example of the moving image which copied the passage of a determination target, and the division into an element. It is an example of the graph of the vibration data of the passage. This is an example of a graph obtained by Fourier transforming vibration data. It is a flowchart which shows the process executed by the passage determination system which concerns on embodiment of this invention. It is a figure which shows the example of the moving image at the time of traveling of a vehicle. This is an example of a graph of vibration (displacement) data calculated from a moving image of a vehicle while traveling. This is an example of a graph obtained by Fourier transforming the vibration data when the vehicle is running. It is a figure which shows another example schematically such as the vibration of a vehicle and a bridge which is a passage, and the image pickup from the camera mounted on the vehicle. This is an example of a graph obtained by Fourier transforming vibration data. It is a figure which shows the hardware configuration of the server included in the passage determination system which concerns on embodiment of this invention.

Hereinafter, embodiments of the passage determination system according to the present invention will be described in detail together with the drawings. In the description of the drawings, the same elements are designated by the same reference numerals, and duplicate description will be omitted.

FIG. 1 shows a passage determination system 1 according to the present embodiment. As shown in FIG. 1, the passage determination system 1 includes a camera 10, a vibration sensor 20, and a server 30.

The passage determination system 1 is a system for determining the state of the passage. The passage to be determined is a bridge or a road through which a vehicle such as an automobile passes. The state of the passage to be determined is, for example, whether or not an abnormality has occurred in the passage (for example, deterioration has occurred). Alternatively, the state of the passage to be determined is a deterioration state (deterioration degree) of the passage, a sound condition (health degree) of the passage, or the like. The result of the determination by the passage determination system 1 is used, for example, for determination of reinforcement or repair of the passage. In the passage determination system 1, the server 30 determines the state of the passage using the moving image captured by the camera 10 and the vibration detected by the vibration sensor 20.

Subsequently, the configurations of the camera 10, the vibration sensor 20, and the server 30 included in the passage determination system 1 will be described. The camera 10 is an imaging device that acquires a moving image of the passage by taking an image including the passage to be determined. As the camera 10, a well-known camera capable of taking an image at a resolution that can be used for processing by the server 30 described later can be used. The camera 10 is mounted on a vehicle 40 such as an automobile, and takes an image from the vehicle 40 including a passage. As the camera 10, a drive recorder mounted on the vehicle 40 can be usually used. The camera 10 and the server 30 can send and receive information to and from each other. When the vehicle 40 is in operation, the camera 10 takes an image including a passage at a preset frame rate (fps), and transmits the captured moving image to the server 30. The operation of the vehicle 40 includes the running time and the non-running time of the vehicle 40 (for example, the engine is running but stopped at a traffic light or the like).

The vibration sensor 20 is provided on the vehicle 40 and is a sensor that detects the vibration of the vehicle 40 when the vehicle 40 is in operation. As the vibration sensor 20, a well-known vibration sensor can be used. The vibration sensor 20 detects vibration (displacement) at each time when the vehicle 40 is in operation. The vibration sensor 20 may detect vibration only in the vertical direction. The vibration sensor 20 and the server 30 can send and receive information to and from each other. The vibration sensor 20 transmits the detected vibration data of the vehicle 40 to the server 30.

The image pickup by the camera 10 and the vibration detection by the vibration sensor 20 are performed at the same timing. The time on the moving image obtained by the image pickup of the camera 10 and the time on the vibration data obtained by the detection of the vibration sensor 20 are synchronized in the server 30. For example, time information is included in each data.

Here, the concept of determining the state of the passage in the present embodiment is shown. The determination of the state of the passage in the present embodiment is performed based on the vibration of the passage. For example, as shown in FIG. 2, the camera 10 mounted on the vehicle 40 takes an image including the bridge B, which is the passage to be determined, and acquires a moving image. The vibration of the bridge B portion (movie vibration) specified from the moving image includes the vibration of the vehicle 40 (vehicle vibration) in addition to the vibration of the bridge B itself (bridge vibration). Therefore, as shown in FIG. 3, the bridge vibration y ₁ (t) -α y ₂ (t) is calculated by subtracting the vehicle vibration y ₂ (t) (multiplied by the coefficient α) from the moving image vibration y ₁ (t). do. From the calculated bridge vibration y ₁ (t) -α y ₂ (t), the state of the bridge B is determined.

As shown in FIG. 1, the server 30 functionally includes a moving image acquisition unit 31, a vehicle vibration detection unit 32, a passage vibration calculation unit 33, a determination unit 34, and an output unit 35. It is composed.

The moving image acquisition unit 31 is a functional unit that acquires a moving image captured from the operating vehicle 40 including the passageway. The moving image acquisition unit 31 receives the moving image transmitted from the camera 10 and acquires the moving image. The moving image acquisition unit 31 outputs the acquired moving image to the passage vibration calculation unit 33. The moving image acquired by the moving image acquisition unit 31 may be a case where the vehicle 40 is operating on the passage target to be determined, or a case where the vehicle 40 is operating on a road other than the passage target to be determined. ..

The vehicle vibration detection unit 32 is a functional unit that detects the vibration of the vehicle 40 during operation. The vehicle vibration detection unit 32 detects the vibration of the vehicle 40 based on the sensor provided in the vehicle 40. The vehicle vibration detection unit 32 receives the vibration data of the vehicle 40 from the vibration sensor 20 and detects the vibration of the vehicle 40. The vibration corresponds to the vehicle vibration y ₂ (t) shown in FIG. When the vehicle 40 is operating on the passage to be determined, the vibration includes the vibration of the passage (where the vehicle is located) (this vibration also corresponds to the vibration of the vehicle 40 during operation). ). The vehicle vibration detection unit 32 outputs the vibration data of the vehicle 40 to the passage vibration calculation unit 33.

The passage vibration calculation unit 33 is a functional unit that calculates information according to the vibration of the passage from the motion image acquired by the motion image acquisition unit 31 and the vibration of the vehicle 40 detected by the vehicle vibration detection unit 32. be. The passage vibration calculation unit 33 calculates vibration from the portion of the moving image acquired by the moving image acquisition unit 31 in which the passage is shown, and the vibration of the vehicle 40 detected by the vehicle vibration detecting unit 32 from the calculated vibration. Is subtracted to calculate the vibration of the passage. Specifically, the passage vibration calculation unit 33 calculates information according to the vibration of the passage as follows.

The passage vibration calculation unit 33 inputs a moving image from the moving image acquisition unit 31 and vibration data of the vehicle 40 from the vehicle vibration detecting unit 32. The passage vibration calculation unit 33 calculates the vibration from the portion of the input moving image in which the passage is shown. When the vehicle 40 is not traveling (when the vehicle speed is 0), the passage vibration calculation unit 33 calculates the vibration as follows, for example.

The passage vibration calculation unit 33 detects the displacement in sub-pixel units from the moving image by a conventional technique such as POC (Phase Only Correlation), and detects the vibration of the passage.

The passage vibration calculation unit 33 divides the acquired moving image as shown in FIG. 4 into individual frames (images) constituting the moving image. The passage vibration calculation unit 33 extracts (divides into partial images) partial images (phase images) of a plurality of regions at preset positions from individual frames. In this embodiment, each region is referred to as an element E. The size of the element E is preset. The position of the element E is the position where the passage is shown in the moving image. The position of the element E is set in advance by the user of the passage determination system 1, for example, and is stored in the passage vibration calculation unit 33. Alternatively, the passage vibration calculation unit 33 may detect the position where the passage is shown in the moving image and set the detected position as the position of the element E.

The passage vibration calculation unit 33 calculates the positional deviation of the same (positional) element E between the front and rear frames with sub-pixel accuracy. The time difference Δt between the preceding and following frames is the reciprocal of the frame rate (fps) (Δt = 1 / fps). The passage vibration calculation unit 33 calculates the positional deviation in the y direction of the moving image, for example, when determining the state of the bridge. This is because the deterioration of the bridge proceeds in response to vibration in the vertical direction (vertical direction). The passage vibration calculation unit 33 calculates the positional deviation (displacement y) between the front and rear frames for each element as vibration from the moving image. The vibration data obtained from the moving image by the passage vibration calculation unit 33 is the data of the element i and the displacement y for each time (time) t as shown in FIG. In the above example, vibrations at a plurality of points (elements) of the passage are detected, but the vibrations at a plurality of points do not have to be detected, and vibrations at one place may be detected.

The vibration calculated as described above corresponds to the moving motion vibration _y1 (t) shown in FIG. That is, the vibration calculated as described above includes both the vibration of the passage (the part shown in the moving image) and the vibration of the vehicle 40. Further, when the vehicle 40 is operating on the passage path to be determined, the calculated vibration includes the vibration of the passage path (where it is located).

The passage vibration calculation unit 33 is based on the calculated moving image vibration y ₁ (t) and the vehicle vibration y ₂ (t) indicated by the data input from the vehicle vibration detection unit 32, as shown in FIG. Vibration y ₁ (t) -αy ₂ (t) is calculated. Here, the coefficient α is for adjusting the amplitude (difference in the unit of each vibration, etc.) between the moving image vibration y ₁ (t) and the vehicle vibration y ₂ (t), and is, for example, by the least squares method. It is calculated. Specifically, the coefficient α may be a value that minimizes ∫ (y ₁ (t) −αy ₂ (t)) ² dt. When the vibration y ₁ (t) of a plurality of points of the passage is calculated from the moving image, the vibration y ₁ (t) −αy ₂ (t) of the passage may be calculated for the plurality of points.

The passage vibration calculation unit 33 Fourier transforms the data of the vibration y ₁ (t) −αy ₂ (t) (data of the displacement y for each time (time) t). By the Fourier transform, the vibration data becomes the intensity (power) data for each frequency (Hz) as shown in FIG. A high-pass filter may be applied to the frequency data after the Fourier transform to remove low frequency components of 0 to several Hz. The passage vibration calculation unit 33 calculates (detects) the natural frequency (predominant frequency) from the vibration data obtained by the Fourier transform. The natural frequency is, for example, the lowest frequency among the frequencies at which the intensity reaches the maximum value. It is generally said that when a bridge deteriorates, its natural frequency decreases. The passage vibration calculation unit 33 outputs the calculated natural frequency to the determination unit 34 as information corresponding to the vibration of the passage.

The determination unit 34 is a functional unit that determines the state of the passage from the information corresponding to the vibration of the passage calculated by the passage vibration calculation unit 33. The determination unit 34 inputs the natural frequency as information according to the vibration of the passage from the passage vibration calculation unit 33. The determination unit 34 determines the state by comparing the natural frequencies at different timings, for example. Specifically, the determination unit 34 detects a change in the input natural frequency from the standard value, and detects deterioration of the passage due to the change. As the standard value of the natural frequency, the natural frequency at the time of construction of the passage or the natural frequency at the time of soundness (when the passage is confirmed to be sound by inspection or the like) is used. The standard value of the natural frequency is set and stored in the determination unit 34 in advance.

The determination unit 34 calculates fn / fs as the soundness using the input natural frequency (measured value of the natural frequency) fn and the standard value fs of the natural frequency. The higher the value, the healthier the passage is (not deteriorated). The determination unit 34 may indicate the determination result of the passage route itself with the calculated soundness itself. Alternatively, the determination unit 34 may compare the soundness with a preset threshold value and determine that the passage has deteriorated when the soundness is equal to or less than the threshold value. When the vibration of the passage is detected for each element, it may be determined for each element (location).

Further, the natural frequency may be calculated at a plurality of timings (from the moving images captured at a plurality of timings) in a fixed time width, and the change in the distribution thereof may be detected. For example, the average value of the natural frequency may be calculated, and the soundness may be calculated using the average value. The determination unit 34 outputs information indicating the determination result, for example, information indicating whether or not the soundness or the passage has deteriorated to the output unit 35.

The output unit 35 is a functional unit that outputs information indicating a determination result of the state of the passage by the determination unit 34. The output unit 35 transmits, for example, the information to a terminal connected to the server 30 and displays the information. The output by the output unit 35 is referred to by, for example, the user of the passage determination system 1, and is used for determination of reinforcement or repair of the passage as described above. The output by the output unit 35 may be performed on a device other than the above, or may be performed in a form other than the above. The above is the configuration of the passage determination system 1.

Subsequently, using the flowchart of FIG. 7, a process executed by the passage determination system 1 according to the present embodiment (operation method performed by the passage determination system 1) will be described. First, the camera 10 mounted on the operating vehicle 40 captures the passage path to be determined and acquires a moving image (S01). The captured moving image is transmitted from the camera 10 to the server 30. In the server 30, the moving image is received and acquired by the moving image acquisition unit 31. Further, at the same timing as the above-mentioned imaging, the vibration sensor 20 mounted on the vehicle 40 detects the vibration of the vehicle 40 during operation (S02). The detected vibration data of the vehicle 40 is transmitted from the vibration sensor 20 to the server 30. In the server 30, the vibration data of the vehicle 40 is received and acquired by the vehicle vibration detection unit 32.

Subsequently, the passage vibration calculation unit 33 calculates the moving image vibration y ₁ (t) from the portion of the moving image showing the passage, and subtracts the vehicle vibration y ₂ (t) from the calculated vibration to pass the passage. Road vibration y ₁ (t) -α y ₂ (t) is calculated (detected) (S03). Subsequently, the passage vibration calculation unit 33 calculates the natural frequency of the passage from the calculated vibration y ₁ (t) −αy ₂ (t) of the passage (S04). Subsequently, the determination unit 34 determines the state of the passage from the natural frequency (S05). Subsequently, the output unit 35 outputs information indicating the determination result of the state of the passage by the determination unit 34 (S06). The above is the process executed by the passage determination system 1 according to the present embodiment.

In the above-described embodiment, the vehicle vibration detection unit 32 receives the vibration data of the vehicle 40 from the vibration sensor 20 and detects the vibration of the vehicle 40. However, the vehicle vibration detection unit 32 detects the vibration of the vehicle 40 as follows. May be good. That is, the vehicle vibration detection unit 32 may calculate the vibration of the vehicle 40 from the portion other than the passage path of the moving image acquired by the moving image acquisition unit 31, that is, the non-animal portion. When an image is taken by the camera 10 mounted on the moving (vibrating) vehicle 40, the camera 10 vibrates in accordance with the vehicle 40, so that the non-animal portion in the moving image vibrates. Therefore, the vibration of the non-animal portion of the moving image coincides with the vibration of the vehicle 40.

In this case, the vehicle vibration detection unit 32 inputs a moving image from the moving image acquisition unit 31. The calculation of the vibration from the portion other than the passage in the moving image can be performed in the same manner as the calculation of the vibration from the moving image by the passage vibration calculation unit 33. For example, the position of the portion other than the passage path is set in advance by the user of the passage path determination system 1 and stored in the vehicle vibration detection unit 32. Alternatively, the vehicle vibration detection unit 32 may detect a position where the passage is not shown in the moving image. The vibration calculated from the moving image can be used for processing in the vehicle vibration detection unit 32 in the same manner as the vibration detected by the vibration sensor 20. When this configuration is adopted, the vibration sensor 20 may not be included in the passage determination system 1. That is, the vibration sensor 20 may not be mounted on the vehicle 40.

In the above-described embodiment, the calculation of the vibration from the moving image when the vehicle 40 is not running has been described. When the vehicle 40 is traveling (moving), the passage vibration calculation unit 33 calculates vibration as follows, for example, from the portion of the input moving image in which the passage is shown. The vehicle 40 may be traveling on the passageway (during passage) or other than the passageway (for example, before entering the passageway). The passage vibration calculation unit 33 sets a predetermined point of the passage in the moving image as a feature point. The points to be feature points are designated in advance by, for example, the user of the passage determination system 1. Alternatively, the passage vibration calculation unit 33 may detect a position where the passage is shown in the moving image and set the detected position as a feature point.

As shown in FIG. 8, the passage vibration calculation unit 33 detects the coordinates on the frame of the feature points for each frame constituting the moving image. The coordinates of the feature points can be detected by a conventional method, for example, a method using a SIFT (Scale-Invariant Feature Transform) feature quantity or a SURF (Speeded Up Robust Features) feature quantity. The passage vibration calculation unit 33 calculates the displacement of the coordinates of each frame from the average of the coordinates of the feature points of each frame as vibration. The displacement may be calculated for the vertical method of the passage. For example, the displacement of the moving image in the y direction is calculated. Alternatively, the vertical method of the passage in the moving image may be detected and the displacement in that direction may be calculated.

The vibration calculated as described above corresponds to the moving motion vibration _y1 (t) shown in FIG. That is, the vibration calculated as described above includes both the vibration of the passage (the part shown in the moving image) and the vibration of the vehicle 40. Further, in this case, the calculated vibration (displacement) includes the displacement according to the traveling of the vehicle. FIG. 9 shows an example of the displacement for each time (time) calculated in this case. Since the displacement according to the traveling of the vehicle 40 is included as described above, the displacement according to the time is large. Depending on the moving direction of the vehicle 40 and the like, the displacement may become smaller or may not change with time.

Even in this case, the natural frequency can be calculated by the same processing as described above. FIG. 10A shows a graph in which the moving image vibration y ₁ (t) in this case is Fourier transformed into a frequency component. In this graph, the intensity of low frequency components below 5 Hz is high. FIG. 10 (b) shows an enlarged graph of the intensity of the graph of FIG. 10 (a). FIG. 10 (c) shows a graph (the unit is the same as the graph of FIG. 10 (b)) in which the vehicle vibration y ₂ (t) is Fourier transformed into a frequency component. The vehicle vibration y ₂ (t) is calculated from the non-animal portion in the moving image. FIG. 10 (d) shows a graph in which the calculated vibration of the passage y ₁ (t) -αy ₂ (t) is Fourier transformed into a frequency component (the unit is the same as the graph in FIG. 10 (b)). show.

As shown in these graphs, it is considered that the frequency of the passage (in this case, the bridge) is 10 Hz, and the frequency of the vehicle 40 is 20 Hz. It is considered that the low frequency component having a frequency of less than 5 Hz is due to the displacement of the vehicle 40 due to the movement (movement of the feature point). Therefore, by applying a high-pass filter to remove the low frequency component of 0 to several Hz, the component related to the displacement due to the movement of the vehicle 40 (movement of the feature point) can be removed, and the natural frequency can be calculated. can. It should be noted that the calculation of the vibration from the moving image when the vehicle 40 is running or not running may be performed by both or only one of them.

Further, the passage vibration calculation unit 33 may determine whether or not the vehicle 40 is traveling when the moving image is captured. The passage vibration calculation unit 33 calculates the vibration from the above-mentioned moving image when the vehicle 40 is not running for the moving image captured at the timing when the vehicle 40 is not running, and calculates the vibration from the moving image when the vehicle 40 is not running. For the moving image captured at the timing when the vehicle is running, the vibration is calculated from the moving image when the vehicle 40 is not running.

For example, the passage vibration calculation unit 33 acquires information indicating the speed of the vehicle 40, and determines whether or not the vehicle 40 is traveling when the moving image is captured, based on the speed of the vehicle 40. .. Specifically, if the speed of the vehicle 40 is 0, it is determined that the vehicle is not traveling, and if the speed of the vehicle 40 is not 0, it is determined that the vehicle is traveling. The passage vibration calculation unit 33 acquires information indicating the speed of the vehicle 40, for example, as follows. The vehicle 40 continuously transmits information indicating the speed measured by the speedometer provided in the vehicle 40 to the server 30, and the passage vibration calculation unit 33 receives the information. Alternatively, the vehicle 40 continuously transmits information indicating the position of the vehicle 40 measured by the GPS (Global Positioning System) of the drive recorder provided in the vehicle 40 to the server 30, and the passage vibration calculation unit 33 , The information is received, and the speed of the vehicle 40 is calculated from the change in the position of the vehicle 40 indicated by the information.

It should be noted that whether the vibration is calculated from the moving image when the vehicle 40 is running or not is automatically determined by the passage vibration calculation unit 33 as described above. It may not be necessary, for example, it may be specified by the user of the passage determination system 1.

In the above-described embodiment, the moving image captured by the camera 10 is an image captured including the passage to be determined. In addition, vibration including vibration of the passage was detected from the portion of the moving image in which the passage was shown, and information corresponding to the vibration of the passage was calculated. However, unlike the above, the information corresponding to the vibration of the passage may be calculated as follows.

For example, as shown in FIG. 11, the camera 10 mounted on the vehicle 40 (operating on the bridge B) located on the bridge B, which is the passage to be determined, captures the non-animal portion other than the bridge B, that is, the non-animal portion. Take an image and acquire a moving image. The vibration specified from the moving image includes the vibration of the vehicle 40 (vehicle vibration) in addition to the vibration of the bridge B itself (bridge vibration). Further, the camera 10 mounted on the vehicle 40 located on a vehicle other than the bridge B, which is the passage to be determined (operating on a vehicle other than the bridge B), captures a moving image of the vehicle other than the bridge B, that is, the non-animal portion. To get. The vibration specified from the moving image includes the vibration of the vehicle 40 (vehicle vibration). From these vibrations, information corresponding to the vibration of the bridge B, which is a passageway, is calculated. When this configuration is adopted, the vibration sensor 20 may not be included in the passage determination system 1.

In this case, the moving image acquisition unit 31 acquires the moving images in each of the above cases from the camera 10. The moving image acquisition unit 31 inputs the moving image captured from the vehicle 40 operating on the traffic road to the passage vibration calculation unit 33, and the moving image captured from the vehicle 40 operating on a non-traffic road is input to the vehicle vibration detecting unit 32. input. It should be noted that each moving image was taken at different timings.

The vehicle vibration detection unit 32 calculates the vibration of the vehicle 40 from the moving image acquired by the moving image acquisition unit 31 from the vehicle 40 operating on a road other than the road. The calculation of the vibration from the moving image can be performed in the same manner as the calculation of the vibration from the moving image by the passage vibration calculation unit 33 described above. The position where the vibration in the moving image is calculated is set in advance by the user of the passage determination system 1, for example, and is stored in the vehicle vibration detection unit 32. Alternatively, any position on the moving image may be set as the position for calculating the vibration. The vehicle vibration detection unit 32 outputs the calculated vibration data of the vehicle 40 to the passage vibration calculation unit 33. The vibration of the vehicle 40 may be detected by the vibration sensor 20 provided in the vehicle 40 when the vehicle 40 is operating other than on the road.

The passageway vibration calculation unit 33 calculates the frequency component related to the vibration of the vehicle 40 from the vibration of the vehicle 40 detected by the vehicle vibration detection unit 32. Further, the passage vibration calculation unit 33 calculates the vibration from the moving image captured from the vehicle 40 operating on the passage acquired by the moving image acquisition unit 31, and calculates and calculates the frequency component related to the calculated vibration. Information according to the vibration of the passage is calculated from each of the frequency components.

The calculation of the vibration from the moving image captured on the passage by the passage vibration calculation unit 33 can be performed in the same manner as the calculation of the vibration from the moving image described above. The vibration calculated by the passageway vibration calculation unit 33 and the vibration calculated by the vehicle vibration detection unit 32 are vibrations at different timings. Therefore, as shown in FIG. 3, the vibration cannot be subtracted. Therefore, the passage vibration calculation unit 33 Fourier transforms each vibration data to obtain a frequency component, and calculates information corresponding to the vibration of the passage.

FIG. 12 (a) shows an example of the frequency component obtained by Fourier transforming the vibration data obtained from the moving image captured on the traffic path, and FIG. 12 (b) captures an example of the frequency component other than on the traffic path. Examples of frequency components obtained by Fourier transforming the vibration data obtained from the moving image are shown. The passage vibration calculation unit 33 identifies a frequency at which the intensity becomes a maximum value (peak) from the frequency component related to the moving image captured on a non-passage. It is considered that the frequency component of the frequency is not related to the vibration of the passage but is related to the vibration of the vehicle 40. In the example of FIG. 12B, it is considered that the frequency component around 10 Hz is related to the vibration of the vehicle 40.

The passage vibration calculation unit 33 removes the frequency component of the specified frequency from the frequency component related to the moving image captured on the passage. At this time, frequency components in a certain range (for example, about a few to several Hz) before and after the specified frequency are removed so that appropriate removal can be performed. The passage vibration calculation unit 33 calculates the natural frequency from the frequency component after removal as information according to the vibration of the passage as described above. The determination using the calculated natural frequency is performed in the same manner as in the above-described embodiment.

In the above-described embodiment, the moving image captured from the vehicle 40 is used to determine the state of the passage. Therefore, it is not necessary to fixedly install a camera for taking an image of the passage near the passage, and it is possible to determine the state of the passage. In particular, if a drive recorder is used as described above, the condition of the passage can be determined (for example, diagnosis of damage to the bridge as described above) without installing a new camera for determining the passage. It can be carried out. Therefore, according to the present embodiment, it is possible to easily determine the state of a passage such as a bridge.

Further, the vibration of the vehicle 40 may be detected by a sensor such as the vibration sensor 20 as in the present embodiment described above. According to this configuration, the vibration of the vehicle 40 can be detected appropriately and surely, and the present invention can be surely carried out.

Alternatively, the vibration of the vehicle 40 may be detected from the moving image. According to this configuration, it is possible to determine the state of the passage only from the moving image without providing a sensor such as the vibration sensor 20.

Further, as shown in FIG. 3, the vibration of the passage may be calculated by subtracting the two types of vibration. According to this configuration, the vibration of the vehicle 40 can be detected appropriately and surely, and the present invention can be surely carried out.

Further, as shown in FIG. 11, the determination may be performed using a moving image captured from a vehicle operating on a traffic road. The present invention can also be carried out with such a configuration.

Since the passage determination system according to the present invention only needs to be able to acquire a moving image, it may be configured to be able to acquire a moving image from other than the passage determination system. For example, if the server 30 according to the present embodiment is configured to be able to acquire an image or a moving image from the camera 10 (other than the passage determination system), even if the passage determination system is configured only by the server 30. good.

Further, in the present embodiment, the natural frequency is calculated as information according to the vibration of the passage, and the state of the passage is determined from the natural frequency, but it is not always necessary to use the natural frequency. Any information may be used to determine the state of the passage as long as the information corresponds to the vibration of the passage. For example, the state of the passage may be determined from the vibration pattern of the vibration y ₁ (t) − αy ₂ (t) of the passage.

Further, the passage to be determined may be a track on which a train or the like passes. In this case, the vehicle 40 may be a train or the like passing through a railroad track. However, even if the determination target is a railroad track, the determination may be made using a moving image captured from a vehicle 40 traveling on a vehicle other than the railroad track.

The block diagram used in the description of the above embodiment shows a block of functional units. These functional blocks (components) are realized by any combination of hardware and / or software. Further, the means for realizing each functional block is not particularly limited. That is, each functional block may be realized by one physically and / or logically coupled device, or directly and / or indirectly by two or more physically and / or logically separated devices. (For example, wired and / or wireless) may be connected and realized by these plurality of devices.

For example, the server 30 in one embodiment of the present invention may function as a computer that processes the server 30 of the present embodiment. FIG. 13 is a diagram showing an example of the hardware configuration of the server 30 according to the present embodiment. The server 30 described above may be physically configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, and the like.

In the following description, the word "device" can be read as a circuit, a device, a unit, or the like. The hardware configuration of the server 30 may be configured to include one or more of the devices shown in the figure, or may be configured not to include some of the devices.

For each function in the server 30, by loading predetermined software (program) on hardware such as the processor 1001 and the memory 1002, the processor 1001 performs an operation, and the communication device 1004 communicates with the memory 1002 and the storage 1003. It is realized by controlling the reading and / or writing of data.

Processor 1001 operates, for example, an operating system to control the entire computer. The processor 1001 may be configured by a central processing unit (CPU) including an interface with a peripheral device, a control device, an arithmetic unit, a register, and the like. For example, each function of the server 30 may be realized by the processor 1001.

Further, the processor 1001 reads a program (program code), a software module and data from the storage 1003 and / or the communication device 1004 into the memory 1002, and executes various processes according to these. As the program, a program that causes a computer to execute at least a part of the operations described in the above-described embodiment is used. For example, each function of the server 30 may be stored in the memory 1002 and realized by a control program running on the processor 1001. Although it has been described that the various processes described above are executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. Processor 1001 may be mounted on one or more chips. The program may be transmitted from the network via a telecommunication line.

The memory 1002 is a computer-readable recording medium, and is composed of at least one such as a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), and a RAM (Random Access Memory). May be done. The memory 1002 may be referred to as a register, a cache, a main memory (main storage device), or the like. The memory 1002 can store a program (program code), a software module, or the like that can be executed to carry out the method according to the embodiment of the present invention.

The storage 1003 is a computer-readable recording medium, for example, an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, an optical magnetic disk (for example, a compact disk, a digital versatile disk, a Blu-ray). It may consist of at least one (registered trademark) disk), smart card, flash memory (eg, card, stick, key drive), floppy (registered trademark) disk, magnetic strip, and the like. The storage 1003 may be referred to as an auxiliary storage device. The storage medium described above may be, for example, a database, server or other suitable medium containing memory 1002 and / or storage 1003.

The communication device 1004 is hardware (transmission / reception device) for communicating between computers via a wired and / or wireless network, and is also referred to as, for example, a network device, a network controller, a network card, a communication module, or the like. For example, each function of the server 30 may be realized by the communication device 1004.

The input device 1005 is an input device (for example, a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that receives an input from the outside. The output device 1006 is an output device (for example, a display, a speaker, an LED lamp, etc.) that outputs to the outside. The input device 1005 and the output device 1006 may have an integrated configuration (for example, a touch panel).

Further, each device such as the processor 1001 and the memory 1002 is connected by a bus 1007 for communicating information. The bus 1007 may be composed of a single bus or may be composed of different buses between the devices.

Further, the server 30 includes hardware such as a microprocessor, a digital signal processor (DSP: Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). The hardware may implement some or all of each functional block. For example, the processor 1001 may be implemented on at least one of these hardware.

Although the present embodiment has been described in detail above, it is clear to those skilled in the art that the present embodiment is not limited to the embodiment described in the present specification. This embodiment can be implemented as an amendment or modification without departing from the spirit and scope of the present invention as determined by the description of the scope of claims. Therefore, the description of the present specification is for the purpose of illustration and does not have any limiting meaning to the present embodiment.

The notification of information is not limited to the embodiments / embodiments described herein, and may be performed by other methods. For example, information notification includes physical layer signaling (eg, DCI (Downlink Control Information), UCI (Uplink Control Information)), higher layer signaling (eg, RRC (Radio Resource Control) signaling, MAC (Medium Access Control) signaling, etc. It may be carried out by broadcast information (MIB (Master Information Block), SIB (System Information Block)), other signals, or a combination thereof. Further, the RRC signaling may be referred to as an RRC message, and may be, for example, an RRC Connection Setup message, an RRC Connection Reconfiguration message, or the like.

The processing procedures, sequences, flowcharts, and the like of each aspect / embodiment described in the present specification may be rearranged in order as long as there is no contradiction. For example, the methods described herein present elements of various steps in an exemplary order and are not limited to the particular order presented.

The input / output information and the like may be stored in a specific place (for example, a memory) or may be managed by a management table. Information to be input / output may be overwritten, updated, or added. The output information and the like may be deleted. The input information or the like may be transmitted to another device.

The determination may be made by a value represented by 1 bit (0 or 1), by a boolean value (Boolean: true or false), or by comparing numerical values (for example, a predetermined value). It may be done by comparison with the value).

Each aspect / embodiment described in the present specification may be used alone, in combination, or may be switched and used according to the execution. Further, the notification of predetermined information (for example, the notification of "being X") is not limited to the explicit one, but is performed implicitly (for example, the notification of the predetermined information is not performed). May be good.

Software, whether called software, firmware, middleware, microcode, hardware description language, or other names, is an instruction, instruction set, code, code segment, program code, program, subprogram, software module. , Applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, features, etc. should be broadly interpreted.

Further, software, instructions, and the like may be transmitted and received via a transmission medium. For example, the software may use wired technology such as coaxial cable, fiber optic cable, twisted pair and digital subscriber line (DSL) and / or wireless technology such as infrared, wireless and microwave to website, server, or other. When transmitted from a remote source, these wired and / or wireless technologies are included within the definition of transmission medium.

The information, signals, etc. described herein may be represented using any of a variety of different techniques. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description are voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these. It may be represented by a combination of.

The terms described herein and / or the terms necessary for understanding the present specification may be replaced with terms having the same or similar meanings.

The terms "system" and "network" used herein are used interchangeably.

Further, the information, parameters, etc. described in the present specification may be represented by an absolute value, a relative value from a predetermined value, or another corresponding information. ..

The names used for the parameters mentioned above are not limited in any way. Further, mathematical formulas and the like using these parameters may differ from those expressly disclosed herein.

The terms "determining" and "determining" as used herein may include a wide variety of actions. "Judgment" and "decision" are, for example, judgment, calculation, computing, processing, deriving, investigating, looking up (for example, table). , Searching in a database or another data structure), ascertaining can be considered as a "judgment" or "decision". Also, "judgment" and "decision" are receiving (for example, receiving information), transmitting (for example, transmitting information), input (input), output (output), and access. It may include (for example, accessing data in memory) to be regarded as "judgment" or "decision". In addition, "judgment" and "decision" are considered to be "judgment" and "decision" when the things such as solving, selecting, choosing, establishing, and comparing are regarded as "judgment" and "decision". Can include. That is, "judgment" and "decision" may include considering some action as "judgment" and "decision".

The terms "connected", "coupled", or any variation thereof, mean any direct or indirect connection or connection between two or more elements and each other. It can include the presence of one or more intermediate elements between two "connected" or "combined" elements. The connection or connection between the elements may be physical, logical, or a combination thereof. As used herein, the two elements are by using one or more wires, cables and / or printed electrical connections, and, as some non-limiting and non-comprehensive examples, radio frequencies. By using electromagnetic energies such as electromagnetic energies with wavelengths in the region, microwave region and light (both visible and invisible) regions, they can be considered to be "connected" or "coupled" to each other.

The phrase "based on" as used herein does not mean "based on" unless otherwise stated. In other words, the statement "based on" means both "based only" and "at least based on".

As used herein by designations such as "first", "second", etc., any reference to that element does not generally limit the quantity or order of those elements. These designations can be used herein as a convenient way to distinguish between two or more elements. Thus, references to the first and second elements do not mean that only two elements can be adopted there, or that the first element must somehow precede the second element.

As long as "include", "including", and variations thereof are used herein or within the scope of the claims, these terms are similar to the term "comprising". In addition, it is intended to be inclusive. Moreover, the term "or" as used herein or in the claims is intended to be non-exclusive.

In the present specification, a plurality of devices shall be included unless the device has only one device apparently in context or technically. The entire disclosure is intended to include more than one, unless the context clearly indicates the singular.

1 ... Traffic path determination system, 10 ... Camera, 20 ... Vibration sensor, 30 ... Server, 31 ... Motion image acquisition unit, 32 ... Vehicle vibration detection unit, 33 ... Traffic path vibration calculation unit, 34 ... Judgment unit, 35 ... Output Department, 40 ... vehicle, 1001 ... processor, 1002 ... memory, 1003 ... storage, 1004 ... communication device, 1005 ... input device, 1006 ... output device, 1007 ... bus.

Claims (6)

A moving image acquisition unit that acquires a moving image captured from a moving vehicle including a traffic road, or a moving image captured from a vehicle operating on a traffic road, and a moving image acquisition unit.
A vehicle vibration detection unit that detects the vibration of the vehicle during operation,
A passage vibration calculation unit that calculates information according to the vibration of the passage from the motion image acquired by the motion image acquisition unit and the vibration of the vehicle detected by the vehicle vibration detection unit.
A determination unit that determines the state of the passage from the information corresponding to the vibration of the passage calculated by the passage vibration calculation unit.
Equipped with
Vibration is a time-series displacement.
The passage vibration calculation unit calculates moving image vibration including the vibration of the passage and the vibration of the vehicle from the moving image acquired by the moving image acquisition unit, and detects the calculated moving image vibration and the vehicle vibration detecting unit. A passage determination system that calculates information according to the vibration of the passage from the vibration of the vehicle. The passage determination system according to claim 1, wherein the vehicle vibration detection unit detects vibration of the vehicle based on a sensor provided on the vehicle. The passage determination system according to claim 1 or 2, wherein the vehicle vibration detection unit calculates vibration of the vehicle from a portion of a moving image acquired by the motion image acquisition unit other than the passage path. The passage vibration calculation unit calculates vibration from the portion of the moving image acquired by the motion image acquisition unit in which the passage is reflected, and the vehicle vibration detection unit detects the vibration from the calculated vibration of the vehicle. The passage determination system according to any one of claims 1 to 3, wherein the vibration is subtracted to calculate the vibration of the passage as information corresponding to the vibration of the passage. The moving image acquisition unit acquires a moving image captured from the vehicle operating on the traffic road, and obtains a moving image.
The passage vibration calculation unit calculates a frequency component related to the vibration of the vehicle from the vibration of the vehicle detected by the vehicle vibration detection unit, and operates on the passage acquired by the moving image acquisition unit. The vibration is calculated from the moving image captured from the vehicle, the frequency component related to the calculated vibration is calculated, and the frequency component related to the vibration calculated from the moving image taken from the vehicle operating on the passageway is used. The passage according to any one of claims 1 to 4, wherein the frequency component related to the vibration of the vehicle is removed, and the natural frequency is calculated from the removed frequency component as information corresponding to the vibration of the passage. Judgment system. The moving image acquisition unit also acquires a moving image captured from the vehicle operating on a road other than the traffic road.
Any one of claims 1 to 5, wherein the vehicle vibration detection unit calculates the vibration of the vehicle from a moving image captured from the vehicle operating on a road other than the traffic road acquired by the moving image acquisition unit. The traffic route determination system described in.

Images