JP2021143563A - Information processing device, information processing method, and program - Google Patents

Abstract

To provide an information processing device capable of efficiently and accurately determining abnormality of a road surface, a railroad, an overhead wire, and so on.SOLUTION: An information processing device includes a detection device communication section for acquiring a detection signal of a physical amount which is varied in accordance with movement of a vehicle 70 while in contact with a road surface R, an imaging device control section for acquiring image information of the vehicle 70 taken by imaging devices 40A, 40B, and a display control section for displaying the image information on a display. The imaging device control section controls the imaging devices on the basis of the detection signal.SELECTED DRAWING: Figure 2

Description

The present invention relates to information processing devices, information processing methods and programs.

In Patent Document 1, the pavement surface is photographed by the thermal infrared camera 3 and the visible light camera 4 while traveling on the road, the vibration of the vehicle according to the state of the pavement surface is detected by the vibration sensor 5, and the running sound detection unit. 6 detects the sound generated by the running of the vehicle, the water permeability detection unit 7 detects the water permeability of the pavement, and the IC tag communication unit 8 collects the detection information of the sensors arranged in advance around the road. The inspection vehicle 1 that collects the shape of the pavement surface by the pavement surface measurement unit 9 is disclosed.

An object of the present invention is to provide an information processing device, an information processing method, and a program capable of efficiently and accurately determining an abnormality in a target portion.

The information processing apparatus according to the present invention includes a detection information acquisition unit that acquires detection information of a physical quantity that changes when a moving object moves while contacting the target unit, and image information of the target unit that is captured by the imaging unit. An image information acquisition unit for acquiring information processing and a display control unit for displaying image information on the display unit are provided, and the image pickup unit is controlled based on the detection information.

According to the present invention, it is possible to provide an information processing device, an information processing method and a program capable of efficiently and accurately determining an abnormality in a target portion.

It is a figure which shows an example of the system structure of the abnormality detection system which concerns on embodiment of this invention. It is a figure which shows the vehicle which concerns on this embodiment. It is a figure which shows an example of the hardware composition of an information processing apparatus. It is a figure which shows an example of the hardware composition of the control device of a vehicle. It is a figure which shows an example of the functional structure of the abnormality detection system which concerns on this embodiment. It is a figure which shows an example of the detailed functional structure of the signal processing part which concerns on this embodiment. It is a sequence diagram which shows an example of the storage process of the detection signal in the abnormality detection system which concerns on this embodiment. It is a conceptual diagram which shows an example of the condition information management table which concerns on this embodiment. It is a flowchart which shows an example of the signal processing with respect to the detection signal in the information processing apparatus which concerns on this embodiment. (A) is a diagram showing an example of a spectrogram of a detection signal during normal operation, and (b) is a diagram showing an example of a spectrogram of a detection signal during an abnormality occurrence. It is a conceptual diagram which shows an example of the detection signal management table which concerns on this embodiment. It is a flowchart which shows an example of the processing with respect to the image information in the information processing apparatus which concerns on this embodiment. It is a conceptual diagram which shows an example of the image information management table which concerns on this embodiment. It is a flowchart which shows an example of the score display process and the model information storage process in the information processing apparatus which concerns on this embodiment. It is a figure which shows an example of the output signal selection screen displayed on the information processing apparatus which concerns on this embodiment. It is a flowchart which shows an example of the display selection process in the information processing apparatus which concerns on this embodiment. It is a figure which shows an example of the display selection screen displayed on the information processing apparatus which concerns on this embodiment. It is a figure which shows an example of the image information displayed on the display selection screen which concerns on this embodiment. It is a conceptual diagram which shows an example of the model information management table which concerns on this embodiment. It is a figure which shows an example of the display selection screen displayed on the information processing apparatus which concerns on the 1st modification of this embodiment. It is a figure which shows an example of the image information displayed on the display selection screen which concerns on the 1st modification. It is a figure which shows an example of the display selection screen displayed on the information processing apparatus which concerns on the 2nd modification of this embodiment. It is a figure which shows the railroad vehicle which concerns on the 3rd modification of this embodiment. It is a conceptual diagram which shows an example of the condition information management table which concerns on the 3rd modification. It is a figure which shows an example of the display selection screen displayed on the information processing apparatus which concerns on 3rd modification. It is a figure which shows the railroad vehicle which concerns on the 4th modification of this embodiment. It is a conceptual diagram which shows an example of the condition information management table which concerns on 4th modification. It is a figure which shows an example of the display selection screen displayed on the information processing apparatus which concerns on 4th modification.

Hereinafter, modes for carrying out the invention will be described with reference to the drawings. In the description of the drawings, the same elements are designated by the same reference numerals, and duplicate description will be omitted.

● System Configuration FIG. 1 is a diagram showing an example of a system configuration of an abnormality detection system according to an embodiment of the present invention.

The abnormality detection system 1A according to the present embodiment includes a vehicle 70 and an information processing system 5. The information processing system 5 includes a detection device 30, an image pickup device 40, and an information processing device 10.

The vehicle 70 is an example of a moving body that moves while contacting a target portion such as a road surface. The detection device 30 detects a physical quantity that changes as the vehicle 70 moves while contacting the road surface. The image pickup device 40 is an example of an image pickup device that captures an image of a moving image or a still image of a target portion such as a road surface.

The abnormality detection system 1A may include a plurality of vehicles 70, a plurality of detection devices 30 corresponding to the plurality of vehicles 70, and a plurality of imaging devices 40.

The information processing device 10 is a diagnostic device that is connected to the vehicle 70 so as to be able to communicate with the vehicle 70 and diagnoses an abnormality in a target portion such as a road surface. The information processing device 10 may be a general-purpose PC (Personal Computer) in which a dedicated software program is installed. Further, the information processing device 10 may be composed of a single computer or a plurality of computers.

The information processing device 10 and the vehicle 70 may be connected in any connection form. For example, the information processing device 10 and the vehicle 70 may be connected by a dedicated connection line, a wired network such as a wired LAN (Local Area Network), a wireless network, or the like.

The detection device 30 detects a physical quantity such as vibration or sound generated when the tire installed on the vehicle 70 and the road surface or the like come into contact with each other while the vehicle 70 is moving, and detects the information of the detected physical quantity as a detection signal (sensor data). Is a sensor that outputs to the information processing device 10. The detection device 30 is composed of, for example, a microphone, a vibration sensor, an acceleration sensor, an AE sensor, or the like, and detects a change in a physical quantity such as vibration or sound.

These detecting means are installed in the vicinity of the tire. The installation method is fixed by a screw, fixed by a magnet, bonded by an adhesive, or installed by embedding it in a hole. The number of detection devices 30 may be arbitrary. Further, a plurality of detection devices 30 for detecting the same physical quantity may be provided, or a plurality of detection devices 30 for detecting different physical quantities may be provided.

Here, between the information processing device 10 and the detection device 30, several types of filters for filtering the output signal from the detection device 30 or filter selection means for selecting the filters may be provided as necessary.

FIG. 2 is a diagram showing a vehicle according to the present embodiment. The vehicle 70 includes a control device 70A, a tire 50, and a detection device 30. Further, the vehicle 70 has a first imaging device 40A arranged so as to image the road surface R on the traveling direction side of the vehicle 70, and a first imaging device 40A arranged so as to image the road surface R behind the vehicle 70 from the vertical direction. The image pickup device 40B of 2 is provided. The tire 50 is an example of a moving member that moves while contacting a target portion such as a road surface R.

The detection device 30 may be provided in advance in the vehicle 70, or may be attached to the vehicle 70 later.

Hardware Configuration Next, the hardware configuration of the information processing device 10 and the vehicle 70 according to the present embodiment will be described with reference to FIGS. 3 and 4. The hardware configurations shown in FIGS. 3 and 4 may have similar configurations in each embodiment, and components may be added or deleted as necessary.

〇 Hardware configuration of the information processing device 〇 FIG. 3 is a diagram showing an example of the hardware configuration of the information processing device according to the present embodiment.

The information processing device 10 is constructed by a computer, and as shown in FIG. 23, the CPU (Central Processing Unit) 101, the ROM (Read Only Memory) 102, the RAM (Random Access Memory) 103, and the HD (Hard Disk) 104. , HDD (Hard Disk Drive) controller 105, display I / F 106 and communication I / F 107.

Of these, the CPU 101 controls the operation of the entire information processing device 10. The ROM 102 stores a program used for driving the CPU 101 such as an IPL (Initial Program Loader). The RAM 103 is used as a work area of the CPU 101. The HD104 stores various data such as programs. The HDD controller 105 controls reading or writing of various data to the HD 104 according to the control of the CPU 101. The display I / F 106 is a circuit for displaying an image on the display 106a. The display 106a is a kind of display unit such as a liquid crystal display or an organic EL that displays an image of a subject, various icons, and the like. The communication I / F 107 is an interface used for communication with an external device such as a vehicle 70. The communication I / F 107 is, for example, a NIC (Network Interface Card) corresponding to TCP (Transmission Protocol) / IP (Internet Protocol).

Further, the information processing device 10 includes a sensor I / F 108, a sound input / output I / F 109, an input I / F 110, a media I / F 111, and a DVD-RW (Digital Versaille Disk Rewritable) drive 112.

The sensor I / F 108 is an interface that receives a detection signal via the sensor amplifier 302 included in the detection device 30. The sound input / output I / F 109 is a circuit that processes sound signal input / output between the speaker 109a and the microphone 109b under the control of the CPU 101. The input I / F 110 is an interface for connecting a predetermined input means to the information processing device 10. .. The keyboard 110a is a kind of input means including a plurality of keys for inputting characters, numerical values, various instructions, and the like. The mouse 110b is a kind of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, and the like. Instead of the keyboard 708a and the mouse 708b, a touch panel or the like may be used as the input means. The media I / F 111 controls reading or writing (storage) of data to a recording medium 111a such as a flash memory. The DVD-RW drive 112 controls reading or writing of various data to the DVD-RW 112a as an example of a removable recording medium. In addition, it is not limited to DVD-RW, and may be DVD-R or the like. Further, the DVD-RW drive 112 may be a Blu-ray drive that controls reading or writing of various data to a Blu-ray disc.

Further, the information processing device 10 includes a bus line 113. The bus line 113 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 101.

Note that any recording medium such as an HD or CD-ROM in which each of the above programs is stored can be provided as a program product (Program Product) in Japan or overseas.

〇 Vehicle hardware configuration 〇 FIG. 4 is a diagram showing an example of the hardware configuration of the vehicle control device according to the present embodiment.

As shown in FIG. 4, the control device 70A includes a CPU 701, a ROM 702, a RAM 703, a display I / F 704, a communication I / F 705, a drive circuit 706, a sound output I / F 707, an input I / F 708, and a sensor I / F 709. ing.

Of these, the CPU 701 controls the operation of the entire control device 70A. The ROM 702 stores a program used to drive the CPU 701 such as an IPL. The RAM 703 is used as a work area for the CPU 701. The display I / F 704 is a circuit for displaying an image on the display 704a. The display 704a is a kind of display unit such as a liquid crystal display or an organic EL that displays an image of a subject, various icons, and the like.

The communication I / F 705 is an interface used for communication with an external device such as the information processing device 10. The communication I / F705 is, for example, a NIC or the like corresponding to TCP / IP.

The drive circuit 706 is a circuit that controls the drive of the motor 706a. The motor 706a drives the tire 50.

The sound output I / F 707 is a circuit that processes the output of a sound signal between the speaker 707a and the microphone 707b under the control of the CPU 701. The input I / F 708 is an interface for connecting a predetermined input means to the control device 70A. The keyboard 708a is a kind of input means including a plurality of keys for inputting characters, numerical values, various instructions, and the like. The mouse 708b is a kind of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, and the like. Instead of the keyboard 708a and the mouse 708b, a touch panel or the like may be used as the input means.

Further, the control device 70A includes a bus line 710. The bus line 710 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 701.

Further, the detection device 30 includes a sensor 301 and a sensor amplifier 302, and detects a physical quantity such as vibration or sound that changes as the vehicle 70 moves while contacting the road surface R. As described above, the sensor 301 detects a physical quantity such as vibration or sound generated when the tire 50 of the vehicle 70 and the road surface R come into contact with each other during the movement of the vehicle 70. Further, the sensor 301 acquires a detection signal (sensor data) based on the detected physical quantity information. The sensor 301 is, for example, a microphone, a vibration sensor, an acceleration sensor, an AE sensor, or the like. The sensor amplifier 302 adjusts the detection sensitivity of the sensor 301 and outputs the detection signal detected by the sensor 301.

● Functional configuration FIG. 5 is a diagram showing an example of the functional configuration of the abnormality detection system according to the present embodiment.

〇Functional configuration of information processing device 〇First, the functional configuration of the information processing device 10 will be described. The functions realized by the information processing device 10 are the transmission / reception unit 11, the detection device communication unit 12, the reception unit 13, the display control unit 14, the sound control unit 15, the generation unit 16, the signal processing unit 17, the selection unit 18, and the judgment unit. 21, a storage / reading unit 19, an image pickup device control unit 41, and a storage unit 1000 are included.

The transmission / reception unit 11 is a function of transmitting / receiving various data (or information) to / from an external device such as a vehicle 70. The transmission / reception unit 11 receives, for example, processing information (travel information) related to the current operation of the vehicle 70. The transmission / reception unit 11 is mainly realized by the communication I / F 107 shown in FIG. 3, a program executed by the CPU 101, and the like. The transmission / reception unit 11 is an example of a processing information acquisition unit.

The detection device communication unit 12 is a function of performing data communication with the detection device 30. The detection device communication unit 12 receives, for example, a detection signal (sensor data) related to a physical quantity detected by the detection device 30. The detection device communication unit 12 is mainly realized by a program or the like executed by the CPU 101 shown in FIG. The detection device communication unit 12 is an example of a detection information acquisition unit that acquires detection information. Further, the detection signal received by the detection device communication unit 12 is an example of detection information relating to a physical quantity that changes as the vehicle 70 moves while being in contact with the road surface R.

The reception unit 13 is a function of receiving user input to an input means such as the keyboard 110a shown in FIG. The reception unit 13 receives selection of output items in response to an input to the output signal selection screen 200 (see FIG. 15), for example. The reception unit 13 is mainly realized by a program or the like executed by the CPU 101 shown in FIG.

The display control unit 14 is a function of displaying various screens on the display 106a shown in FIG. The display control unit 14 causes, for example, display the output signal selection screen 200 (see FIG. 15) on the display 106a. Specifically, the display control unit 14 includes at least HTML (HyperText Markup Language) by starting and executing a software application running on the OS, for example, and CSS (Cascading Style Sheets) or JAVA SCPRIPT (registered trademark). ) Etc. are downloaded. Then, the display control unit 14 causes the display 106a to display various image data generated by the Web APP. The display control unit 14 displays image data generated by HTML5, for example, displaying data in HTML5 (Extensible Markup Language), JSON (Javascript Object Notification), SOAP (Simple Object Access Protocol) format, or the like. The display control unit 14 is mainly realized by the display I / F 106 shown in FIG. 3, a program executed by the CPU 101, and the like.

The sound control unit 15 is a function of outputting a sound signal from the speaker 109a shown in FIG. The sound control unit 15 sets a detection signal to be output from the speaker 109a, and acoustically outputs the set detection signal from the speaker 109a. The sound control unit 15 is mainly realized by the sound input / output I / F 109 shown in FIG. 3, a program executed by the CPU 101, and the like.

The generation unit 16 is a function of generating various image data to be displayed on the display 106a. The generation unit 16 generates, for example, image data related to the output signal selection screen 200 (see FIG. 15) to be displayed on the display 106a. The generation unit 16 renders the data stored in the storage unit 1000, for example, and generates image data for displaying the rendered data. Rendering is a process of interpreting data described in a language for describing a Web page (HTML, CSS, XML, etc.) and calculating the arrangement of characters, image data, etc. actually displayed on the screen. Further, the generation unit 16 generates a condition ID for identifying the condition information including the received processing information when the processing information is received by the transmission / reception unit 11. The generation unit 16 is mainly realized by a program or the like executed by the CPU 101 shown in FIG.

The signal processing unit 17 is a function of performing signal processing of the detection signal received by the detection device communication unit 12. A detailed description of the signal processing unit 17 will be described later. The signal processing unit 17 is mainly realized by a program or the like executed by the CPU 101 shown in FIG.

The selection unit 18 is a function of selecting a detection signal to be acoustically output based on a signal output request from the user. The selection unit 18 selects, for example, a detection signal stored in association with the condition information corresponding to the output item data included in the signal output request received by the reception unit 13. The selection unit 18 is mainly realized by a program or the like executed by the CPU 101 shown in FIG.

The determination unit 21 is a function that is realized mainly by the processing of the CPU 101 shown in FIG. 3 and makes various determinations. The determination unit 21 calculates, for example, the difference in signal data related to the plurality of detection signals selected by the selection unit 18. The determination unit 21 is an example of an abnormality determination unit.

The image pickup device control unit 41 is a function of communicating control signals and image data with the image pickup device 40 including the first image pickup device 40A and the second image pickup device 40B. The image pickup device control unit 41 transmits, for example, an image pickup start signal to the image pickup device 40. Further, the image pickup device control unit 41 is an example of an image information acquisition unit that acquires image information of the road surface R, and receives, for example, image information of a moving image or a still image of the road surface R captured by the image pickup device 40. The image pickup apparatus control unit 41 is mainly realized by a program or the like executed by the CPU 101 shown in FIG.

The storage / reading unit 19 is a function of storing various data in the storage unit 1000 or reading various data from the storage unit 1000. The storage / reading unit 19 is mainly realized by a program or the like executed by the CPU 101 shown in FIG. The storage unit 1000 is mainly realized by the ROM 102, HD 104 and the recording medium 111a shown in FIG.

Further, the storage unit 1000 is constructed with a condition information management DB 1001, a detection signal management DB 1003, a model information management DB 1005, and an image information management DB 1007. Of these, the condition information management DB 1001 is composed of a condition information management table described later. The detection signal management DB 1003 is composed of a detection signal management table described later. The model information management DB 1005 is composed of a model information management table described later. The image information management DB 1007 is composed of an image information management table described later. The storage / reading unit 19 is an example of a storage control unit.

〇 Functional configuration of the detection device 〇 Next, the functional configuration of the detection device 30 will be described. The functions realized by the detection device 30 include the device connection unit 31 and the detection signal acquisition unit 32.

The device connection unit 31 is a function of transmitting the detection signal acquired by the detection signal acquisition unit 32 to the information processing device 10. The device connection portion 31 is mainly realized by the sensor amplifier 302 shown in FIG.

The detection signal acquisition unit 32 is a function of detecting a physical quantity such as vibration or sound that changes as the vehicle 70 moves while contacting the road surface R, and acquires information on the physical quantity as a detection signal. The detection signal acquisition unit 32 is mainly realized by the sensor 301 shown in FIG. The detection signal acquisition unit 32 detects vibration or sound generated when the tire 50 of the vehicle 70 and the road surface R come into contact with each other while the vehicle 70 is moving, and acquires the detected physical quantity information as detection information (sensor data). do. For example, when the road surface R is cracked or the like, the sound at the time of contact between the tire 50 and the road surface R changes. Therefore, the detection signal acquisition unit 32 detects acoustic data using a sensor 301 such as a microphone, and transmits the detection signal related to the detected acoustic data to the information processing device 10 via the device connection unit 31. The detection signal acquisition unit 32 is mainly realized by the sensor 301 shown in FIG.

〇Vehicle functional configuration 〇Next, the functions realized by the control device 70A of the vehicle 70 are the transmission / reception unit 71, the main control unit 72, the drive control unit 73, the drive unit 74, the setting unit 75, the reception unit 76, and the display control. A unit 77 and a sound control unit 78 are included.

The transmission / reception unit 71 is a function of transmitting / receiving various data (or information) to / from an external device such as the information processing device 10. The transmission / reception unit 71 transmits the processing information related to the current operation of the vehicle 70 to the information processing device 10. The transmission / reception unit 71 is mainly realized by the communication I / F 705 shown in FIG. 4, a program executed by the CPU 701, and the like.

The main control unit 72 generates and outputs control data for controlling the operation of the drive unit 74. Further, the main control unit 72 outputs the processing information related to the operation of the vehicle 70 to the transmission / reception unit 71. For example, the main control unit 72 sequentially transmits information corresponding to the current operation of the vehicle 70 to the information processing device 10 via the transmission / reception unit 71. When the vehicle 70 travels, the main control unit 72 appropriately changes the type of the drive unit 74 to be driven or the drive state (rotation speed, rotation speed, etc.) of the drive unit 74. Each time the main control unit 72 changes the type of operation, the main control unit 72 sequentially transmits information corresponding to the changed type of operation to the information processing device 10 via the transmission / reception unit 71. The main control unit 72 is mainly realized by a program or the like executed by the CPU 701 shown in FIG.

The drive control unit 73 is a function of driving and controlling the drive unit 74 based on the control data obtained by the main control unit 72. The drive control unit 73 is realized by, for example, the drive circuit 706 shown in FIG. The drive control unit 73 is mainly realized by the drive circuit 706 shown in FIG. 4, a program executed by the CPU 701, and the like.

The drive unit 74 is a function that is the target of drive control by the drive control unit 73. The drive unit 74 drives the tire 50 under the control of the drive control unit 73. The drive unit 74 is an actuator that is drive-controlled by the drive control unit 73, and is mainly realized by the motor 706a or the like shown in FIG. The drive unit 74 may be any actuator as long as it is used for traveling the vehicle 70 and is a target of control. Further, two or more drive units 74 may be provided.

The setting unit 75 is a function of setting condition information corresponding to the current operation of the vehicle 70. The setting unit 75 is mainly realized by a program or the like executed by the CPU 701 shown in FIG.

The reception unit 76 is a function of receiving user input to an input means such as the keyboard 708a shown in FIG. The reception unit 76 receives selection of output items in response to an input to the output signal selection screen 200 (see FIG. 15) displayed on the display 704a, for example. The reception unit 76 is mainly realized by the input I / F 708 shown in FIG. 4, a program executed by the CPU 701, and the like.

The display control unit 77 is a function of displaying various screen information on the display 704a shown in FIG. The display control unit 77 causes the display 704a to display the output signal selection screen 200 (see FIG. 15), for example. The display control unit 77 is mainly realized by the display I / F 704 shown in FIG. 4, a program executed by the CPU 701, and the like.

The sound control unit 78 is a function realized by a command from the CPU 701 shown in FIG. 4 and outputs a sound signal from the speaker 707a. The sound control unit 78 sets a detection signal to be output from the speaker 707a, and causes the set detection signal to be acoustically output from the speaker 707a. The sound control unit 78 is mainly realized by the sound output I / F707 shown in FIG. 4, a program executed by the CPU 701, and the like.

FIG. 6 is a diagram showing an example of a detailed functional configuration of the signal processing unit according to the present embodiment. The signal processing unit 17 shown in FIG. 6 includes an amplification processing unit 171, an A / D conversion unit 172, a feature amount extraction unit 173, a D / A conversion unit 174, and a score calculation unit 175.

The amplification processing unit 171 is a function of performing amplification processing of the detection signal received by the detection device communication unit 12. The amplification processing unit 171 amplifies the analog signal received by the detection device communication unit 12, for example, to an arbitrary size. Further, the amplification processing unit 171 amplifies the digital signal converted by the A / D conversion unit 172 to an arbitrary size, for example.

The A / D conversion unit 172 is a function of performing a process of converting an analog signal amplified by the amplification processing unit 171 into a digital signal.

The feature amount extraction unit 173 is a function of extracting the feature amount (feature information) indicating the feature of the detection signal received by the detection device communication unit 12. The feature amount may be any information as long as it is information indicating the features of the detection signal. For example, when the detection signal is acoustic data, the feature amount extraction unit 173 may extract energy, frequency spectrum, time, MFCC (mel frequency cepstrum coefficient), or the like as feature amounts.

The D / A conversion unit 174 is a function of converting a digital signal amplified by the amplification processing unit 171 into an analog signal.

The score calculation unit 175 calculates a score as an example of possibility information indicating the possibility of abnormality of the road surface R from the feature amount (for example, frequency spectrum) of the detection signal extracted by the feature amount extraction unit 173. The score calculation unit 175 is an example of a possibility information determination unit that determines the possibility information indicating the possibility of an abnormality of the road surface R.

〇 Memory processing of detection signal 〇 FIG. 7 is a sequence diagram showing an example of storage processing of detection signal in the abnormality detection system according to the present embodiment.

In step S11, the transmission / reception unit 71 of the control device 70A transmits the processing information related to the current operation of the vehicle 70 to the information processing device 10 constituting the information processing system 5. Specifically, the setting unit 75 of the control device 70A sets the processing information indicating the specific traveling content at the start of traveling of the vehicle 70. As described above, the processing information is information defined for each type of operation of the vehicle 70. Then, the transmission / reception unit 71 transmits the processing information set by the setting unit 75 to the information processing device 10. As a result, the transmission / reception unit 11 of the information processing device 10 receives the processing information transmitted from the control device 70A (an example of the processing information acquisition step).

In step S12, the generation unit 16 of the information processing device 10 generates a condition ID for identifying the condition information including the processing information received by the transmission / reception unit 11.

Then, in step S13, the storage / reading unit 19 stores and manages the condition ID generated by the generation unit 16 and the condition information associated with the processing information received by the transmission / reception unit 11 in the condition information management DB 1001. (Example of memory control step). At that time, the storage / reading unit 19 also stores and manages the related ID given to the processing information indicating the related processing in the condition information management DB 1001. The condition information management DB 1001 stores and manages the processing information indicating the content of the specific traveling process executed by the vehicle 70 in the condition information management table in association with each condition ID.

In step S14, the detection signal acquisition unit 32 of the detection device 30 constituting the information processing system 5 detects a physical quantity such as vibration or sound generated by the traveling of the vehicle 70. Here, the detection signal acquisition unit 32 detects the sound generated by the traveling of the vehicle 70, and acquires the detection signal (acoustic signal) related to the detected sound.

In step S15, the device connection unit 31 of the detection device 30 transmits the detection signal acquired in step S14 to the information processing device 10. As a result, the detection device communication unit 12 of the information processing device 10 receives the detection signal transmitted from the detection device 30 (an example of the detection information acquisition step).

In step S16, the signal processing unit 17 of the information processing device 10 performs signal processing of the detection signal received by the detection device communication unit 12.

In step S17, the storage / reading unit 19 of the information processing device 10 stores the signal data processed by the signal processing unit 17 in the detection signal management DB 1003 in association with the processing information transmitted from the control device 70A. An example of a memory control step). The information processing device 10 uses the travel time data included in the processing information received in step S11, the signal data related to the detection signal received in step S15, and the signal processing unit 17 for each condition ID generated in step S12. The processed signal data (frequency data, score data) and the processing information data included in the processing information received in step S11 are associated and managed in the detection signal management table. The information processing device 10 may store and manage the mileage data instead of the mileage data.

That is, the storage / reading unit 19 stores each feature amount (frequency data) of the plurality of feature amounts in the detection signal management DB 1003 in association with each processing information of the plurality of processing information transmitted from the control device 70A. Let me.

〇 Condition information management table FIG. 8 is a conceptual diagram showing an example of the condition information management table according to the present embodiment. In the storage unit 1000 shown in FIG. 5, the condition information management DB 1001 described in step S13 of FIG. 7 is configured by the condition information management table as shown in FIG.

The condition information management table shown in FIG. 8 is for managing processing information related to the operation of the vehicle 70 for each operation performed by the vehicle 70. In the condition information management table, condition information associated with processing information for each condition ID is stored and managed. The condition ID is identification information for identifying the condition information including the processing information. The processing information is information defined for each type of operation of the vehicle 70. As shown in FIG. 8, the processing information includes information on the name of the road on which the vehicle 70 travels, the pavement material of the road on which the vehicle 70 travels, and the type of the tire 50 of the vehicle 70.

The items included in the processing information further include information indicating the user's operation history information for the vehicle 70, the travel start date and time, the travel time (an example of the travel time of the vehicle 70), the travel speed of the vehicle 70, and the like. May be good. Further, the mileage (an example of the moving distance of the vehicle 70) may be included, and the mileage may be included instead of the traveling time. Travel information including travel time or travel distance is an example of travel information.

Further, in the condition information management table, a related ID for identifying a related operation (process) among the operations by the vehicle 70 is associated and stored and managed. As the related ID, the same related ID is assigned to the processing information indicating the related processing among the processing information included in the condition information management table. In the example of FIG. 8, the same related ID; "R001" is assigned to the processing information identified by the condition ID; "A000001" and the condition ID; "A00003". Here, the related process to which the related ID is assigned is, for example, a process in which the pavement material and the tire type are the same but the road name is different.

Next, processing on the detection signal by the signal processing unit 17 corresponding to step S16 in FIG. 7 will be described with reference to FIGS. 9 to 10.

FIG. 9 is a flowchart showing an example of signal processing for a detection signal in the information processing apparatus according to the present embodiment.

First, in step S151, when the detection signal is received (acquired) by the detection device communication unit 12, the information processing device 10 shifts the process to step S152. On the other hand, the information processing device 10 repeats the process of step S151 until the detection signal is received (acquired) by the detection device communication unit 12.

In step S152, the amplification processing unit 171 of the signal processing unit 17 performs amplification processing of the detection signal received (acquired) by the detection device communication unit 12 to amplify the detection signal to an arbitrary size. In step S153, the A / D conversion unit 172 of the signal processing unit 17 converts the analog signal amplified by the amplification processing unit 171 into a digital signal.

In step S154, the feature amount extraction unit 173 of the signal processing unit 17 performs a process of extracting the feature amount (feature amount information) indicating the feature of the digital signal converted by the A / D conversion unit 172 (feature amount extraction step). One case). Specifically, the feature amount extraction unit 173 extracts the frequency spectrum included in the digital signal converted by the A / D conversion unit 172.

In step S155, the score calculation unit 175 of the signal processing unit 17 calculates a score indicating the possibility of abnormality of the road surface R from the feature amount (for example, frequency spectrum) of the detection signal extracted by the feature amount extraction unit 173.

Specifically, the score calculation unit 175 determines the likelihood indicating the likelihood that the feature information of the detection information is normal, and the model of the feature information indicating the normal data stored in the model information management DB 1005 of the storage unit 1000. It is calculated using information, and the inverse of the likelihood is calculated as a score.

The score calculation unit 175 may calculate by ranking the scores step by step, or may calculate by two values of 0 and 1. Further, the score calculation unit 175 may accumulate the calculated scores.

Further, the score calculation unit 175 calculates the likelihood indicating the plausibility that the feature information of the detection information is abnormal as a score using the model information of the feature information indicating the abnormality data stored in the model information management DB 1005. You may.

Here, the frequency component of the detection signal detected while the vehicle 70 is traveling, which corresponds to step S154 of FIG. 9, will be described.

FIG. 10A shows a spectrogram of a detection signal detected when the vehicle 70 travels on a normal road surface R, and FIG. 10B shows a road surface R in which the vehicle 70 has an abnormality. The spectrogram of the detection signal detected when traveling on the vehicle is shown. As shown in FIG. 10B, when an abnormality occurs on the road surface R, a frequency component appears in the vicinity of 30,000 Hz.

Then, in the score calculation step shown in step S155 of FIG. 9, for example, when the spectrogram shown in FIG. 10B is model information indicating abnormal data, the spectrogram of the detection signal extracted by the feature amount extraction unit 173 is displayed. The likelihood indicating the likelihood that the spectrogram of the detection signal is abnormal is calculated as a score depending on how much the frequency component around 30,000 Hz is included.

〇 Detection signal management table FIG. 11 is a conceptual diagram showing an example of the detection signal management table according to the present embodiment. In the storage unit 1000, the detection signal management DB 1003 described in step S17 of FIG. 7 is configured by a detection signal management table as shown in FIG.

The detection signal management table shown in FIG. 11 is for managing the detection signal transmitted from the detection device 30 in association with the processing information transmitted from the control device 70A of the vehicle 70. In the detection signal management table, the traveling time data transmitted from the control device 70A, the detection signal, the frequency data extracted by the feature amount extraction unit 173, the score data calculated by the score calculation unit 175, and the control are displayed for each condition ID. The processing information data for each traveling time transmitted from the device 70A is associated, stored and managed. The condition ID is identification information for identifying the condition information included in the condition information management table shown in FIG. As a result, the signal data (detection signal) is stored in association with the related data (processed signal data (frequency data, score data), travel time data, and processing information data for each travel time) for each condition ID. .. In the detection signal management table, instead of the traveling time data and the processing information data for each traveling time, the traveling distance data and the processing information data for each traveling distance may be stored and managed.

FIG. 12 is a flowchart showing an example of processing for image information in the information processing apparatus according to the present embodiment.

The information processing device 10 includes an image pickup device 40 that captures an image of the road surface R, but there is a problem in terms of efficiency when determining an abnormality of the road surface R based on the image information.

That is, in order to make an abnormality judgment based on the image information, it is necessary to save a huge amount of image information including a normal image, and it takes an enormous amount of time for image processing to make an abnormality judgment.

On the other hand, if the frequency of imaging by the imaging device 40 is reduced in order to improve the efficiency of abnormality determination based on image information, there is a possibility that an image showing an important abnormal state cannot be acquired, and the accuracy of abnormality determination is lowered.

In view of the above problems, it is an object of the present embodiment to provide an information processing device 10 capable of efficiently and accurately determining an abnormality on the road surface R.

The image pickup device control unit 41 of the information processing device 10 determines whether the score calculated by the score calculation unit 175 is equal to or higher than the threshold value (step S21).

When the image pickup device control unit 41 determines in step S21 that the threshold value is equal to or higher than the threshold value, the image pickup device control unit 41 transmits an image pickup start signal for starting image pickup to the image pickup device 40 (step S22, an example of the image pickup control step). That is, the image pickup device control unit 41 controls the image pickup device 40 based on the score calculated based on the detection signal. On the other hand, if the image pickup device control unit 41 determines in step S21 that the value is less than the threshold value, the image pickup device control unit 41 ends the process.

The information processing device 10 causes the image pickup device 40 to start imaging at a timing not related to the score calculated by the score calculation unit 175, and the image pickup device control unit 41 determines that the score calculated by the score calculation unit 175 is equal to or higher than the threshold value. , The timing of transmitting the imaging end signal for ending the imaging to the imaging device 40 may be delayed.

Following step S22, the image pickup device control unit 41 acquires image data of a moving image or a still image captured by the image pickup device 40 from the image pickup device 40 (step S23, an example of an image information acquisition step).

The storage / reading unit 19 of the information processing device 10 associates the image data of the moving image or still image acquired by the image pickup device control unit 41 with the processing information transmitted from the control device 70A of the vehicle 70, and associates the detection signal management DB 1003 with the processing information transmitted from the control device 70A of the vehicle 70. (Step S24).

The information processing apparatus 10 sets the image data of the moving image or the still image in the image information management table in association with the traveling time included in the processing information received in step S11 for each condition ID generated in step S12 shown in FIG. Store and manage in. The information processing device 10 may store and manage the mileage data instead of the mileage data.

The score calculation unit 175 of the signal processing unit 17 of the information processing device 10 is an example of the second possibility information indicating the possibility of an abnormality of the road surface R from the image data of the moving image or the still image acquired by the image pickup device control unit 41. The second score of is calculated (step S25).

Specifically, the score calculation unit 175 calculates the likelihood indicating the likelihood that the image data is normal by using the model information indicating the normal data stored in the model information management DB 1005 of the storage unit 1000. , The inverse of the likelihood is calculated as the second score.

The score calculation unit 175 may calculate by ranking the second score step by step, or may calculate by two values of 0 and 1.

Further, the score calculation unit 175 may calculate the likelihood indicating the likelihood that the image data is abnormal as a second score using the model information indicating the abnormal data stored in the model information management DB 1005. good.

The determination unit 21 of the information processing device 10 determines whether the second score calculated by the score calculation unit 175 is equal to or higher than the threshold value (step S26), and if it is determined to be equal to or higher than the threshold value, determines that the road surface R is abnormal (step S26). Step S27). That is, the determination unit 21 determines the abnormality of the road surface R based on the score and the second score indicating the possibility of the abnormality of the road surface R determined based on the image information.

When the information processing device 10 determines that the road surface R is abnormal, the information processing device 10 outputs an alarm by display or sound using the display 106a, the speaker 109a, or the like.

As described above, the information processing device 10 includes an image pickup device control unit 41 that acquires image information of the road surface R imaged by the image pickup device 40, and the image pickup device control unit 41 is based on the detection signal. To control.

As a result, the information processing device 10 can determine the timing of acquiring the image information of the road surface R based on the detection signal, so that the image information of the road surface R can be efficiently acquired.

The image pickup device control unit 41 controls the image pickup device 40 based on a score indicating the possibility of an abnormality on the road surface R determined based on the detection signal.

As a result, the information processing device 10 can efficiently acquire the image information of the road surface R when there is a high possibility that the road surface R is abnormal based on the score.

The information processing device 10 includes a determination unit 21 that determines an abnormality on the road surface R based on the score and image information. Specifically, the determination unit 21 determines the abnormality of the road surface R based on the score and the second score indicating the possibility of the abnormality of the road surface R determined based on the image information.

As described above, the information processing apparatus 10 can efficiently and accurately determine the abnormality of the road surface R.

FIG. 13 is a conceptual diagram showing an example of an image information management table according to the present embodiment.

In the storage unit 1000, the image information management DB 1007 described with reference to FIG. 5 is configured by an image information management table as shown in FIG.

The image information management table shown in FIG. 13 is for managing the image data of the moving image or the still image acquired by the image pickup apparatus control unit 41.

In the image information management table, traveling time data and moving image or still image image data transmitted from the control device 70A of the vehicle 70 are stored and managed for each condition ID in association with each other. The condition ID is identification information for identifying the condition information included in the condition information management table shown in FIG. The image information management table may store and manage mileage data instead of mileage data.

Here, the detection signal management table shown in FIG. 11 has signal data (detection signal), related data (processed signal data (frequency data, score data), running time data, and running time for each condition ID. The condition ID that is stored and stored in association with the processing information data) and is stored and managed by the detection signal management table is the same as the condition ID that is stored and managed by the image information management table.

Therefore, the information processing device 10 includes the detection signal management table shown in FIG. 11 and the image information management table shown in FIG. 13, so that the image data of the moving image or the still image and the signal data (detection) are provided for each condition ID. The signal) and the related data (processed signal data (frequency data, score data), running time data, and processing information data for each running time) are stored in association with each other.

〇Signal data display processing and model information storage processing 〇 FIG. 14 is a flowchart showing an example of score display processing and model information storage processing in the information processing apparatus according to the present embodiment.

In step S31, the display control unit 14 of the information processing apparatus 10 causes the display 106a to display the output signal selection screen 200. Specifically, the display control unit 14 displays the output signal selection screen 200 when the reception unit 13 receives an input for a predetermined input screen displayed on the display 106a.

In step S32, when the user inputs an output item, the reception unit 13 receives a signal selection request including the input output item data.

In step S33, the information processing apparatus 10 executes a score display process and a signal data selection process to be stored as model information based on the output item data received by the reception unit 13 (an example of the selection step).

In step S34, the storage / reading unit 19 stores the selected signal data as model information in the model information management DB 1005.

FIG. 15 is a diagram showing an example of an output signal selection screen displayed on the information processing apparatus according to the present embodiment.

FIG. 15 is a diagram showing an example of an output signal selection screen displayed on the information processing apparatus corresponding to steps S31 and S32 of the flowchart shown in FIG. The output signal selection screen 200 shown in FIG. 15 is a display screen for allowing the user to select signal data to be stored as a score to be displayed and model information. On the output signal selection screen 200, an output item selection area 210 for specifying a score to be displayed and a detection signal to be stored as model information, a "VIEW" button 225 for displaying the score, and a button for canceling processing are pressed. The "CANCEL" button 203 to be performed is included.

Here, in the output item selection area 210, data of various items included in the processing information can be selected. The output item selection area 210 includes, for example, a road selection area 211 in which a road can be selected, a pavement material selection area 212 in which a pavement material can be selected, and a tire selection area 213 in which a tire 50 can be selected. In the example of FIG. 15, the reception unit 13 receives data of road; “R101”, pavement material; “asphalt”, and tire; “A” as output item data. The items corresponding to each selection area included in the output item selection area 210 are not limited to this, and may be added and changed as appropriate according to the items shown in the processing information.

Then, by pressing the "VIEW" button 225 shown in FIG. 15, display processing is performed on the signal data selected in the output item selection area 210.

In FIGS. 14 and 15 described above, an example has been described in which the information processing device 10 displays the output signal selection screen 200 and the user selects the detection signal to be output, but the control device 70A of the vehicle 70 selects the output signal. The screen 200 may be displayed so that the user can select the detection signal to be output.

FIG. 16 is a flowchart showing an example of display selection processing in the information processing apparatus according to the present embodiment. FIG. 16 is a flowchart showing an example of the display selection process corresponding to step S33 of the flowchart shown in FIG.

In step S331, the selection unit 18 selects the processing information corresponding to the output item data received in step S32 shown in FIG. 14 from the processing information stored in the condition information management table shown in FIG. Specifically, the storage / reading unit 19 reads the condition information management table from the condition information management DB 1001. Then, the selection unit 18 selects the condition information including the processing information corresponding to the output item data received by the reception unit 13 from the condition information included in the read condition information management table. In this case, the selection unit 18 selects, for example, the condition information of the condition ID; “A000001”, which is the processing information corresponding to the output item data input to the output item selection area 210 shown in FIG.

In step S332, the selection unit 18 has a signal associated with the same condition ID as the condition ID associated with the processing information selected by step S331 among the data stored in the detection signal management table shown in FIG. Select data and related data. Specifically, the storage / reading unit 19 reads the detection signal management table from the detection signal management DB 1003. Then, the selection unit 18 selects the signal data and the related data associated with the condition ID included in the selected condition information from the data included in the read detection signal management table. In this case, the selection unit 18 selects, for example, the signal data and the related data associated with the condition ID; “A000001”.

In step S333, the selection unit 18 has the image associated with the same condition ID as the condition ID associated with the processing information selected by step S331 among the data stored in the image information management table shown in FIG. Select data. Specifically, the storage / reading unit 19 reads the image information management table from the image information management DB 1007. Then, the selection unit 18 selects the image data associated with the condition ID included in the selected condition information from the data included in the read image information management table. In this case, the selection unit 18 selects, for example, the image data associated with the condition ID; “A000001”.

In step S334, the display control unit 14 of the information processing device 10 causes the display 106a to display the information display selection screen 250. Then, the display control unit 14 displays the signal data and related data selected in step S332 and the image data selected in step S333 in association with each other via the condition ID on the information display selection screen 250.

In step S335, when the user selects a section on the information display selection screen 250, the reception unit 13 accepts the selection of the section (an example of the reception step).

In step S336, the selection unit 18 selects the signal data and related data related to the section selected by step S335 among the signal data and related data selected by step S332.

FIG. 17 is a diagram showing an example of an information display selection screen displayed on the information processing apparatus according to the present embodiment. FIG. 18 is a diagram showing an example of image information displayed on the display selection screen according to the present embodiment.

FIG. 17 is a diagram showing an example of an information display selection screen displayed on the information processing apparatus corresponding to steps S334 and S335 of the flowchart shown in FIG. The information display selection screen 250 shown in FIG. 17 is a display screen for displaying the score of the detection signal related to the signal data selected by the selection unit 18 and selecting a specific section of the detection signal.

The user can confirm the possibility of the road surface R being abnormal by checking the score displayed on the information display selection screen 250, but it is necessary to visually check the road surface R in order to actually determine that the road surface R is abnormal. There was a problem in terms of accuracy of abnormality judgment.

In order to visually confirm the road surface R, the image data of the vehicle 70 is displayed on the information display selection screen 250, but it is difficult to visually confirm the image data indicating an abnormality from a large amount of image data. Is. Therefore, it is conceivable that the information processing device 10 performs image processing on the image data to determine an abnormality, but in that case, it is necessary to store a huge amount of image information, and it takes an enormous amount of time to determine the abnormality by the image processing. Therefore, there is a problem in terms of efficiency.

In view of the above problems, it is an object of the present embodiment to provide an information processing device 10 capable of efficiently and accurately determining an abnormality on the road surface R. ..

On the information display selection screen 250, the traveling time data related to the related data selected by the selection unit 18 is displayed in a state in which the user can confirm the traveling time data in the order of traveling time.

Further, on the information display selection screen 250, frequency spectrogram as an example of the feature amount related to the related data selected by the selection unit 18, score data, and user operation history data as an example of processing information data for each running time are displayed. , Each is displayed in a state in which the user can confirm it in succession in association with the travel time data. Further, the image data A is displayed in a state in which the user can confirm it in association with the score of the maximum value.

That is, the display control unit 14 displays a plurality of traveling times, frequency spectra, operation history information, and scores on the information display selection screen 250 of the display 106a, and a plurality of traveling times are associated with the respective traveling times of the plurality of traveling times. Each frequency spectrum of the frequency spectrum, each operation history information of the plurality of operation history information, each score of the plurality of scores, and image information are displayed on the information display selection screen 250 (an example of a display step). In the example of FIG. 17, the display control unit 14 connects a plurality of scores and displays them as a line graph, and displays image information in association with the positions of the line graph showing the plurality of scores in the horizontal axis direction. There is.

Further, the display control unit 14 displays the image information on the information display selection screen 250 in association with the score of the maximum value among the plurality of scores. Here, the maximum value score is a score equal to or higher than the threshold value described in step S21 of FIG.

Further, on the information display selection screen 250, an input area 222 for inputting a traveling time section as a specific section, an "OK" button 251 pressed when performing section selection processing, and a case of canceling section selection processing. Includes a "CANCEL" button 253 that is pressed against.

As shown in FIG. 17, among the scores of the traveling time 1001 to 1250, the score increases after the traveling time 1230, and the score near the traveling time 1250 is the highest.

Here, the image data A of the road surface R is displayed on the information display selection screen 250 in association with the score, and the user can display the image data A of the road surface R corresponding to the maximum score near the traveling time of 1250. You can check.

In the case of the example of FIG. 17, since the image data of the road surface R contains more cracks than usual, the user can determine that the cause of the high score is the cracks.

That is, by confirming the score and the image information associated with the score, it is possible to accurately determine the abnormality of the road surface R.

The image data may be displayed not only in association with the maximum score, but also in association with all of the plurality of scores equal to or greater than the threshold value described in step S21 of FIG. In that case, if all the image data is displayed at the same time, the screen becomes complicated. Therefore, among the image data associated with each of the plurality of scores, only the image data associated with the score selected with the mouse or the like should be displayed. Is preferable. Further, when the change in the score is large, the image data may be displayed in association with the score before, after the change, or before and after the change.

Further, FIG. 17 shows an example of selecting normal data from the data of traveling times 1001 to 1250. Looking at the score as an example of the score data, it can be seen that the values are low for the running times 1001 to 1230. If the system automatically sets "a part with a small score is normal" or "a cycle with a young cycle (for example, 1001 to 1105) is normal", this section can be selected as normal data.

However, looking at the frequency data in the section of traveling time 1001 to 1230, it can be seen that noise is present in the low frequency range. On the other hand, looking at the frequency data of the section of the traveling time 1105-1207, it can be seen that the low frequency noise is not included as in the section of the traveling time 1001-1230.

Therefore, the user can appropriately determine that the data of the section of the traveling time 1001 to 1230, which is a part of the traveling time 1001 to 1230, is more suitable as normal data than the data of the section of the traveling time 1001 to 1230.

Further, looking at the user operation history in the section of the traveling time of 1001 to 1230, it can be seen that the operation in the special mode, for example, the frequent speed change is repeated. It can be seen that the above-mentioned low-frequency noise is generated in the execution section of the operation in the special mode. From this, it can be judged that the execution section of the operation in the special mode is not suitable for selecting normal data. Therefore, the user can more appropriately determine that the traveling time 1105 to 1207 in which the operation in the special mode is not performed is suitable as normal data.

Then, when the user inputs "1105-1207" in the input area 222 and presses the "OK" button 251, the reception unit 13 accepts the selection of "travel time 1105-1207". As a result, the detection signal or frequency spectrum corresponding to "travel time 1105-1207", which is a part of the travel time 1001 to 1230, can be appropriately set as normal data.

Further, since the information "travel time 1105-1207" received by the reception unit 13 is displayed in the input area 222, the user can check the content displayed in the input area 222 and check the "travel time 1105-1207". The detection signal or frequency spectrum corresponding to "1207" can be reliably set as normal data.

In the above description, when "1105-1207" is input to the input area 222 and the "OK" button 251 is pressed, the reception unit 13 accepts the selection of "travel time 1105-1207". Of the displayed frequency data, score, and user operation history, by selecting the "running time 1105-1207" part with a mouse or the like and pressing the "OK" button 251, the reception unit 13 can "running time". 1105-1207 ”may be accepted.

As described above, the information processing apparatus 10 is a display control for displaying the score indicating the possibility of abnormality of the road surface R determined based on the detection signal and the image information on the information display selection screen 250 of the display 106a in association with each other. A unit 14 is provided.

As a result, the user can accurately determine the abnormality of the road surface R by checking the image information in association with the score, as compared with the case of checking only the score. Further, the user can more efficiently determine the abnormality of the road surface R as compared with the case of confirming only the image information. That is, the user can efficiently and accurately determine the abnormality of the road surface R by the information processing device 10 according to the present embodiment.

The display control unit 14 displays a plurality of frequency spectra of the detection signal on the information display selection screen 250, and displays and selects each frequency spectrum of the plurality of frequency spectra in association with each traveling time of the plurality of traveling times. Display on screen 250.

As a result, the user can check the respective running times of the plurality of running times, the respective frequency spectra of the plurality of frequency spectra, and the image information in association with the respective scores of the plurality of scores. For each score of the score, the abnormality of the road surface R can be efficiently and accurately determined.

The display control unit 14 displays a plurality of operation history information on the information display selection screen 250, and displays and selects each operation history information of the plurality of operation history information in association with each travel time of the plurality of travel times. Display on screen 250.

As a result, the user confirms each running time of the plurality of running times, each operation history information of the plurality of operation history information, and image information in association with each score of the plurality of scores. It is possible to efficiently and accurately determine the abnormality of the road surface R for each score of the plurality of scores.

The information processing device 10 includes a reception unit 13 that receives input of a part of a plurality of traveling times in a plurality of traveling times.

As a result, the user can input a part of a plurality of running times in a plurality of running times after confirming the score and the image information associated with the score on the display 106a. Therefore, it is possible to appropriately set each frequency spectrum of the plurality of frequency spectra corresponding to the respective traveling times of a plurality of plurality of traveling times as normal data or abnormal data.

The information processing device 10 causes the display control unit 14 to display a plurality of traveling times received by the reception unit 13 on the information display selection screen 250. As a result, the user can confirm a plurality of input running times, and therefore, normal data is obtained for each frequency spectrum of the plurality of frequency spectra corresponding to the respective running times of some of the plurality of running times. Alternatively, it can be reliably set as abnormal data.

Further, as described above, the information processing device 10 includes a display control unit 14 for displaying image information on the information display selection screen 250 of the display 106a, and is imaged by the image pickup device 40 as described with reference to FIG. An image pickup device control unit 41 for acquiring image information of the road surface R is provided, and the image pickup device control unit 41 controls the image pickup device 40 based on a detection signal.

As a result, the information processing device 10 can determine the timing of acquiring the image information of the road surface R based on the detection signal, so that the image information of the road surface R can be efficiently acquired. Further, the user can accurately determine the abnormality of the road surface R by checking the image information on the information display selection screen 250. That is, the user can efficiently and accurately determine the abnormality of the road surface R by the information processing device 10 according to the present embodiment.

The image pickup device control unit 41 controls the image pickup device 40 based on a score indicating the possibility of an abnormality on the road surface R determined based on the detection signal.

As a result, the information processing device 10 can efficiently acquire the image information of the road surface R when there is a high possibility that the road surface R is abnormal based on the score.

The display control unit 14 displays the score on the information display selection screen 250. As a result, the user can accurately determine the abnormality of the road surface R by checking the score and the image information on the information display selection screen 250.

The display control unit 14 displays the image information on the information display selection screen 250 in association with the score. As a result, the user can accurately determine the abnormality of the road surface R by checking the score and the image information in association with each other on the information display selection screen 250.

FIG. 19 is a conceptual diagram showing an example of a model information management table according to the present embodiment. As described in step S34 of FIG. 14, the storage unit 1000 is constructed with a model information management DB 1005 composed of a model information management table as shown in FIG.

In the model information management table, the storage / reading unit 19 stores and manages a plurality of travel time data, detection signals, frequency data, and score data in association with each related ID.

Then, the storage / reading unit 19 stores and updates the detection signal and the frequency data associated with each of the traveling times of a part of the plurality of traveling times received by the reception unit 13. ..

In the example of FIG. 19, as the model information of the related ID; “R001”, the travel time data received by the reception unit 13 described with reference to FIG. , Score data is associated, stored and managed. Thereby, the detection signal, the frequency data, and the score data corresponding to the traveling times 1105 to 1207, which are a part of the traveling times 1001 to 1230, can be appropriately stored as normal data.

In the model information management table, the detection signal, the frequency data, and the score data may be associated with each related ID and stored and managed in chronological order without storing the traveling time data. Further, without storing the frequency data and the score data in the model information management table, the processes of steps S154 and S155 of FIG. 9 are performed again based on the detection signal stored in the model information management table to perform the frequency. Data may be extracted and score data may be calculated.

As described above, the information processing apparatus 10 stores the frequency spectra of the plurality of frequency spectra associated with the respective travel times of the plurality of travel times received by the reception unit 13 in the storage unit 1000. A storage / reading unit 19 for storing is provided.

Thereby, each frequency spectrum of the plurality of frequency spectra corresponding to the respective traveling times of a plurality of plurality of traveling times can be appropriately stored as normal data or abnormal data.

As described with reference to FIG. 9, the information processing device 10 determines the score based on the plurality of frequency spectra stored in the storage unit 1000 and the frequency spectra showing the characteristics of the detection signal acquired by the detection device communication unit 12. The score calculation unit 175 is provided.

Here, as described with reference to FIG. 19, since the plurality of frequency spectra stored in the storage unit 1000 are updated, the information processing apparatus 10 can accurately determine the score indicating the possibility of abnormality of the road surface R. can.

FIG. 20 is a diagram showing an example of an information display selection screen displayed on the information processing apparatus according to the first modification of the present embodiment. FIG. 21 is a diagram showing an example of image information displayed on the display selection screen according to the first modification of the present embodiment.

Similar to FIG. 17, the information display selection screen 250 displays the frequency spectrogram, the score data, and the user operation history data in a state in which a plurality of them can be continuously confirmed by the user in association with the traveling time data. There is. Further, the image data is displayed in a state in which the user can confirm it in association with the score of the maximum value. In the example of FIG. 20, since there are two maximum scores, the image data A and B are displayed in association with the respective scores of the two maximum scores. Further, the image data A and B are image data A1 and B1 of the road surface R on the traveling direction side of the vehicle 70 imaged by the first image pickup device 40A and a vehicle imaged from the vertical direction by the second image pickup device 40B. The image data A2 and B2 of the road surface R behind 70 are included.

That is, the display control unit 14 displays a plurality of scores and image information on the information display selection screen 250 of the display 106a, and associates the scores with the respective scores of the plurality of scores to display the respective image information of the plurality of image information. It is displayed on the information display selection screen 250.

FIG. 20 shows an example of selecting abnormal data from the data of traveling times 801-900. Looking at the score as an example of the score data, it can be seen that the values around the running time 840 and after the running time 865 are high. If the system automatically sets "a place with a high score is normal", these two sections can be selected as abnormal data.

However, looking at the frequency data immediately before the running time 840, it can be seen that the data is temporarily interrupted. Further, it can be seen that in the vicinity of the traveling time of 840, the frequency data has the same value as usual. That is, it is considered unlikely that an abnormality in the road surface R has occurred because the score indicating the change in the frequency data has increased due to the interruption of the frequency data.

On the other hand, after the traveling time of 865, the frequency data shows that abnormal vibration is present in the low frequency range, and it is highly possible that some abnormality has occurred on the road surface R. From this, the user can appropriately determine that the data of the traveling time 856-900, which is a part of the number of traveling times 801 to 900, is more suitable as the abnormal data than the data of the traveling time of around 840.

Further, looking at the user operation history immediately before the running time 840, it can be seen that the power is turned off and then on. It is considered that the high score immediately after this is due to the influence of warm-up operation. On the other hand, after the traveling time of 865, although there is no evidence that the user has operated the vehicle, it is highly possible that the frequency data shows abnormal vibration in the low frequency range and some abnormality has occurred on the road surface R. From this, the user can more appropriately determine that the data of the traveling time 856-900, which is a part of the traveling time 801 to 900, is more suitable as abnormal data than the data of the traveling time of around 840.

Then, the user can change the state of the road surface R shown in the image data corresponding to the maximum score near the traveling time 840 and the state of the road surface R shown in the image data corresponding to the maximum score after the traveling time 865. By comparing, it can be confirmed from the image data that the data of the traveling time 856-900, which is a part of the traveling time 801 to 900, is suitable as the abnormal data.

That is, by confirming the score and the image information associated with the score, it is possible to accurately determine the abnormality of the road surface R.

Although the example in which the traveling time is used has been described in FIG. 20, the same applies when the traveling distance is used instead of the traveling time.

As described above, the display control unit 14 displays a plurality of scores and image information on the information display selection screen 250, and associates the scores with the scores of the plurality of scores to provide the image information of the plurality of image information. Is displayed on the information display selection screen 250.

As a result, the user can accurately identify the abnormality of the road surface R for each score of the plurality of scores by checking each image information of the plurality of image information in association with each score of the plurality of scores. You can judge.

The display control unit 14 displays a plurality of travel times on the information display selection screen 250, and associates the travel times with the travel times of the plurality of travel times to correspond to the respective travel times of the plurality of scores and the respective scores of the plurality of image information. The image information is displayed on the information display selection screen 250.

As a result, the user can confirm each running time of the plurality of running times and each image information of the plurality of image information in association with each score of the plurality of scores, thereby making each of the plurality of scores. It is possible to accurately determine the abnormality of the road surface R for each score.

FIG. 22 is a diagram showing an example of an information display selection screen displayed on the information processing apparatus according to the second modification of the present embodiment.

On the information display selection screen 250, score data is displayed in a state in which a plurality of score data can be continuously confirmed by the user in association with the travel time data. Further, as in FIG. 1920, the image data A and B are displayed in association with the respective scores of the scores of the two maximum values, and the image data A and B are imaged by the first imaging device 40A. The image data A1 and B1 of the road surface R on the traveling direction side of the vehicle 70 and the image data A2 and B2 of the road surface R behind the vehicle 70 imaged from the vertical direction by the second imaging device 40B are included.

FIG. 22 shows an example of selecting a plurality of abnormal data from the traveling times 801-900.
Looking at the score as an example of the score data, it can be seen that the score between 801 and 900 of the running time is higher by two points. If the system automatically sets "a place with a high score is normal", these two sections can be selected as abnormal data.

However, here, it may be possible to compare the image data corresponding to the scores at these two locations and determine that they tend to be completely different. In this case, if these two different abnormality data are referred to as abnormality 1 and abnormality 2, respectively, the user can determine each as separate abnormality data.

In FIG. 22, an example in which the traveling time is used has been described, but the same applies when the traveling distance is used instead of the traveling time.

FIG. 23 is a diagram showing a railroad vehicle according to a third modification of the present embodiment.

The railroad vehicle 70R is a modification of the vehicle 70 shown in FIG. 2, and includes a control device 70A, wheels 50w, a detection device 30, and an image pickup device 40 arranged so as to image the track r.

The railroad vehicle 70R is an example of a moving body that moves while contacting the railroad track r, which is an example of the target portion. The wheel 50w is an example of a moving member that moves while contacting the track r. The detection device 30 detects a physical quantity that changes as the wheels 50w of the railway vehicle 70R move while in contact with the track r. The image pickup device 40 is an example of an image pickup unit that captures an image of a moving image or a still image of a track r.

FIG. 24 is a conceptual diagram showing an example of the condition information management table according to the third modification.

In the storage unit 1000 shown in FIG. 5, the condition information management DB 1001 described in step S13 of FIG. 7 is configured by the condition information management table as shown in FIG. 24.

The condition information management table shown in FIG. 24 is for managing processing information related to the operation of the railway vehicle 70R for each operation performed by the railway vehicle 70R. In the condition information management table, condition information associated with processing information for each condition ID is stored and managed. The condition ID is identification information for identifying the condition information including the processing information. The processing information is information defined for each type of operation of the railway vehicle 70R. As shown in FIG. 24, the processing information includes information on the type of the line on which the railroad vehicle 70R travels, the type of the material of the railroad r on which the railroad vehicle 70R travels, and the type of the material of the wheels 50w of the railroad vehicle 70R. included. The type of the material of the track r is an example of the type of the target portion, and the type of the material of the wheel 50w is an example of the type of the moving member.

The items included in the processing information further include information indicating the user's operation history information for the railway vehicle 70R, the travel start date and time, the travel time (an example of the travel time of the railway vehicle 70R), the travel speed of the railway vehicle 70R, and the like. It may be. Further, the mileage (an example of the moving distance of the railway vehicle 70R) may be included, and the mileage may be included instead of the traveling time. Travel information including travel time or travel distance is an example of travel information.

The condition information management table is further stored and managed in association with a related ID for identifying a related operation (process) among the operations by the railway vehicle 70R. As the related ID, the same related ID is assigned to the processing information indicating the related processing among the processing information included in the condition information management table. In the example of FIG. 24, the same related ID; "R001" is assigned to the processing information identified by the condition ID; "A000001" and the condition ID; "A00003". Here, the related process to which the related ID is assigned is, for example, a process in which the track type and the wheel type are the same but the track names are different.

FIG. 25 is a diagram showing an example of a display selection screen displayed on the information processing apparatus according to the third modification.

FIG. 25 is a diagram showing an example of an information display selection screen displayed on the information processing apparatus corresponding to steps S334 and S335 of the flowchart shown in FIG. The information display selection screen 250 shown in FIG. 25 is a display screen for displaying the score of the detection signal related to the signal data selected by the selection unit 18 and selecting a specific section of the detection signal.

On the information display selection screen 250, the traveling time data related to the related data selected by the selection unit 18 is displayed in a state in which the user can confirm the traveling time data in the order of traveling time.

Further, on the information display selection screen 250, frequency spectrogram as an example of the feature amount related to the related data selected by the selection unit 18, score data, and user operation history data as an example of processing information data for each running time are displayed. , Each is displayed in a state in which the user can confirm it in succession in association with the travel time data. Further, the image data A of the line r is displayed in a state in which the user can confirm it in association with the score of the maximum value.

That is, the display control unit 14 displays a plurality of traveling times, frequency spectra, operation history information, and scores on the information display selection screen 250 of the display 106a, and a plurality of traveling times are associated with the respective traveling times of the plurality of traveling times. Each frequency spectrum of the frequency spectrum, each operation history information of the plurality of operation history information, each score of the plurality of scores, and image information are displayed on the information display selection screen 250 (an example of a display step).

Further, the display control unit 14 displays the image information on the information display selection screen 250 in association with the score of the maximum value among the plurality of scores. Here, the maximum value score is a score equal to or higher than the threshold value described in step S21 of FIG.

Further, on the information display selection screen 250, an input area 222 for inputting a traveling time section as a specific section, an "OK" button 251 pressed when performing section selection processing, and a case of canceling section selection processing. Includes a "CANCEL" button 253 that is pressed against.

As described above, the information processing apparatus 10 has a display control for displaying the score indicating the possibility of abnormality of the line r determined based on the detection signal and the image information on the information display selection screen 250 of the display 106a in association with each other. A unit 14 is provided.

As a result, the user can accurately determine the abnormality of the line r by checking the image information in association with the score, as compared with the case of checking only the score. Further, the user can more efficiently determine the abnormality of the line r as compared with the case of confirming only the image information. That is, the user can efficiently and accurately determine the abnormality of the line r by the information processing device 10 according to the present embodiment.

FIG. 26 is a diagram showing a railroad vehicle according to a fourth modification of the present embodiment.

The railway vehicle 70R shown in FIG. 26 is a modification of the railway vehicle 70R shown in FIG. 23, and includes a control device 70A, a pantograph 50P, a detection device 30, and an image pickup device 40 arranged so as to image an overhead wire W.

The railroad vehicle 70R is an example of a moving body that moves while contacting the overhead wire W, which is an example of the target portion. The pantograph 50P is an example of a moving member that moves while contacting the overhead wire W. The detection device 30 detects a physical quantity that changes as the pantograph 50P of the railway vehicle 70R moves while contacting the overhead wire W. The image pickup device 40 is an example of an image pickup unit that captures an image of a moving image or a still image of the overhead wire W.

FIG. 27 is a conceptual diagram showing an example of the condition information management table according to the fourth modification.

In the storage unit 1000 shown in FIG. 5, the condition information management DB 1001 described in step S13 of FIG. 7 is configured by the condition information management table as shown in FIG. 27.

The condition information management table shown in FIG. 27 is for managing processing information related to the operation of the railway vehicle 70R for each operation performed by the railway vehicle 70R. In the condition information management table, condition information associated with processing information for each condition ID is stored and managed. The condition ID is identification information for identifying the condition information including the processing information. The processing information is information defined for each type of operation of the railway vehicle 70R. As shown in FIG. 24, the processing information includes information on the line name on which the railroad vehicle 70R travels, the type of the material of the overhead wire W, and the type of the material of the pantograph 50P of the railroad car 70R. The type of the material of the overhead wire W is an example of the type of the target portion, and the type of the material of the pantograph 50P is an example of the type of the moving member.

The items included in the processing information further include information indicating the user's operation history information for the railway vehicle 70R, the travel start date and time, the travel time (an example of the travel time of the railway vehicle 70R), the travel speed of the railway vehicle 70R, and the like. It may be. Further, the mileage (an example of the moving distance of the railway vehicle 70R) may be included, and the mileage may be included instead of the traveling time. Travel information including travel time or travel distance is an example of travel information.

The condition information management table is further stored and managed in association with a related ID for identifying a related operation (process) among the operations by the railway vehicle 70R. As the related ID, the same related ID is assigned to the processing information indicating the related processing among the processing information included in the condition information management table. In the example of FIG. 27, the same related ID; "R001" is assigned to the processing information identified by the condition ID; "A000001" and the condition ID; "A00003". Here, the related processing to which the related ID is assigned is, for example, a processing in which the overhead line type and the pantograph type are the same but the route names are different.

FIG. 28 is a diagram showing an example of a display selection screen displayed on the information processing apparatus according to the fourth modification.

FIG. 28 is a diagram showing an example of an information display selection screen displayed on the information processing apparatus corresponding to steps S334 and S335 of the flowchart shown in FIG. The information display selection screen 250 shown in FIG. 28 is a display screen for displaying the score of the detection signal related to the signal data selected by the selection unit 18 and selecting a specific section of the detection signal.

On the information display selection screen 250, the traveling time data related to the related data selected by the selection unit 18 is displayed in a state in which the user can confirm the traveling time data in the order of traveling time.

Further, on the information display selection screen 250, frequency spectrogram as an example of the feature amount related to the related data selected by the selection unit 18, score data, and user operation history data as an example of processing information data for each running time are displayed. , Each is displayed in a state in which the user can confirm it in succession in association with the travel time data. Further, the image data A of the overhead wire W is displayed in a state in which the user can confirm it in association with the score of the maximum value.

That is, the display control unit 14 displays a plurality of traveling times, frequency spectra, operation history information, and scores on the information display selection screen 250 of the display 106a, and a plurality of traveling times are associated with the respective traveling times of the plurality of traveling times. Each frequency spectrum of the frequency spectrum, each operation history information of the plurality of operation history information, each score of the plurality of scores, and image information are displayed on the information display selection screen 250 (an example of a display step).

Further, the display control unit 14 displays the image information on the information display selection screen 250 in association with the score of the maximum value among the plurality of scores. Here, the maximum value score is a score equal to or higher than the threshold value described in step S21 of FIG.

Further, on the information display selection screen 250, an input area 222 for inputting a traveling time section as a specific section, an "OK" button 251 pressed when performing section selection processing, and a case of canceling section selection processing. Includes a "CANCEL" button 253 that is pressed against.

As described above, the information processing apparatus 10 displays the score indicating the possibility of abnormality of the overhead wire W determined based on the detection signal on the information display selection screen 250 of the display 106a in association with the image information of the overhead wire W. The display control unit 14 is provided.

As a result, the user can accurately determine the abnormality of the overhead wire W by checking the image information in association with the score, as compared with the case of checking only the score. Further, the user can more efficiently determine the abnormality of the overhead wire W as compared with the case of confirming only the image information. That is, the user can efficiently and accurately determine the abnormality of the overhead wire W by the information processing device 10 according to the present embodiment.

● Summary ●
As described above, the information processing device 10 according to the embodiment of the present invention moves while the moving bodies such as the vehicle 70 and the railroad vehicle 70R are in contact with the target portions such as the road surface R, the line r, and the overhead line W. The detection device communication unit 12 (an example of the detection information acquisition unit) that acquires the detection signal (an example of the detection information) of the physical amount that changes by the operation, and the road surface R or the like imaged by the image pickup device 40 (an example of the image pickup unit). The image pickup device control unit 41 includes an image pickup device control unit 41 (an example of an image information acquisition unit) for acquiring image information and a display control unit 14 for displaying image information on a display 106a (an example of a display unit). The image pickup apparatus 40 is controlled based on the detection signal.

As a result, the information processing device 10 can determine the timing of acquiring image information such as the road surface R based on the detection signal, so that the image information such as the road surface R can be efficiently acquired. Further, the user can accurately determine an abnormality such as the road surface R by checking the image information on the display 106a. That is, the user can efficiently and accurately determine an abnormality such as a road surface R by the information processing device 10 according to the present embodiment.

The image pickup device control unit 41 controls the image pickup device 40 based on a score (an example of possibility information) indicating the possibility of an abnormality such as a road surface R determined based on a detection signal.

As a result, the information processing device 10 can efficiently acquire image information such as the road surface R when there is a high possibility of an abnormality such as the road surface R based on the score.

The display control unit 14 causes the score to be displayed on the display 106a. As a result, the user can efficiently and accurately determine an abnormality such as the road surface R by checking the score and the image information on the display 106a.

The display control unit 14 causes the display 106a to display the image information in association with the score. As a result, the user can efficiently and accurately determine an abnormality such as the road surface R by checking the score and the image information in association with each other on the display 106a.

The information processing device 10 includes a determination unit 21 (an example of an abnormality determination unit) for determining an abnormality such as a road surface R based on a score and image information. Specifically, the determination unit 21 determines the road surface R based on the score and the second score (an example of the second possibility information) indicating the possibility of an abnormality such as the road surface R determined based on the image information. Judge abnormalities such as.

As a result, the information processing device 10 can efficiently and accurately determine an abnormality such as a road surface R.

● Supplement ●
The functions of each embodiment can be realized by a computer-executable program written in a legacy programming language such as an assembler, C, C ++, C #, Java (registered trademark), an object-oriented programming language, or the like. Programs for performing the functions of can be distributed through telecommunications lines.

Further, the programs for executing the functions of the respective embodiments are ROM, EPROM (Electrically Erasable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), CDisCompory, and a flash memory. Store and distribute in a device-readable recording medium such as ROM, CD-RW (Re-Writer), DVD-ROM, DVD-RAM, DVD-RW, Blu-ray disk, SD card, MO (Magnet-Optical disc). You can also.

Further, some or all of the functions of each embodiment can be implemented on a programmable device (PD) such as FPGA (Field Programmable Gate Array), or can be implemented as an ASIC, and each embodiment. Circuit configuration data (bit stream data) to be downloaded to PD in order to realize the above functions on PD, HDL (Hardware Description Language) for generating circuit configuration data, VHDL (Very High Speed Integrated Circuits Hardware) It can be distributed by a recording medium as data described by Verilog-HDL or the like.

Although the information processing apparatus, information processing method, and program according to one embodiment of the present invention have been described so far, the present invention is not limited to the above-described embodiment, and other embodiments may be added, modified, or modified. It can be changed within the range that can be conceived by those skilled in the art, such as deletion, and is included in the scope of the present invention as long as the action and effect of the present invention are exhibited in any of the embodiments.

1A, 1C Abnormality detection system 5 Information processing system 10 Information processing device 11 Transmission / reception unit (example of processing information acquisition unit)
12 Detection device communication unit (an example of detection information acquisition unit)
18 Selection unit 19 Storage / reading unit (an example of storage control unit)
21 Judgment unit (an example of abnormality judgment unit)
30 Detection device 40 Imaging device (example of imaging unit)
41 Image pickup device control unit (example of image information acquisition unit)
70 vehicle (an example of a moving body)
70R railroad vehicle (an example of a moving body)
R road surface (an example of the target part)
r track (an example of the target part)
W overhead wire (an example of the target part)
106a display (example of display unit)
173 Feature extraction unit 175 Score calculation unit (an example of possibility information determination unit)
200 Output signal selection screen 250 Information display selection screen 1000 Storage unit

Japanese Patent No. 5014464

Claims (8)

A detection information acquisition unit that acquires detection information of a physical quantity that changes when a moving object moves while contacting the target unit.
An image information acquisition unit that acquires image information of the target unit imaged by the image pickup unit, and an image information acquisition unit.
A display control unit for displaying the image information on the display unit is provided.
An information processing device that controls the imaging unit based on the detection information. The information processing apparatus according to claim 1, wherein the image pickup unit is controlled based on the possibility information indicating the possibility of abnormality of the target unit determined based on the detection information. The information processing device according to claim 2, wherein the display control unit displays the possibility information on the display unit. The information processing device according to claim 3, wherein the display control unit displays the image information on the display unit in association with the possibility information. The information processing apparatus according to any one of claims 1 to 4, further comprising an abnormality determination unit that determines an abnormality in the target unit based on the possibility information and the image information. 5. The abnormality determination unit determines an abnormality in the target portion based on the possibility information and a second possibility information indicating the possibility of the abnormality in the target portion determined based on the image information. The information processing device described. A detection information acquisition step for acquiring detection information of a physical quantity that changes as the moving body moves while contacting the target portion,
An image information acquisition step of acquiring image information of the target portion captured by the imaging unit, and
A display step for displaying the image information on the display unit, and
An imaging control step that controls the imaging unit based on the detection information,
Information processing method with. A program that causes a computer to execute the information processing method according to claim 7.

Images