WO2024070532A1 - 情報処理装置、情報処理方法、プログラムおよび情報処理システム - Google Patents
情報処理装置、情報処理方法、プログラムおよび情報処理システム Download PDFInfo
- Publication number
- WO2024070532A1 WO2024070532A1 PCT/JP2023/032361 JP2023032361W WO2024070532A1 WO 2024070532 A1 WO2024070532 A1 WO 2024070532A1 JP 2023032361 W JP2023032361 W JP 2023032361W WO 2024070532 A1 WO2024070532 A1 WO 2024070532A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- image
- data
- slope
- unit
- information processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- E—FIXED CONSTRUCTIONS
- E01—CONSTRUCTION OF ROADS, RAILWAYS, OR BRIDGES
- E01C—CONSTRUCTION OF, OR SURFACES FOR, ROADS, SPORTS GROUNDS, OR THE LIKE; MACHINES OR AUXILIARY TOOLS FOR CONSTRUCTION OR REPAIR
- E01C23/00—Auxiliary devices or arrangements for constructing, repairing, reconditioning, or taking-up road or like surfaces
- E01C23/01—Devices or auxiliary means for setting-out or checking the configuration of new surfacing, e.g. templates, screed or reference line supports; Applications of apparatus for measuring, indicating, or recording the surface configuration of existing surfacing, e.g. profilographs
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Definitions
- the present invention relates to an information processing device, an information processing method, a program, and an information processing system.
- Patent Document 1 describes a method and device for creating a panoramic image that extends long in the direction of movement and is wider than the viewing angle of each line camera by repeatedly taking images with each line camera while the moving object is moving.
- the objective of the present invention is to confirm the position of a target part in an image captured by a camera installed on a moving object.
- the information processing device includes a generation means for generating a display screen that displays a composite image including the boundary between the object and objects other than the object in the moving direction of the moving body by stitching together images captured by a photographing device installed on the moving body in which the object area including the object and objects other than the object is divided into a plurality of photographing areas along the moving direction of the moving body.
- the present invention makes it possible to confirm the position of a target part in an image captured by a photographing device installed on a moving object.
- FIG. 1 is a diagram showing an example of an overall configuration of a state inspection system according to an embodiment
- FIG. 13 is a diagram showing an example of a state in which a slope condition is inspected using the mobile body system according to the embodiment.
- FIG. 13 is a diagram illustrating the state of a slope.
- FIG. 2 is a diagram illustrating an example of a hardware configuration of a data acquisition device.
- FIG. 2 is a diagram illustrating an example of a hardware configuration of an evaluation device and a data management device.
- FIG. 2 is a diagram illustrating an example of a functional configuration of a state inspection system.
- FIG. 13 is a conceptual diagram illustrating an example of a status type management table.
- FIG. 13 is a conceptual diagram illustrating an example of a status type management table.
- FIG. 1A is a conceptual diagram showing an example of an acquired data management table
- FIG. 1B is a conceptual diagram showing an example of a processed data management table.
- FIG. 2 is a diagram for explaining a captured image acquired by a mobile system. 1A and 1B are explanatory diagrams of a photographed image and a distance measurement image. FIG. 2 is an explanatory diagram of a plurality of shooting regions.
- FIG. 1 is a diagram showing a mobile system including a plurality of image capturing devices according to an embodiment.
- FIG. 11 is a sequence diagram showing an example of a data acquisition process using a mobile system.
- FIG. 11 is a sequence diagram showing an example of a generation process of evaluation target data.
- FIG. 1 is an illustration of a composite image of a state inspection system.
- FIG. 1 is an illustration of a composite image of a state inspection system.
- FIG. 13 is an explanatory diagram of operations on an input/output screen of the status inspection system.
- FIG. 13 is another explanatory diagram of operations on the input/output screen of the status inspection system.
- 19 is a flowchart showing a process based on the operations shown in FIGS. 17 and 18.
- FIG. 1 is an illustration of an integrated partial image of a condition inspection system.
- FIG. 11 is a sequence diagram showing a modified example of the process of generating evaluation target data.
- FIG. 11 is a sequence diagram showing an example of a process for generating a report that is an evaluation result of a slope condition.
- 13 is a flowchart showing an example of a process for detecting a slope state.
- FIG. 11 is a sequence diagram showing an example of a display process in the state inspection system.
- FIG. 13 is an explanatory diagram of operations on a display screen of the state inspection system.
- 26 is a flowchart showing a process based on the operation shown in FIG. 25 .
- 27 is an example of a display screen after the processing shown in FIG. 26.
- FIG. 13 is a diagram showing a modified example of the functional configuration of the state inspection system.
- 30 is a flowchart showing a process in the modified example shown in FIG. 28 .
- FIG. 30 is a diagram showing an example of a detection data display screen in the modified example shown in FIG. 28.
- FIG. 30 is a diagram showing an example of a map screen in the modified example shown in FIG. 28.
- FIG. 13 is a diagram showing an example of how a slope condition is inspected using a mobile system according to the first modified example.
- FIG. 11 is a diagram showing an example of inspecting a slope condition using a mobile system according to Modification 2.
- FIG. 11 is a diagram showing an example of inspecting a slope condition using
- Figure 1 is a diagram showing an example of the overall configuration of a condition inspection system according to an embodiment.
- the condition inspection system 1 shown in Figure 1 is an example of an information processing system, and is a system for inspecting the condition of road earthwork structures using various data acquired by a mobile system 60.
- Road earthwork structures are a general term for structures that are mainly made of ground materials such as soil and rocks constructed to build roads, and structures associated with them, and refer to cut and slope stabilization facilities, embankments, culverts, and similar structures.
- slopes road earthwork structures are referred to as slopes.
- the condition inspection system 1 is composed of a mobile system 60, an evaluation system 4, a communication terminal 1100 of the national or local government, and a communication terminal 1200 of a commissioned business operator.
- the mobile system 60 is an example of an imaging system, and is composed of a data acquisition device 9 and a mobile body 6 such as a vehicle equipped with the data acquisition device 9.
- the vehicle may be a vehicle that runs on a road or a vehicle that runs on a railroad.
- the data acquisition device 9 has an imaging device 7, which is an example of a measuring device that measures a structure, as well as a distance sensor 8a and a GNSS (Global Navigation Satellite System) sensor 8b.
- GNSS Global Navigation Satellite System
- GNSS Global Navigation Satellite System
- the imaging device 7 is a line camera equipped with a line sensor in which photoelectric conversion elements are arranged in one or more rows.
- the imaging device 7 captures images of positions along a predetermined imaging range on an imaging surface along the traveling direction of the moving body 6.
- the imaging device is not limited to a line camera, and may be a camera equipped with an area sensor in which photoelectric conversion elements are arranged in a planar manner.
- the imaging device may also be composed of multiple cameras.
- the distance sensor 8a is a ToF sensor (Time of Flight) that measures the distance to the subject photographed by the photographing device 7.
- the GNSS sensor 8b is a positioning means that receives signals transmitted at each time by multiple GNSS satellites and calculates the distance to the satellite from the difference in the time at which each signal was received, thereby measuring a position on the earth.
- the positioning means may be a device dedicated to positioning, or may be an application dedicated to positioning installed on a PC (Personal Computer), smartphone, etc.
- the distance sensor 8a and the GNSS sensor 8b are examples of sensor devices.
- the distance sensor 8a is also an example of a three-dimensional sensor.
- the ToF sensor used as distance sensor 8a measures the distance from a light source to an object by irradiating the object with laser light from the light source and measuring the scattered and reflected light.
- the distance sensor 8a is a LiDAR (Light Detection and Ranging) sensor.
- LiDAR is a method of measuring the time of flight of light using pulses, but as another method of the ToF sensor, the distance may be measured using a phase difference detection method.
- the phase difference detection method a laser light amplitude modulated at a fundamental frequency is irradiated onto the measurement range, the reflected light is received, and the phase difference between the irradiated light and the reflected light is measured to obtain time, and the distance is calculated by multiplying this time by the speed of light.
- the distance sensor 8a may also be configured with a stereo camera, etc.
- the mobile system 60 can obtain three-dimensional information that is difficult to obtain from two-dimensional images, such as the height, inclination angle, or protrusion of a slope.
- the mobile system 60 may further include an angle sensor 8c.
- the angle sensor 8c is a gyro sensor or the like for detecting the angle (attitude) or angular velocity (or each acceleration) of the shooting direction of the image capture device 7.
- the evaluation system 4 is constructed by an evaluation device 3 and a data management device 5.
- the evaluation device 3 and data management device 5 constituting the evaluation system 4 can communicate with a mobile system 60, a communication terminal 1100 and a communication terminal 1200 via a communication network 100.
- the communication network 100 is constructed by the Internet, a mobile communication network, a LAN (Local Area Network), etc.
- the communication network 100 may include not only wired communication but also networks using wireless communication such as 3G (3rd Generation), 4G (4th Generation), 5G (5th Generation), Wi-Fi (Wireless Fidelity) (registered trademark), WiMAX (Worldwide Interoperability for Microwave Access), or LTE (Long Term Evolution).
- the evaluation device 3 and the data management device 5 may have a communication function using a short-range communication technology such as NFC (Near Field Communication) (registered trademark).
- the data management device 5 is an example of an information processing device, and is a computer such as a PC that manages various data acquired by the data acquisition device 9.
- the data management device 5 receives various acquired data from the data acquisition device 9, and transfers the received various acquired data to the evaluation device 3 that performs data analysis.
- the method of transferring the various acquired data from the data management device 5 to the evaluation device 3 may be manual transfer using a USB (Universal Serial Bus) memory or the like.
- the evaluation device 3 is a computer such as a PC that evaluates the condition of the slope based on various acquired data transferred from the data management device 5.
- a dedicated application program for evaluating the condition of the slope is installed in the evaluation device 3.
- the evaluation device 3 detects the type or structure of the slope from the captured image data and sensor data, extracts shape data, and performs a detailed analysis by detecting the presence or absence of deformation and the degree of deformation.
- the evaluation device 3 also generates a report to be submitted to a road administrator such as the country, local government, or a commissioned business operator, using the captured image data, sensor data, evaluation target data, and the detailed analysis results.
- the data of the report generated by the evaluation device 3 is submitted to the country or local government via the commissioned business operator in the form of electronic data or printed paper.
- the report generated by the evaluation device 3 is called an investigation record sheet, inspection sheet, investigation ledger, or report.
- the evaluation device 3 is not limited to a PC, and may be a smartphone or tablet terminal.
- the evaluation system 4 may be configured to construct the evaluation device 3 and the data management device 5 as a single device or terminal.
- the communication terminal 1200 is provided to the commissioned business operator, and the communication terminal 1100 is provided to the national or local government.
- the evaluation device 3, the communication terminal 1100, and the communication terminal 1200 are examples of communication terminals that can communicate with the data management device 5, and various data managed by the data management device 5 can be viewed.
- FIG. 2 is a diagram showing an example of how a slope condition is inspected using a mobile body system according to an embodiment. As shown in FIG. 2, the mobile body system 6 photographs a predetermined range of the slope with a photographing device 7 while driving a mobile body 6 equipped with a data acquisition device 9 along a road.
- the mobile system 6 drives the mobile unit 6 on the road for several to several tens of kilometers while the imaging device 7 images a predetermined range including the slope and areas other than the slope.
- Areas other than the slope include earthwork structures other than the slope, such as rockfall protection nets and rockfall protection fences, roads, side streets, natural slopes, traffic lights, signs, stores, the sea (when traveling along the coastline), cars, etc.
- a cut slope is a slope where the soil has been piled up
- a bank slope is a slope where the soil has been piled up.
- the slope on the side of a road that runs along the side of a mountain is called a natural slope.
- Cut slopes and bank slopes can be made more durable by planting plants on the surface of the slope, and can be left unchanged for decades. However, this is not always the case.
- cut slopes, bank slopes, and natural slopes deteriorate due to wind and rain, surface collapses occur, causing rocks and soil to fall, or the mountain collapses, causing road closures.
- Earthwork structures include retaining walls that are installed between natural slopes and roads, and rockfall protection fences that prevent rocks from falling onto the road, but both are intended to prevent road closures or human injury caused by soil or falling rocks flowing onto the road.
- the condition inspection system 1 acquires photographed image data of the slope of an earthwork structure using the photographing device 7, and acquires sensor data including three-dimensional information using a three-dimensional sensor such as a distance sensor 8a.
- the evaluation system 4 then combines the acquired photographed image data and sensor data to evaluate the condition of the slope, thereby detecting shape data indicating the three-dimensional shape of the slope and detecting abnormalities such as cracks and peeling. This allows the condition inspection system 1 to efficiently perform evaluations that are difficult to inspect with the human eye.
- Figure 3 is a diagram explaining the condition of the slope.
- Figure 3(a) is an image showing the surface of the slope five years before the collapse
- Figure 3(b) is an explanatory diagram of the image shown in Figure 3(a).
- cracks in the surface layer of the slope are noticeable, and image analysis shown in an unfolded view, etc. is effective in detecting changes or signs of changes in the surface layer, such as cracks, peeling, and seepage.
- Figure 3(c) is an image showing the surface of the slope two years before the collapse
- Figure 3(d) is an explanatory diagram of the image shown in Figure 3(c).
- the inside of the slope has turned to soil, the soil has pushed against the surface of the slope, and the slope has bulged.
- three-dimensional analysis of images such as development drawings + cross-sections is effective.
- Figure 3(d) is an image showing the surface of the slope five years before the collapse
- Figure 3(b) is an explanatory diagram of the image shown in Figure 3(a). In this state, the surface layer of the slope was unable to contain the soil and sand, and collapsed.
- ⁇ Hardware configuration of data acquisition device ⁇ 4 is a diagram showing an example of a hardware configuration of a data acquisition device 9.
- the data acquisition device 9 includes the image capture device 7 and the sensor device 8 as shown in FIG.
- the controller 900 includes an imaging device I/F (Interface) 901, a sensor device I/F 902, a bus line 910, a CPU (Central Processing Unit) 911, a ROM (Read Only Memory) 912, a RAM (Random Access Memory) 913, a HD (Hard Disk) 914, a HDD (Hard Disk Drive) controller 915, a network I/F 916, a DVD-RW (Digital Versatile Disk Rewritable) drive 918, a media I/F 922, an external device connection I/F 923, and a timer 924.
- an imaging device I/F Interface
- sensor device I/F 902 a bus line 910
- a CPU Central Processing Unit
- ROM Read Only Memory
- RAM Random Access Memory
- HD Hard Disk
- HDD Hard Disk Drive
- the imaging device I/F 901 is an interface for transmitting and receiving various data or information to and from the imaging device 7.
- the sensor device I/F 902 is an interface for transmitting and receiving various data or information to and from the sensor device 8.
- the bus line 910 is an address bus, data bus, etc. for electrically connecting each component such as the CPU 911 shown in FIG. 4.
- the CPU 911 also controls the operation of the entire data acquisition device 9.
- the ROM 912 stores programs used to drive the CPU 911, such as IPL.
- the RAM 913 is used as a work area for the CPU 911.
- the HD 914 stores various data such as programs.
- the HDD controller 915 controls the reading and writing of various data from the HD 914 under the control of the CPU 911.
- the network I/F 916 is an interface for data communication using the communication network 100.
- the DVD-RW drive 918 controls the reading and writing of various data from and to a DVD-RW 917, which is an example of a removable recording medium.
- the medium is not limited to a DVD-RW, and may be a DVD-R or a Blu-ray (registered trademark) Disc, etc.
- the media I/F 922 controls the reading and writing (storing) of data from and to a recording medium 921 such as a flash memory.
- the external device connection I/F 923 is an interface for connecting an external device such as an external PC 930 having a display, a reception unit, and a display control unit.
- the timer 924 is a measurement device with a time measurement function.
- the timer 924 may be a computer-based software timer. It is preferable that the timer 924 be synchronized with the time of the GNSS sensor 8b. This makes it easy to synchronize the time and associate the positions of each sensor data and captured image data.
- Fig. 5 is a diagram showing an example of the hardware configuration of the evaluation device 3. Each piece of the hardware configuration of the evaluation device 3 is indicated by a reference number in the 300 series.
- the evaluation device 3 is constructed by a computer, and includes a CPU 301, a ROM 302, a RAM 303, a HD 304, a HDD controller 305, a display 306, an external device connection I/F 308, a network I/F 309, a bus line 310, a keyboard 311, a pointing device 312, a DVD-RW drive 314, and a media I/F 316.
- the CPU 301 controls the operation of the entire evaluation device 3.
- the ROM 302 stores programs such as IPL used to drive the CPU 301.
- the RAM 303 is used as a work area for the CPU 301.
- the HD 304 stores various data such as programs.
- the HDD controller 305 controls the reading or writing of various data from the HD 304 according to the control of the CPU 301.
- the display 306 displays various information such as a cursor, menu, window, character, or image.
- the display 306 is an example of a display unit.
- the external device connection I/F 308 is an interface for connecting various external devices. In this case, the external device is, for example, a USB memory or a printer.
- the network I/F 309 is an interface for data communication using the communication network 100.
- the bus line 310 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 301 shown in FIG. 5.
- the keyboard 311 is a type of input means equipped with multiple keys for inputting characters, numbers, various instructions, etc.
- the pointing device 312 is a type of input means for selecting and executing various instructions, selecting a processing target, moving the cursor, etc.
- the DVD-RW drive 314 controls the reading and writing of various data from the DVD-RW 313, which is an example of a removable recording medium. Note that this is not limited to DVD-RW, and may be a DVD-R or Blu-ray Disc, etc.
- the media I/F 316 controls the reading and writing (storing) of data from the recording medium 315, such as a flash memory.
- Fig. 5 is a diagram showing an example of the hardware configuration of the data management device.
- Each hardware configuration of the data management device 5 is indicated by a reference number in the 500 range in parentheses.
- the data management device 5 is constructed by a computer, and has the same configuration as the evaluation device 3, as shown in Fig. 5, and therefore a description of each hardware configuration will be omitted.
- the communication terminals 1100 and 1200 are also constructed by a computer and have the same configuration as the evaluation device 3, but a description of each hardware configuration will be omitted.
- the above programs may be recorded in the form of installable or executable files on a computer-readable recording medium and distributed.
- Examples of recording media include CD-Rs (Compact Disc Recordable), DVDs (Digital Versatile Discs), Blu-ray Discs, SD cards, USB memories, etc.
- the recording media may be provided domestically or internationally as a program product.
- the evaluation system 4 according to the embodiment realizes the evaluation method according to the present invention by executing the program according to the present invention.
- Fig. 6 is a diagram showing an example of the functional configuration of the state inspection system according to the first embodiment.
- Fig. 6 shows devices shown in Fig. 1 that are related to the processes or operations described below.
- the data acquisition device 9 has a communication unit 91, a calculation unit 92, a photographing device control unit 93, a sensor device control unit 94, a photographed image data acquisition unit 95, a sensor data acquisition unit 96, a time data acquisition unit 97, a request reception unit 98, and a storage/readout unit 99.
- Each of these units is a function or means realized by any of the components shown in FIG. 4 operating according to an instruction from the CPU 911 according to a program for the data acquisition device expanded from the HD 914 onto the RAM 913.
- the data acquisition device 9 also has a storage unit 9000 constructed by the ROM 912 and HD 914 shown in FIG. 4.
- the external PC 930 connected to the data acquisition device 9 shown in FIG. 4 also has a reception unit and a display control unit.
- the communication unit 91 is mainly realized by the processing of the CPU 911 on the network I/F 916, and communicates various data or information with other devices via the communication network 100.
- the communication unit 91 transmits, for example, data acquired by the photographed image data acquisition unit 95 and the sensor data acquisition unit 96 to the data management device 5.
- the calculation unit 92 is realized by the processing of the CPU 911, and performs various calculations.
- the imaging device control unit 93 is mainly realized by the processing of the CPU 911 for the imaging device I/F 901, and controls the imaging processing by the imaging device 7.
- the sensor device control unit 94 is mainly realized by the processing of the CPU 911 for the sensor device I/F 902, and controls the data acquisition processing for the sensor device 8.
- the imaging device control unit 93 is an example of an angle change unit.
- the captured image data acquisition unit 95 is mainly realized by the processing of the CPU 911 for the imaging device I/F 901, and acquires captured image data relating to an image captured by the imaging device 7.
- the sensor data acquisition unit 96 is mainly realized by the processing of the CPU 911 for the sensor device I/F 902, and acquires sensor data that is the detection result by the sensor device 8.
- the sensor data acquisition unit 96 is an example of a distance information acquisition unit and a position information acquisition unit.
- the time data acquisition unit 97 is mainly realized by the processing of the CPU 911 for the timer 924, and acquires time data indicating the time when data was acquired by the captured image data acquisition unit 95 or the sensor data acquisition unit 96.
- the request receiving unit 98 is mainly realized by the processing of the CPU 911 on the external device connection I/F 923, and receives a specific request from the external PC 930 etc.
- the storage/reading unit 99 is mainly realized by the processing of the CPU 911, and stores various data (or information) in the storage unit 9000 and reads various data (or information) from the storage unit 9000.
- the evaluation device 3 has a communication unit 31, a reception unit 32, a display control unit 33, a judgment unit 34, an evaluation target data generation unit 35, a detection unit 36, a map data management unit 37, a report generation unit 38, and a storage/readout unit 39.
- Each of these units is a function or means realized by any of the components shown in Fig. 5 being loaded from the HD 304 onto the RAM 303 and operating in response to an instruction from the CPU 301 according to a program for the evaluation device.
- the evaluation device 3 also has a storage unit 3000 constructed by the ROM 302 and the HD 304 shown in Fig. 5.
- the communication unit 31 is mainly realized by the processing of the CPU 301 on the network I/F 309, and communicates various data or information with other devices via the communication network 100.
- the communication unit 31 transmits and receives various data related to the evaluation of the slope condition with the data management device 5, for example.
- the reception unit 32 is mainly realized by the processing of the CPU 301 on the keyboard 311 or the pointing device 312, and receives various selections or inputs from the user.
- the reception unit 32 receives various selections or inputs on the evaluation screen 400 described below.
- the display control unit 33 is mainly realized by the processing of the CPU 301, and causes the display 306 to display various images.
- the display control unit 33 causes the display 306 to display the evaluation screen 400 described below.
- the judgment unit 34 is realized by the processing of the CPU 301, and makes various judgments.
- the reception unit 32 is an example of an operation reception means.
- the evaluation target data generation unit 35 is realized by the processing of the CPU 301, and generates data to be evaluated.
- the detection unit 36 is mainly realized by the processing of the CPU 301, and performs a process of detecting the state of the slope using the evaluation target data generated by the evaluation target data generation unit 35.
- the map data management unit 37 is mainly realized by the processing of the CPU 301, and manages map information acquired from an external server, etc.
- the map information includes position information for any position on the map.
- the report generation unit 38 is mainly realized by the processing of the CPU 301, and generates an evaluation report to be submitted to the road administrator based on the evaluation results.
- the storage/reading unit 39 is mainly realized by the processing of the CPU 301, and stores various data (or information) in the storage unit 3000 and reads various data (or information) from the storage unit 3000.
- the setting unit 40 is mainly realized by the processing of the CPU 301, and performs various settings.
- the data management device 5 has a communication unit 51, a judgment unit 52, a data management unit 53, and a storage/readout unit 59. Each of these units is a function or means realized when any of the components shown in Fig. 5 is loaded from the HD 504 onto the RAM 503 and operates according to an instruction from the CPU 501 in accordance with a program for the data management device.
- the data management device 5 also has a storage unit 5000 constructed by the ROM 502 and the HD 504 shown in Fig. 5.
- the communication unit 51 is mainly realized by the processing of the CPU 501 on the network I/F 509, and communicates various data or information with other devices via the communication network 100.
- the communication unit 51 receives, for example, captured image data and sensor data transmitted from the data acquisition device 9.
- the communication unit 51 also transmits and receives various data related to the evaluation of the slope condition, for example, with the evaluation device 3, etc.
- the communication unit 51 is an example of an instruction receiving means.
- the judgment unit 52 is an example of a position generating means, and is realized by the processing of the CPU 501, and makes various judgments.
- the data management unit 53 is mainly realized by the processing of the CPU 501, and manages various data related to the evaluation of the slope condition.
- the data management unit 53 for example, registers photographed image data and sensor data transmitted from the data acquisition device 9 in the acquired data management DB 5001.
- the data management unit 53 also registers, for example, data processed or generated by the evaluation device 3 in the processed data management DB 5003.
- the generation unit 54 is mainly realized by the processing of the CPU 501, and generates various image data related to the slope.
- the setting unit 55 is mainly realized by the processing of the CPU 501, and performs various settings.
- the storage/reading unit 59 is mainly realized by the processing of the CPU 501 , and stores various data (or information) in the storage unit 5000 and reads various data (or information) from the storage unit 5000 .
- the communication terminal 1100 has a communication unit 1101, a reception unit 1102, a display control unit 1103, a judgment unit 1104, and a storage/reading unit 1105.
- Each of these units is a function or means realized when any of the components shown in FIG. 5 is loaded from the HD onto the RAM and operates according to an instruction from the CPU in accordance with a program for the terminal device.
- the data management device 5 also has a storage unit 1106 constructed from the ROM and HD shown in FIG. 5.
- the communication unit 1101 is mainly realized by the processing of the CPU for the network I/F, and communicates various data or information with other devices via the communication network 100.
- the reception unit 1102 is mainly realized by CPU processing on a keyboard or pointing device, and receives various selections or inputs from the user.
- the display control unit 1103 is mainly realized by CPU processing, and causes the display to display various images.
- the judgment unit 1104 is realized by CPU 301 processing, and makes various judgments.
- the reception unit 1102 is an example of an operation reception means.
- the storage/reading unit 1105 is realized mainly by the processing of the CPU, and stores various data (or information) in the storage unit 1106 and reads various data (or information) from the storage unit 1106.
- the communication terminal 1200 has a communication unit 1201, a reception unit 1202, a display control unit 1203, a judgment unit 1204, and a storage/reading unit 1205. Each of these units is a function or means realized when any of the components shown in FIG. 5 is loaded from the HD onto the RAM and operates according to an instruction from the CPU in accordance with a program for the terminal device.
- the data management device 5 also has a storage unit 1206 constructed from the ROM and HD shown in FIG. 5.
- the communication unit 1201 is mainly realized by the CPU's processing of the network I/F, and communicates various data or information with other devices via the communication network 100.
- the reception unit 1202 is mainly realized by CPU processing on the keyboard or pointing device, and receives various selections or inputs from the user.
- the display control unit 1203 is mainly realized by CPU processing, and causes various images to be displayed on the display.
- the judgment unit 1204 is realized by CPU 301 processing, and makes various judgments.
- the storage/readout unit 1205 is realized mainly by the processing of the CPU, and stores various data (or information) in the storage unit 1206 and reads various data (or information) from the storage unit 1206.
- Condition Type Management Table Figures 7 and 8 are conceptual diagrams showing an example of a condition type management table.
- the condition type management table is a table for managing teacher data for detecting the condition type of a slope.
- a condition type management DB 3001 configured by the condition type management table as shown in Figures 7 and 8 is constructed in the storage unit 3000.
- a type name indicating the condition type, a teacher image, and a remarks column are associated and managed for each type number.
- the type name is a name indicating a condition type for identifying the state of the slope, the physical quantities around the slope, and the site information.
- the condition type includes the type of the slope itself, which is a structure such as a retaining wall, a crest, a sprayed mortar, a wire mesh, a fence, a drainage hole, a pipe, a small drainage channel, and the like, and types indicating the physical quantities around the slope, such as spring water, moss, plants, falling rocks, soil, and sunlight.
- the condition type also includes types such as poles, utility poles, signs, or billboards, as site information that supports data acquisition by the mobile system 60.
- condition type may include, as additional information of the structure, information on markers such as chalking that indicate the presence of an abnormality, installed during past inspections or construction, and man-made objects such as measuring devices and traces of countermeasures.
- the teacher image is an example of teacher data, and is a teacher image used in machine learning to determine the state type of the slope, the physical quantities around the slope, and the site information from the captured image data.
- the training data here is not limited to luminance images or RGB images, which are generally referred to as images, but may be in the form of depth information, text, audio, etc., as long as it contains information for determining the state type.
- the remarks column shows information that will be the detection criteria for detecting the state type.
- Acquired Data Management Table Fig. 9A is a conceptual diagram showing an example of an acquired data management table.
- the acquired data management table is a table for managing various acquired data acquired by the data acquisition device 9.
- An acquired data management DB 5001 configured by an acquired data management table as shown in Fig. 9A is constructed in the storage unit 5000. In this acquired data management table, photographed image data, sensor data, and acquisition time are associated and managed for each folder.
- the photographed image data and sensor data are data files of acquired data transmitted from the data acquisition device 9.
- the acquisition time indicates the time when the photographed image data and sensor data were acquired by the data acquisition device 9.
- Data acquired in one inspection process is stored in the same folder.
- the photographed image data and the three-dimensional sensor data contained in the sensor data are stored in association with coordinates, as described below.
- the photographed image data and the three-dimensional sensor data contained in the sensor data are stored in association with the positioning data contained in the sensor data.
- Processing Data Management Table Fig. 9(B) is a conceptual diagram showing an example of a processing data management table.
- the processing data management table is a table for managing various processing data processed by the evaluation device 3.
- a processing data management DB 5003 configured by a processing data management table as shown in Fig. 9(B) is constructed in the storage unit 5000.
- evaluation target data, evaluation data, positioning data, and comments are associated and managed for each folder.
- the evaluation target data is a data file used for the detection and evaluation of the slope condition by the evaluation device 3.
- the evaluation data is a data file showing the evaluation results by the evaluation device 3.
- the positioning data is data showing the position information measured by the GNSS sensor 8b.
- the comment is bibliographic information input by the evaluator regarding the evaluation target data or the evaluation data.
- Figure 10 is a diagram explaining the captured images acquired by the mobile system.
- the mobile body system 60 uses the imaging device 7 provided in the data acquisition device 9 to capture images of slopes on a road while the mobile body 6 is traveling.
- the X-axis direction shown in FIG. 10 indicates the direction of movement of the mobile body 6, the Y-axis direction is the vertical direction, and the Z-axis direction is perpendicular to the X-axis and Y-axis directions and indicates the depth direction from the mobile body 6 toward the slope.
- the data acquisition device 9 acquires photographed image 1, distance-measured image 1, photographed image 2, and distance-measured image 2 in chronological order, as shown in FIG. 10.
- Distance-measured image 1 and distance-measured image 2 are images acquired by distance sensor 8a.
- the photographing device 7 and sensor device 8 are time-synchronized, so photographed image 1 and distance-measured image 1, and photographed image 2 and distance-measured image 2 are images of the same area of the slope.
- tilt correction (image correction) of the photographed image is performed based on the attitude of the vehicle at the time of shooting, and the image data and positioning data (north latitude and east longitude) are linked based on the time of the photographed image.
- the mobile system 60 acquires photographed image data of the slope and sensor data acquired in response to photographing by the photographing device 7 while driving the vehicle as the mobile body 6, and uploads them to the data management device 5.
- the data acquisition device 9 may acquire the ranging images and the photographed images while driving separately, but considering changes in the slope shape due to collapses, etc., it is preferable to acquire the ranging images and the photographed images for the same slope shape while driving at the same time.
- Figure 11 is an explanatory diagram of the captured image and the distance measurement image.
- FIG. 11(a) shows captured image data 7A of captured images 1, 2, etc. shown in FIG. 10.
- Each pixel 7A1 of the captured image data 7A acquired by the photographing device 7 is arranged at coordinates corresponding to the X-axis and Y-axis directions shown in FIG. 10, and has brightness information corresponding to the amount of stored power.
- the captured image data 7A is an example of a brightness image.
- each pixel 7A1 of the captured image data 7A is associated with coordinates corresponding to the X-axis and Y-axis directions shown in FIG. 10 and stored in the storage unit 5000 as the captured image data shown in FIG. 9.
- FIG. 11(b) shows distance measurement image data 8A such as distance measurement images 1 and 2 shown in FIG. 10.
- Each pixel 8A1 of distance measurement image data 8A acquired by distance sensor 8a is arranged at coordinates corresponding to the X-axis and Y-axis directions shown in FIG. 10, and has distance information in the Z-axis direction shown in FIG. 10 corresponding to the amount of stored power.
- distance measurement image data 8A is three-dimensional point cloud data, but is generally referred to as distance measurement image data because it is visually displayed with luminance information added when viewed by a user.
- the captured image data 7A and distance measurement image data 8A are collectively referred to as image data.
- the distance information for each pixel 8A1 of the distance measurement image data 8A is then associated with coordinates corresponding to the X-axis and Y-axis directions shown in FIG. 10 and stored in the storage unit 5000 as three-dimensional data included in the sensor data shown in FIG. 9.
- the captured image data 7A shown in FIG. 11(a) and the distance measurement image data 8A shown in FIG. 11(b) are images of the same area of the slope, so the brightness information and distance information are stored in the memory unit 5000 in association with the coordinates corresponding to the X-axis and Y-axis directions shown in FIG. 10.
- FIG. 12 is an explanatory diagram of multiple shooting areas.
- the shooting device 7 moves with the moving body 6 while shooting a slope 80, which is the object of inspection and evaluation. Specifically, the shooting device 7 shoots the target area 70 including the slope 80 by dividing it into multiple shooting areas d11, d12, ... at a constant shooting interval t along the X-axis direction, which is the moving direction of the moving body 6.
- the photographing device 7 photographs the target area 70, which includes the slope 80, which is the object of inspection and evaluation, and areas other than the object of inspection and evaluation, divided into multiple photographing areas d11, d12, ..., and, as described below, multiple photographing areas in which the slope 80 has been photographed are identified from the multiple photographing areas.
- the captured image of multiple shooting areas d11, d12, etc. is a long slit-shaped captured image in the Y-axis direction, and by stitching together the captured images of multiple shooting areas d11, d12, etc., it is possible to obtain a captured image of the target area 70 that is continuous in the X-axis direction.
- FIG. 12(c) is a diagram showing a case where the entire target area 70 is divided into a plurality of target areas and imaged when the entire target area 70 is imaged.
- the entire target area 70 is imaged by dividing the captured image into four target areas, namely, target areas 701A, 702A, 701B, and 702B.
- each of the multiple target areas 701A, 702A, 701B, and 702B is divided into multiple shooting areas d11, d12, etc., and images of the multiple shooting areas d11, d12, etc. are stitched together to obtain images of each of the multiple target areas 701A, 702A, 701B, and 702B. Then, an image of the target area 70 can be obtained by stitching together images of the multiple target areas 701A, 702A, 701B, and 702B.
- the image capture device 7 is equipped with multiple image capture devices, and the target areas 702A and 702B are captured by an image capture device different from the image capture device that captures the target areas 701A and 701B.
- target area 701B is photographed under different photographing conditions by the same photographing device as that used to photograph target area 701A
- target area 702B is photographed under different photographing conditions by the same photographing device as that used to photograph target area 702A.
- the distance sensor 8a when the image capture device 7 captures the target area of the slope 80 divided into a plurality of image capture areas d11, d12, etc., it is desirable that the distance sensor 8a also acquires distance information indicating the distance from the distance sensor 8a to each of the plurality of image capture areas d11, d12, etc.
- By associating the luminance information of each image of the captured image capturing the target area of the slope 80 with the distance information of each pixel of the distance measurement image capturing the target area of the slope 80 it is possible to perform a highly accurate inspection of the target area of the slope 80.
- FIG. 13 is a diagram showing a mobile system equipped with multiple image capture devices according to an embodiment.
- the photographing device 7 includes multiple photographing devices 71, 72, and 73, and the photographing devices 71, 72, and 73 photograph a target area 701 on the slope 80, a target area 702 above the target area 701, and a target area 703 above the target area 702, respectively.
- the first and second target areas refer to any two of target areas 701, 702, and 703, and the first and second imaging devices refer to the imaging devices corresponding to the first and second target areas among the multiple imaging devices 71, 72, and 73.
- FIG. 14 is a sequence diagram showing an example of data acquisition processing using a mobile system.
- an inspection worker performs a predetermined input operation or the like on the external PC 330, and the request receiving unit 98 of the data acquisition device 9 receives a request to start data acquisition (step S11).
- the data acquisition device 9 executes data acquisition processing using the imaging device 7 and the sensor device 8 (step S12).
- the imaging device control unit 93 issues an imaging request to the imaging device 7 to start imaging processing for a specified area.
- the mobile system 6 uses the imaging device 7 to capture images of a predetermined area including the slope and areas other than the slope while the mobile unit 6 is moving, and the imaging device control unit 93 starts imaging processing for the areas other than the slope, completes imaging processing for the slope, and then ends imaging processing for the areas other than the slope. This makes it possible to capture images of the entire area of the slope from one end to the other in the direction of movement of the mobile unit 6.
- the sensor device control unit 94 also starts detection processing by the distance sensor 8a and the GNSS sensor 8b in synchronization with the photographing processing by the photographing device 7.
- the photographed image data acquisition unit 95 then acquires the photographed image data acquired by the photographing device 7, and the sensor data acquisition unit 96 acquires the sensor data acquired by the distance sensor 8a and the GNSS sensor 8b.
- the time data acquisition unit 97 also acquires time data indicating the time at which various data were acquired by the photographed image data acquisition unit 95 and the sensor data acquisition unit 96.
- the inspection worker performs a predetermined input operation on the external PC 330 or the like, and the request receiving unit 98 receives a request to upload the various acquired data (step S13).
- the communication unit 91 uploads (transmits) the captured image data, sensor data, and time data, which are the acquired data acquired in step S12, to the data management device 5 (step S14).
- the communication unit 51 of the data management device 5 receives the acquired data transmitted from the data acquisition device 9.
- the data management unit 53 of the data management device 5 registers the acquired data received in step S14 in the acquired data management DB 5001 (see FIG. 9(A)) (step S15).
- the data management unit 53 stores the captured image data and sensor data in one folder in association with time data indicating the acquisition time of each data included in the acquired data.
- FIG. 15 is a sequence diagram showing an example of a process for generating evaluation target data.
- the reception unit 32 of the evaluation device 3 accepts the selection of data to be generated (step S31).
- the user of the evaluation device 3 may select an arbitrary position in the map information managed by the map data management unit 37 of the evaluation device 3, and the reception unit 32 of the evaluation device 3 may accept the selection of position information in the map information.
- the communication unit 31 transmits a request to generate evaluation target data related to the generation target data selected in step S11 to the data management device 5, and the communication unit 51 of the data management device 5 receives the request transmitted from the evaluation device 3 (step S32).
- This request includes the folder name selected in step S31. Alternatively, this request may include location information in the map information.
- the storage/reading unit 59 of the data management device 5 searches the acquired data management DB 5001 using the folder name included in the generation request received in step S32 as a search key, thereby reading out the acquired data associated with the folder name included in the generation request.
- the storage/reading unit 59 searches the acquired data management DB 5001 using the location information included in the request received in step S32 as a search key, thereby reading out the acquired data associated with the location information included in the request.
- This acquired data includes captured image data, sensor data, and time data.
- the generating unit 54 of the data management device 5 generates data to be evaluated based on the acquired data read by the storing and reading unit 59 (step S33). Specifically, the generating unit 54 performs tilt correction of the captured image data from the attitude of the imaging device 7 (mobile body 6) at the time of shooting, based on the acquired sensor data of the distance sensor 8a. The generating unit 54 also links the captured image data to the positioning data, which is the acquired sensor data of the GNSS sensor 8b, based on the acquired time data. Furthermore, the generating unit 54 performs a process of synthesizing multiple captured image data into one image data.
- the generation unit 54 generates a composite image by stitching together the captured images of the multiple captured areas, thereby obtaining the captured images of the target area 70 and the multiple target areas 701A, 702A, 701B, and 702B.
- the generating unit 54 also generates a composite image by stitching together the captured images of the multiple target areas 701A, 702A, 701B, and 702B, thereby obtaining a captured image of the entire target area 70.
- the target area 70 includes the slope 80 and the area other than the slope 80.
- the generation unit 54 has a tilt correction function for image data, a function for linking image data with position information, and a function for combining image data.
- the generation unit 54 uses the acquired data to perform image correction on the acquired captured image data so that processing by the detection unit 36 and report generation unit 38, which will be described later, can be easily performed.
- the generating unit 54 generates an input/output screen including the composite image (step S34).
- This input/output screen is an example of a display screen that displays a composite image that is created by stitching together images captured by dividing the target area 70 into a plurality of shooting areas dn along the moving direction of the moving body 6, and step S34 is an example of a generating step.
- the generation unit 54 generates a composite image with a lower resolution than the composite image generated in step S33, and generates an input/output screen that includes this lower resolution composite image.
- the generation unit 54 generates an input/output screen so as to display the composite image 2500 at a lower resolution than each of the captured images captured separately in the multiple shooting areas dn stored in the acquired data management DB 5001. This improves the processing speed when generating an input/output screen including a composite image.
- the generation unit 54 also generates an input/output screen including multiple composite images corresponding to the target areas 701, 702, and 703 captured by the image capture devices 71, 72, and 73 described in FIG. 13, respectively.
- the target area 70 includes a first target area and a second target area which are different ranges in a direction intersecting the moving direction of the moving body 66
- the generation unit 54 generates an input/output screen 2000 including at least one of the first composite image and the second composite image, which is a first composite image obtained by stitching together first captured images pn obtained by dividing the first target area into a plurality of first captured areas dn along the moving direction of the moving body 66 and capturing the first captured images pn, and a second composite image obtained by stitching together second captured images pn obtained by dividing the second target area into a plurality of second captured areas dn along the moving direction of the moving body 66.
- the communication unit 51 transmits input/output screen information relating to the input/output screen generated in step S34 to the evaluation device 3, and the communication unit 31 of the evaluation device 3 receives the input/output screen information transmitted from the data management device 5 (step S35).
- the input/output screen includes a composite image generated at a lower resolution than the multiple captured images stored in the acquired data management DB 5001, thereby reducing the communication load when transmitting the input/output screen including the composite image.
- the display control unit 33 of the evaluation device 3 displays the input/output screen received in step S34 on the display 306, and the reception unit 32 of the evaluation device 3 receives a predetermined input operation by the user on the displayed input/output screen (step S36).
- This input operation includes a determination operation for determining to identify a partial area in the composite image.
- the input/output screen includes a composite image generated at a lower resolution than the multiple captured images stored in the acquired data management DB 5001, improving the processing speed when displaying the input/output screen including the composite image.
- the communication unit 31 transmits input information related to the input operation received by the reception unit 32 to the data management device 5, and the communication unit 51 of the data management device 5 receives the input information transmitted from the evaluation device 3. (Step S37).
- This input information includes specific area information and comments that identify a partial area in the composite image, identification information that identifies a specific slope among multiple slopes, etc.
- the setting unit 55 updates the evaluation target data generated in step S33 based on the input information received in step S37, and stores the updated evaluation target data in the processing data management DB 5003 (see FIG. 9(B)) (step S38).
- the setting unit 55 is an example of a setting means.
- the setting unit 55 updates the evaluation target data by setting a partial image corresponding to the partial area, position information, and a specific point group in a three-dimensional point cloud corresponding to multiple shooting areas dn based on specific area information that identifies a partial area in the composite image, and associates the evaluation target data, positioning data, and comments included in the generated data and stores them in one folder.
- the composite image included in the input/output screen was generated at a lower resolution than the multiple captured images stored in the acquired data management DB 5001.
- the partial image stored in step S38 is an image with the same high resolution as the multiple captured images stored in the acquired data management DB 5001, so that the processing by the detection unit 36 and report generation unit 38 described below can be performed with high accuracy.
- the communication unit 51 transmits partial image information indicating the partial image included in the generated data updated in step S38 to the evaluation device 3, and the communication unit 31 of the evaluation device 3 receives the partial image information transmitted from the data management device 5 (step S39). Then, the display control unit 33 of the evaluation device 3 displays the partial image received in step S39 on the display 306.
- the functions of the data management device 5 in FIG. 6 may be integrated into the evaluation device 3, and the processing of the data management device 5 in FIG. 15 may also be executed by the evaluation device 3.
- Figure 16 is an explanatory diagram of a composite image of the condition inspection system.
- FIG. 16(a) shows a composite image 2500 generated in step S33 of FIG. 15.
- the composite image 2500 is an image created by stitching together the images p1 to pl captured by dividing the target area 70 into a number of capture areas dn along the direction of movement of the moving body 6, and as mentioned above, when the position of the slope is unknown, it corresponds to a distance of several kilometers to several tens of kilometers, making it difficult to view all of them.
- FIG. 16(b) shows a composite image 2500 included in the input/output screen generated in step S34 of FIG. 15.
- step S34 of FIG. 15 the generation unit 54 divides the composite image 2500 into a plurality of divided image groups 250A, 250B, etc., and generates an input/output screen so that each of the divided images 250A1 to Am is displayed side by side in each of the divided image groups 250A, 250B, etc.
- Each of the divided images 250A1 to Am is an image formed by connecting a plurality of captured images p1 to pn, pn+1 to p2n, etc.
- step S34 of FIG. 15 the generation unit 54 generates the input/output screen so that the number of divided image groups 250A, 250B, etc., the number of divided images 250A1-Am included in one divided image group, and the number of captured images p1-pn included in one divided image vary depending on the resolution of the display 306 or the like on which the input/output images are displayed.
- the generation unit 54 generates the input/output screen so that the length of the composite image 2500 in the direction of movement of the moving object 66, which corresponds to the distance traveled by the moving object 66, differs.
- Figure 17 is an explanatory diagram of operations on the input/output screen of the status inspection system.
- FIG. 17 is an explanatory diagram of operations on the input/output screen of the status inspection system.
- FIG. 17 shows the input/output screen 2000 displayed on the display 306 of the evaluation device 3 in step S36 of the sequence diagram shown in FIG. 15, but the same is true for the input/output screen 2000 displayed on the respective displays of the data acquisition device 9, the communication terminal 1100, and the communication terminal 1200.
- the display control unit 33 of the evaluation device 3 displays an input/output screen 2000 including an identification reception screen 2010 that receives an identification operation for identifying a partial area in the composite image 2500, and a decision reception screen 2020 that receives a decision operation for deciding to identify a partial area in the composite image 2500.
- the display control unit 33 displays the composite image 2500 on the specific reception screen 2010, and also displays the pointer 2300 operated by the pointing device 312 on the composite image 2500.
- the composite image 2500 is an image created by stitching together images captured by dividing the target area 70 into a plurality of shooting areas dn along the direction of movement of the moving body 6, and is displayed as a group of divided images in which each of the divided images of the plurality of divided images is arranged, as explained in FIG. 16(b).
- each divided image represents a captured image of a distance of 100 m along the direction of movement of the moving object 6, and a group of seven divided images represents a captured image of a distance of 700 m along the direction of movement of the moving object 6.
- the display control unit 33 displays a start position designation button 2402, an end position designation button 2404, a reduce button 2406, and an enlarge button 2408 on the decision acceptance screen 2020.
- the start position designation button 2402 and the end position designation button 2404 are buttons that instruct the display of a start position bar 250S and an end position bar 250G, respectively, on the composite image 2500.
- the start position bar 250S and the end position bar 250G can be moved to any position on the composite image 2500 by operating the pointer 2300.
- the specific position determination button 2400 is a button that determines the position of the start position bar 250S and the end position bar 250G on the completed image 2500.
- the reduce button 2406 and the enlarge button 2408 are buttons for instructing the display of the composite image 2500 to be reduced or enlarged.
- the screen switching button 2409 is a button for switching between the display of the multiple divided image groups 250A and 250B shown in FIG. 16(b).
- the reception unit 32 receives the operation, and the display control unit 33 displays the start position bar 250S at an arbitrary position on the composite image 2500.
- the reception unit 32 receives the operation, and the display control unit 33 displays the end position bar 250G at an arbitrary position on the composite image 2500.
- the reception unit 32 accepts this as a specific operation that identifies a partial area in the composite image 2500.
- the position information indicating the positions of the start position bar 250S and the end position bar 250G in the composite image 2500 is an example of specific area information that identifies a partial area in the composite image 2500.
- the reception unit 32 accepts this as a determination operation to determine the determination of a partial area in the composite image 2500.
- the composite image 2500 only displayed the area between the start position bar 250S1 and the end position bar 250G1, the user would not be able to confirm the boundaries on either side of the slope 80 in the direction of movement of the moving body 66, and would therefore not be able to accurately confirm the position or extent of the slope 80.
- the generation unit 54 generates the input/output screen 2000 including the composite image 2500 so that the composite image 2500 includes the boundaries on both sides of the slope 80 in the moving direction of the moving body 6. This allows the user to check the composite image 2500 including the boundaries on both sides of the slope 80 displayed on the input/output screen 2000 and accurately check the position and range of the slope 80. Furthermore, the display control unit 33 assigns a higher priority to the divided image groups 250A and 250B described in FIG. 16B, which may have a boundary between the slope 80 and other than the slope 80 in the moving direction of the moving body 6 through image analysis, and displays them on the input/output screen 2000. This reduces the user's unnecessary checking work required to display divided image groups that have no boundaries between the slope 80 and other than the slope 80.
- the generating unit 54 generates the input/output screen 2000 including the composite image 2500 so that the composite image 2500 includes the boundaries on both sides of the multiple slopes 80 at different positions in the moving direction of the moving body 66. This allows the user to check the composite image 2500 including the boundaries on both sides of each of the multiple slopes 80 displayed on the input/output screen 2000, and accurately check the positions and ranges of the multiple slopes 80.
- Figure 18 is another explanatory diagram of operations on the input/output screen of the status inspection system.
- FIG. 18 shows the state after the enlargement button 2408 is operated on the input/output screen shown in FIG. 17.
- the composite image 2500 shown in FIG. 18 is enlarged compared to the composite image 2500 shown in FIG. 17, with each divided image showing a captured image with a distance of 50 m along the direction of movement of the moving object 6, and a group of divided images arranged in four divided images showing a captured image with a distance of 200 m along the direction of movement of the moving object 6.
- the boundary of the slope 80 is unclear in the composite image 2500 shown in FIG. 17, the boundary of the slope 80 can be accurately confirmed by displaying an enlarged composite image 2500 as shown in FIG. 18.
- FIG. 19 is a flowchart showing the processing based on the operations shown in FIGS. 17 and 18.
- FIG. 19(a) shows the processing in the evaluation device 3
- FIG. 19(b) shows the processing in the data management device 5.
- the reception unit 32 of the evaluation device 3 receives this as a specific operation to identify a partial area of the composite image 2500 (step S151), and when the specific position determination button 2400 is operated, this is received as a determination operation to determine that a partial area of the composite image 2500 is to be identified (step S152).
- the judgment unit 34 of the evaluation device 3 detects the X coordinates of the start position bar 250S and the end position bar 250G in the composite image 2500 where the specific operation was performed as specific area information (step S153).
- the communication unit 31 of the evaluation device 3 transmits input information related to the input operation received by the reception unit 32 to the data management device 5 (step S154).
- This input information includes specific area information indicating a specific area by the X coordinate based on the specific operation by the pointer 2300.
- the communication unit 51 of the data management device 5 receives the input information sent from the evaluation device 3, and the setting unit 55 sets the multiple captured images between the X coordinates on both sides of the specific area as partial images in the composite image 2500 generated in step S33 of FIG. 15 based on the specific area information contained in the received input information, and the generation unit 54 performs geometric, color, brightness, and color shift correction on the partial images to make it easier to evaluate the slope 80 in a later process.
- the storage and reading unit 59 stores the partial images and their coordinates in the storage unit 5000 (step S155).
- the setting unit 55 sets, as other partial images, multiple captured images of other capturing areas whose X coordinates correspond to the partial image set in step S155, from among other composite images captured by other capturing devices, and the generating unit 54 performs geometric, color, brightness, and color shift corrections on the other partial images so that the slope 80 can be easily evaluated in a later process.
- the storing and reading unit 59 stores the other partial images and their coordinates in the storage unit 5000 (step S156).
- the partial image set in step S155 is, as an example, a partial image in target area 702 captured by imaging device 72 described in FIG. 13, and the other partial image set in step S156 is, as an example, a partial image in target area 701 or 703 captured by imaging device 71 or 73 described in FIG. 13.
- step S155 the setting unit 55 sets a first partial image corresponding to a partial area in the first composite image based on a first decision operation that decides to identify a partial area in the first composite image, and in step S156, sets a second partial image corresponding to a partial area in the second composite image.
- the setting unit 55 sets an integrated partial image by joining the partial image set in step S155 and the other partial image set in step S156, and the generation unit 54 performs a joining process on the integrated partial image so that the slope 80 can be easily evaluated in a later process.
- the storage/reading unit 59 stores the integrated partial image and its coordinates in the storage unit 5000 (step S157).
- the setting unit 55 sets the 3D point cloud data shown in FIG. 11(B) whose X coordinates correspond to the integrated partial image set in step S157 as a specific point cloud, and the storage/reading unit 59 stores the coordinates of the specific point cloud in the storage unit 5000 (step S158).
- the setting unit 55 sets the location information whose acquisition time corresponds to the integrated partial image set in step S157 from the positioning data linked to the captured image data in step S33, and the storage/reading unit 59 stores the location information whose acquisition time corresponds to the integrated partial image in the storage unit 5000 (step S159).
- the communication unit 51 transmits to the evaluation device 3 integrated partial image information indicating the integrated partial image set in step S157 (step S161).
- the communication unit 31 of the evaluation device 3 receives the integrated partial image information transmitted from the data management device 5, and the display control unit 33 of the evaluation device 3 displays the received integrated partial image on the display 306.
- Figure 20 is an explanatory diagram of an integrated partial image of the status inspection system.
- FIG. 20(a) shows an upper partial image 255 ⁇ , a middle partial image 255M, and a lower partial image 255L.
- the central partial image 255M is a partial image of the target area 702 captured by the image capture device 72 described in FIG. 13, and is set by the setting unit 55 in step S155 shown in FIG. 19.
- the upper partial image 255 ⁇ and the lower partial image 255L are partial images of the target areas 701 and 703 captured by the image capture devices 71 and 73 described in FIG. 13, and are set by the setting unit 55 in step S156 shown in FIG. 19.
- the upper partial image 255 ⁇ , the middle partial image 255M, and the lower partial image 255L have been subjected to geometric, color, brightness, and color shift correction by the generation unit 54 to facilitate evaluation of the slope 80 in a later process.
- FIG. 20(b) shows an integrated partial image 2550 formed by stitching together the upper partial image 255 ⁇ , the middle partial image 255M, and the lower partial image 255L.
- the generation unit 54 applies stitching processing to the integrated partial image 2550 to facilitate evaluation of the slope 80 in a later process.
- FIG. 21 is a sequence diagram showing a modified example of the process for generating evaluation target data.
- the user of the evaluation device 3 specifies a folder, and the reception unit 32 of the evaluation device 3 receives the selection of the data to be generated.
- the user of the evaluation device 3 may select an arbitrary position in the map information managed by the map data management unit 37 of the evaluation device 3, and the reception unit 32 of the evaluation device 3 may receive the selection of position information in the map information.
- the communication unit 31 of the evaluation device 3 transmits a request to generate the data to be evaluated to the data management device 5 (step S41).
- This request includes the name of the folder in which the data to be generated is stored. Alternatively, this request may include location information in map information.
- the communication unit 51 of the data management device 5 receives the request to generate transmitted from the evaluation device 3.
- the storage/reading unit 59 of the data management device 5 searches the acquired data management DB 5001 using the folder name included in the generation request received in step S41 as a search key, thereby reading out the acquired data associated with the folder name included in the generation request (step S42).
- the storage/reading unit 59 searches the acquired data management DB 5001 using the location information included in the request received in step S32 as a search key, thereby reading out the acquired data associated with the location information included in the request.
- the communication unit 51 transmits the acquired data read in step S42 to the evaluation device 3 (step S43).
- This acquired data includes the captured image data, sensor data, and time data.
- the communication unit 31 of the evaluation device 3 receives the acquired data transmitted from the data management device 5.
- the evaluation target data generation unit 35 of the evaluation device 3 generates evaluation target data using the acquired data received in step S43 (step S44). Specifically, the evaluation target data generation unit 35 performs tilt correction of the captured image data from the attitude of the imaging device 7 (mobile body 6) at the time of imaging, based on the received sensor data of the distance sensor 8a. In addition, the evaluation target data generation unit 35 links the captured image data to the positioning data, which is the sensor data received from the GNSS sensor 8b, based on the received time data. Furthermore, the evaluation target data generation unit 35 performs processing to combine multiple captured image data into one image data.
- the evaluation target data generation unit 35 generates a composite image by stitching together the images captured in each of the multiple capture areas, thereby obtaining the captured images of the target area 70 and each of the multiple target areas 701A, 702A, 701B, and 702B.
- the evaluation target data generating unit 35 also generates a composite image by stitching together the captured images of the multiple target areas 701A, 702A, 701B, and 702B, thereby obtaining a captured image of the entire target area 70.
- the target area 70 includes the slope 80 and the area other than the slope 80.
- the evaluation target data generation unit 35 has a tilt correction function for image data, a function for linking image data with position information, and a function for synthesizing image data. Using the acquired data received from the data management device 5, the evaluation target data generation unit 35 performs image correction on the received captured image data so that processing by the detection unit 36 and report generation unit 38, which will be described later, can be easily performed.
- the evaluation target data generation unit 35 generates an input/output screen including a composite image.
- This input/output screen is an example of a display screen that displays a composite image obtained by stitching together images captured by dividing the target area 70 into a plurality of shooting areas dn along the moving direction of the moving body 6, and step S44 is an example of a generation step.
- the display control unit 33 displays the generated input/output screen on the display 306, and the reception unit 32 of the evaluation device 3 receives a predetermined input operation by the user on the displayed input/output screen.
- This input operation includes a decision operation for deciding to identify a partial area in the composite image.
- the setting unit 40 updates the generated evaluation target data based on the input information related to the input operation.
- the setting unit 40 is an example of a setting means.
- the setting unit 55 updates the evaluation target data by setting a partial image corresponding to the partial area, position information, and a specific point group in a three-dimensional point cloud corresponding to multiple shooting areas dn, based on specific area information that identifies a partial area in the composite image.
- the communication unit 31 of the evaluation device 3 transmits the generated data generated and updated in step S44 to the data management device 5 (step S45).
- This generated data includes the evaluation target data, positioning data, and comments generated by the evaluation target data generation unit 35 and updated by the setting unit 55.
- the communication unit 51 of the data management device 5 receives the generated data transmitted from the evaluation device 3.
- the data management unit 53 of the data management device 5 stores the generated data received in step S35 in the processing data management DB 5003 (see FIG. 9(B)) (step S46).
- the data management unit 53 associates the evaluation target data, positioning data, and comments included in the generated data and stores them in one folder.
- the evaluation system 4 performs image processing based on the various data (captured image data, sensor data, and time data) acquired from the data acquisition device 9, thereby generating and updating the evaluation target data used to evaluate the slope condition.
- FIG. 22 is a sequence diagram showing an example of a process for generating a report that is an evaluation result of the slope condition.
- the display control unit 33 of the evaluation device 3 displays the evaluation screen 400 for performing the evaluation process of the slope condition on the display 306 (step S51).
- the reception unit 32 of the evaluation device 3 receives the selection of the data to be evaluated (step S52).
- the communication unit 31 transmits a read request for the evaluation target data selected in step S52 to the data management device 5 (step S53).
- This read request includes the folder name selected in step S52.
- the communication unit 51 of the data management device 5 receives the read request transmitted from the evaluation device 3.
- the storage/reading unit 59 of the data management device 5 searches the processing data management DB 5003 (see FIG. 9(B)) using the folder name included in the read request received in step S53 as a search key, thereby reading out the processing data associated with the folder name included in the read request (step S54).
- the communication unit 51 then transmits the processing data read out in step S54 to the evaluation device 3 (step S55).
- This processing data includes the evaluation target data, positioning data, and comments.
- the communication unit 31 of the evaluation device 3 receives the processing data transmitted from the data management device 5.
- the display control unit 33 of the evaluation device 3 displays the processing data received in step S54 on the display 306 (step S56).
- the evaluation device 3 performs a process for detecting the slope condition using the evaluation target data (step S57). Details of the process for detecting the slope condition will be described later.
- the reception unit 32 receives a request to upload the evaluation results (step S58). Then, the communication unit 31 uploads (transmits) the evaluation results to the data management device 5 (step S59). As a result, the communication unit 51 of the data management device 5 receives the evaluation data transmitted from the evaluation device 3. Then, the data management unit 53 of the data management device 5 registers the evaluation data received in step S59 in the processing data management DB 5003 (see FIG. 9 (B)) (step S60). In this case, the data management unit 53 stores the evaluation data in a single folder in association with the evaluation target data that was evaluated, etc.
- the reception unit 32 also receives a request to generate an evaluation report (step S61).
- the report generation unit 38 then generates an evaluation report based on the detection results of the slope condition by the detection unit 36 (step S62).
- the report generation unit 38 generates an evaluation report by arranging the evaluation data indicating the above-mentioned evaluation results based on the inspection guidelines issued by the government or a format in accordance with the request of the road administrator.
- FIG. 23 is a flowchart showing an example of the process of detecting the slope condition.
- the reception unit 32 receives a shape detection request (step S71).
- the detection unit 36 performs a shape detection process using the evaluation target data (step S72).
- the shape data indicating the shape of the slope is represented by three-dimensional information such as the extension, height, and inclination angle of the slope, as well as position information.
- the extension of the slope is the length of the slope in the plan view (the length in the depth direction of the cross section where the inclination of the slope can be seen).
- the shape data also includes information indicating the type of slope, whether it is a natural slope or an earthwork structure.
- the shape data also includes information on the type of earthwork structure.
- the type of civil engineering structure is, for example, a retaining wall, a slope frame, mortar spraying, the presence or absence of an anchor, or an embankment.
- the detection unit 36 detects the extension, height, and inclination angle of the slope based on the image data and three-dimensional data included in the evaluation target data.
- the detection unit 36 also detects the type of slope shown in the image, which is the evaluation target data, using the condition type management DB 3001 (see FIG. 7). In this case, the detection unit 36 detects the type of slope by image matching processing using the teacher image shown in the condition type management table.
- the display control unit 33 causes the display 306 to display the shape data that is the detection result in step S72 (step S73). Note that in steps S71 to S73 described above, a "structure information detection” process may be performed instead of the "shape detection” process.
- the reception unit 32 receives a structure information detection request (step S71).
- the detection unit 36 performs a structure information detection process using the evaluation target data (step S72).
- the display control unit 33 causes the display 306 to display the structure information detection information, which is the detection result in step S72 (step S73).
- the structure information includes additional information about the structure in addition to the shape data described above.
- the detection unit 36 detects the type of slope shown in the image, which is the evaluation target data, and the type of additional information about the slope, using the condition type management DB 3001 (see Figures 7 and 8), based on the image data and three-dimensional data included in the evaluation target data. In this case, the detection unit 36 detects the type of slope and the additional information about the slope by image matching processing using the teacher image shown in the condition type management table.
- step S74 if the reception unit 32 receives a damage detection request requesting damage detection of the slope condition (YES in step S74), the process proceeds to step S75. On the other hand, if the reception unit 32 does not receive a damage detection request (NO in step S74), the process proceeds to step S77.
- the detection unit 36 performs damage detection processing of the slope condition for the evaluation target data (step S75).
- the slope condition damage detection process detects the presence or absence of deformation on the slope or the degree of deformation as damage data representing the degree of damage to the slope.
- the degree of deformation indicates the degree of deterioration of the deformation, and is the width of the crack, the size of the separation, or the size of the lift, etc.
- the detection unit 36 detects the presence or absence of deformation on the slope or the degree of deformation based on the image data and sensor data included in the evaluation target data. (An example of an evaluation step)
- the detection unit 36 also detects whether the degree of deformation exceeds a predetermined value using a predetermined detection formula for the degree of deterioration of deformation, etc. In this case, the detection unit 36 determines whether the crack width is equal to or larger than a certain value, whether the size of the peeling is equal to or larger than a certain value, whether the lift is large, etc.
- step S38 shown in FIG. 15 the data management unit 53 of the data management device 5 stores the coordinates of the damage location and the type of damage in the processing data management DB 5003, in association with the coordinates corresponding to the X-axis direction and the Y-axis direction in the captured image data 7A shown in FIG. 11.
- the display control unit 33 causes the display 306 to display a display screen showing the damage detection results in step S75 (step S76).
- the display control unit 33 also causes the display 306 to display a cross-sectional image.
- the cross-sectional image shows a cross-sectional view of the slope to be evaluated, drawn based on the shape data detected by the detection unit 36.
- the shape data is detected using sensor data from the distance sensor 8a (three-dimensional sensor), so it is possible to display in detail, including three-dimensional information such as the slope or height of the slope, which cannot be calculated from a two-dimensional image alone.
- the reception unit 32 receives a request to obtain map information (YES in step S77), it transitions the process to step S78. On the other hand, if the reception unit 32 does not receive a request to obtain map information (NO in step S77), it terminates the process.
- the detection unit 36 generates map information indicating the position of the slope condition to be evaluated (step S78). Specifically, the detection unit 36 generates map information in which an image indicating the position of the slope is added to the position (latitude, longitude) indicated by the positioning data obtained in step S55, which corresponds to map data available using a specified service or application provided by an external web server, etc. The map data provided from an external web server, etc. is managed by the map data management unit 37.
- the display control unit 33 causes the map information 490 generated in step S78 to be displayed on the display 306 (step S79).
- reception unit 32 If the reception unit 32 receives a sign detection request requesting detection of signs of damage to the slope condition (YES in step S80), it transitions the process to step S81. On the other hand, if the reception unit 32 does not receive a sign detection request (NO in step S80), it terminates the process.
- the detection unit 36 performs a sign detection process for the slope condition on the evaluation target data (step S81).
- condition inspection system 1 it has been customary to identify the condition and location of a slope when deformation is detected.
- the idea of measuring information that indicates the location of a slope deformation before the deformation occurs has not been known.
- the process of detecting signs of damage to the slope condition detects signs of deformation of the slope based on slope measurement data that includes surrounding data that indicates physical quantities around the slope as signs of damage to the slope.
- the measurement data includes photographed image data of the slope captured by the photographing device 7, or sensor data of the slope measured by a three-dimensional sensor such as the distance sensor 8a.
- the surrounding data includes measurement data of objects other than the slope, and the objects other than the slope include at least one of spring water, soil, rocks, and plants.
- the measurement data for the slope includes surrounding data that indicates spring water occurring on the surface of the slope, it is possible that stagnant water is exerting pressure from the back side of the slope, and it is detected that there are signs of deformation of the slope. Specifically, it is not limited to the presence or absence of spring water, but the amount, type and location of the spring water that will detect the signs of deformation of the slope.
- the measurement data for the slope includes surrounding data that indicates plants or moss growing on the surface of the slope, it is possible that spring water has occurred and that stagnant water is exerting pressure from the back side of the slope, and this is detected as a sign of deformation of the slope. Specifically, it is detected that there are signs of deformation of the slope not only based on the presence or absence of plants or moss, but also based on the amount, type and location of the plants and moss.
- the measurement data for a slope includes surrounding data that indicates fallen rocks and soil around the slope, it is possible that an abnormality has occurred on the rear or upper side of the slope, and it is therefore detected that there are signs of deformation of the slope. Specifically, it is not limited to the presence or absence of fallen rocks and soil, but rather the amount, type and location of the fallen rocks and soil that are detected as signs of deformation of the slope.
- the measurement data for the slope includes surrounding data that indicates blockages in drainage holes, pipes, berm drainage channels, etc.
- the measurement data for the slope includes surrounding data that indicates blockages in drainage holes, pipes, berm drainage channels, etc.
- the measurement data of objects other than the slope described above may be combined to detect signs of deformation of the slope. Specifically, even if surrounding data indicating spring water exists only in a small part of the slope, if moss is spread over the entire slope, it is estimated that spring water spreads over the entire slope on a daily basis, and a sign of deformation of the slope is detected.
- the ambient data includes measurement data of physical quantities other than objects
- the measurement data of physical quantities other than objects includes measurement data of light
- the process of detecting signs of damage to the slope condition generates comments about signs of deformation of the slope based on the measurement data of the slope, including surrounding data indicating physical quantities around the slope, as signs data indicating signs of damage to the slope.
- the data management unit 53 of the data management device 5 stores the coordinates of the position of the signs of deformation and the comments in the processed data management DB 5003, in association with the coordinates corresponding to the X-axis and Y-axis directions in the captured image data 7A shown in FIG. 11.
- the system references the teacher image in the state type management table shown in Figure 8 and generates a comment indicating the type of physical quantity around the slope, such as spring water, as well as its amount and location. As an example, the system generates a comment such as "moss rate 30%, mostly distributed around 3-20m above the starting point.”
- the display control unit 33 causes the display 306 to display a display screen showing the sign detection result in step S81 (step S82).
- the display control unit 33 also displays the cross-sectional image on the display 306.
- the evaluation system 4 detects the shape of the slope including three-dimensional information, the degree of damage to the slope, signs of deformation of the slope, and the position of the slope to be evaluated, as an evaluation of the slope condition.
- Figure 24 is a sequence diagram showing an example of display processing in a status inspection system.
- the reception unit 32 of the evaluation device 3 accepts the selection of the target data (step S91).
- the user of the evaluation device 3 may select an arbitrary position in the map information managed by the map data management unit 37 of the evaluation device 3, and the reception unit 32 of the evaluation device 3 may accept the selection of position information in the map information.
- the communication unit 31 transmits a request for an input/output screen related to the target data selected in step S91 to the data management device 5, and the communication unit 51 of the data management device 5 receives the request transmitted from the evaluation device 3 (step S92).
- This request includes the folder name selected in step S91. Alternatively, this request may include location information in the map information.
- the storage/reading unit 59 of the data management device 5 searches the processing data management DB 5003 (see FIG. 9(B)) using the folder name included in the request received in step S92 as a search key, thereby reading out image data associated with the folder name included in the request.
- the storage/reading unit 59 searches the acquisition data management DB 5001 using the location information included in the request received in step S92 as a search key, thereby reading out image data associated with the location information included in the request.
- the generating unit 54 of the data management device 5 generates an input/output screen including the image data based on the image data read by the storing/reading unit 59 (step S93).
- This input/output screen is a screen that accepts an instruction operation to generate an image showing a specific position in a luminance image showing a slope.
- the communication unit 51 transmits input/output screen information related to the input/output screen generated in step S93 to the evaluation device 3, and the communication unit 31 of the evaluation device 3 receives the input/output screen information transmitted from the data management device 5 (step S94).
- Step S94 is an example of a decision acceptance screen transmission step.
- the display control unit 33 of the evaluation device 3 displays the input/output screen received in step S94 on the display 306 (step S95).
- the reception unit 32 of the evaluation device 3 receives a predetermined input operation by the user on the displayed input/output screen.
- This input operation includes an instruction operation to generate an image showing a specific position in a luminance image showing a slope.
- Step S95 is an example of a reception step.
- the communication unit 31 transmits input information relating to the input operation received by the reception unit 32 to the data management device 5, and the communication unit 51 of the data management device 5 receives the input information transmitted from the evaluation device 3. (Step S96).
- This input information includes instruction information that instructs the generation of an image showing a specific position in the luminance image showing the slope.
- the generating unit 54 of the data management device 5 generates a display image using the image data read by the storing and reading unit 59 in step S93 based on the received input information (step S97).
- This display image includes a surface display image including a surface image showing the surface of the slope and a surface position image showing a specific position in the surface image, and a cross-section display image including a cross-section image showing the cross-section of the slope and a cross-section position image showing a specific position in the cross-section image.
- Step S97 is an example of an image generating step.
- Step S98 is an example of a display image transmission step.
- Step S99 is an example of a display step.
- FIG. 24 shows the sequence of the display process between the evaluation device 3 and the data management device 5, but the evaluation device 3 may execute the display process independently.
- steps S92, 94, 96, and 98 relating to data transmission and reception are omitted, and the evaluation device 3 can perform the same display processing as in FIG. 24 by independently executing steps S91, 93, 95, 97, and 99.
- the data acquisition device 9, communication terminal 1100, and communication terminal 1200 can also independently execute display processing, similar to the evaluation device 3.
- FIG. 25 is an explanatory diagram of an operation on a display screen of a state inspection system.
- Fig. 25 shows an input/output screen 2000 displayed on the display 306 of the evaluation device 3 in step S95 of the sequence diagram shown in Fig. 24, but the same is true for the input/output screen 2000 displayed on each display of the data acquisition device 9, the communication terminal 1100, and the communication terminal 1200.
- the display control unit 33 of the evaluation device 3 displays an input/output screen 2000 including a specific reception screen 2010 that receives a designation operation for designating a specific position in a luminance image showing a slope, and a decision reception screen 2020 that receives a decision operation for deciding to generate an image showing a specific position on the slope.
- the display control unit 33 displays a surface image 2100 showing the surface of the slope on the specific reception screen 2010, and also displays a pointer 2300 operated by the pointing device 312 on the surface image 2100.
- the surface image 2100 is a luminance image read out in step S93 of FIG. 24 from the captured image data shown in FIG. 9(A), and the display control unit 33 displays the surface image 2100 in association with the captured images 1 and 2 shown in FIG. 10 and the X-axis direction and Y-axis direction shown in the captured image data 7A shown in FIG. 11.
- the display control unit 33 displays a decision acceptance screen 2020 including a specific position decision button 2400, a deformation confirmation button 2410, a deformation sign confirmation button 2420, a front view analysis button 2430, a front view comparison button 2440, a cross-sectional view analysis button 2450, and a cross-sectional view comparison button 2460.
- the deformation confirmation button 2410, the deformation sign confirmation button 2420, the front view analysis button 2430, the front view comparison button 2440, the cross-sectional view analysis button 2450, and the cross-sectional view comparison button 2460 are buttons that instruct the generation of an image showing a specific position on the slope, with the position of a part in the surface image 2100 or the cross-sectional image 2200 that satisfies a specified condition as the specific position.
- the specific position determination button 2400 is a button that instructs the system to confirm the specific position on the slope specified on the specific reception screen 2010 and generate an image showing the specific position on the slope.
- the specific position determination button 2400 may determine not only the specific position specified on the specific reception screen 2010, but also a specific position that has been determined by the determination unit 52 or the like and displayed on the specific reception screen 2010.
- the Deformation Confirmation button 2410 is a button that instructs the system to generate an image showing a specific position on the slope, with a position indicating a deformation of the slope set as the specific position
- the Deformation Sign Confirmation button 2420 is a button that instructs the system to generate an image showing a specific position on the slope, with a position indicating a deformation of the slope set as the specific position.
- the front view analysis button 2430 is a button that instructs the system to generate an image showing a specific position on the slope by specifying a portion obtained by analyzing the surface image 2100 as the specific position
- the front view comparison button 2440 is a button that instructs the system to generate an image showing a specific position on the slope by specifying a portion obtained by comparing the surface image 2100 with another image as the specific position.
- the cross-sectional view analysis button 2450 is a button that instructs the system to generate an image showing a specific position on the slope by specifying a portion obtained by analyzing the cross-sectional image (described later) as the specific position
- the cross-sectional view comparison button 2460 is a button that instructs the system to generate an image showing a specific position on the slope by specifying a portion obtained by comparing the cross-sectional image with another image as the specific position.
- FIG. 26 is a flowchart showing the processing based on the operation shown in FIG. 25.
- FIG. 26(a) shows the processing in the evaluation device 3
- FIG. 26(b) shows the processing in the data management device 5.
- the reception unit 32 of the evaluation device 3 receives the pointing operation (step S101), and when the specific position determination button 2400 is operated, the reception unit 32 receives the operation (step S102).
- the judgment unit 34 of the evaluation device 3 detects the XY coordinates of the pointed position in the surface image 2100 as a specific position (step S103).
- This specific position may indicate a point in the XY coordinates, or may indicate an area.
- the communication unit 31 of the evaluation device 3 transmits input information related to the input operation received by the reception unit 32 to the data management device 5 (step S104).
- This input information includes designation information for designating a specific position in XY coordinates based on a pointing operation using the pointer 2300, and instruction information for instructing the generation of an image showing the specific position on the slope based on the operation of the specific position confirmation button 2400.
- the communication unit 51 of the data management device 5 receives the input information sent from the evaluation device 3, and the generation unit 54 uses the image data shown in FIG. 11(A) based on the instruction information and specification information contained in the received input information to generate a surface position image that overlaps with the XY coordinates of the specific position by superimposing it on the surface image to generate a surface display image (step S105).
- the surface position image does not necessarily have to completely match the XY coordinates of the specific position, as long as it overlaps with the XY coordinates of the specific position.
- the generating unit 54 generates a cross-sectional image corresponding to the X-coordinate of the specific position using the image data shown in FIG. 11(A) and the distance measurement data shown in FIG. 11(B) (step S106). If the distance measurement data shown in FIG. 11(B) does not include the X-coordinate of the specific position, the generating unit 54 generates a cross-sectional image based on data in the vicinity of the X-coordinate of the specific position included in the distance measurement data shown in FIG. 11(B).
- step S106 the generating unit 54 generates a cross-sectional image of a cross-section including the Z-axis direction and the vertical direction shown in FIG. 10, but it may also generate a cross-sectional image of a cross-section including the Z-axis direction and a direction inclined from the vertical direction, or a cross-sectional image of a cross-section including a direction inclined from the Z-axis direction.
- the generating unit 54 generates a cross-sectional position image that overlaps with the Y coordinate of the specific position by superimposing it on the edge line of the cross-sectional image, and generates a cross-sectional display image (step S107).
- the communication unit 31 of the evaluation device 3 receives the surface display image and cross-sectional display image transmitted from the data management device 5, and the display control unit 33 of the evaluation device 3 displays the received surface display image and cross-sectional display image on the display 306.
- FIG. 27 is an example of a display screen after the processing shown in FIG. 26.
- FIG. 27 shows an input/output screen 2000 that is displayed on the display 306 of the evaluation device 3 in step S99 of the sequence diagram shown in FIG. 24.
- the display content of the decision reception screen 2020 is the same as that of FIG. 25, but the display content of the specific reception screen 2010 is different from that of FIG. 25.
- the display control unit 33 of the evaluation device 3 displays, on the specific reception screen 2010, a surface display image 2150 including a surface image 2100 showing the surface of the slope and a surface position image 2110 showing a specific position in the surface image 2100, and a cross-section display image 2250 including a cross-section image 2200 showing the cross-section of the slope and a cross-section position image 2210 showing a specific position in the cross-section image 2200.
- the display control unit 33 displays the cross-sectional image 2200 in association with the Y-axis direction and Z-axis direction shown in FIG. 10.
- the user can appropriately evaluate and confirm the condition of a specific position by comparing the surface position image 2110 with the cross-sectional position image 2210.
- FIG. 28 shows a modified example of the functional configuration of the status inspection system.
- the data management device 5 includes a judgment unit 534, evaluation target data generation unit 535, detection unit 536, map data management unit 537, report generation unit 538, and setting unit 540.
- the determination unit 534, evaluation target data generation unit 535, detection unit 536, map data management unit 537, report generation unit 538, and setting unit 540 shown in FIG. 28 are functions or means similar to those of the determination unit 34, evaluation target data generation unit 35, detection unit 36, map data management unit 37, report generation unit 38, and setting unit 40 shown in FIG. 6, respectively.
- the storage unit 5000 of the data management device 5 is provided with a state type management DB 5005.
- the status type management DB 5005 shown in FIG. 28 manages the same data as the status type management DB 3001 shown in FIG. 6.
- FIG. 29 is a flowchart showing the processing in the modified example shown in FIG. 28.
- Figure 29(a) shows the processing in the data management device 5.
- the detection unit 536 uses the state type management DB 3001 (see FIG. 7) to detect the type of slope shown in the composite image shown in FIG. 16 (step S201), similar to the process of the detection unit 36 in step S72 in FIG. 23.
- the detection unit 536 can detect multiple types of slopes.
- the generating unit 54 generates a detection data display screen including the detection data detected in step S201 (step S202).
- the generating unit 54 can generate a detection data display screen including multiple detection data.
- the detection unit 536 estimates the boundary between the slope 80 and the surface other than the slope 80, i.e., the start position and end position of the slope 80 in the direction of movement of the mobile object 6, based on the detection data detected in step S201 (step S203).
- the detection unit 536 can estimate multiple combinations of start positions and end positions.
- the generation unit 54 identified a location where the boundary between the slope 80 and the surface other than the slope 80 may exist, but in the modified example shown in FIG. 28, the detection unit 536 estimates the boundary between the slope 80 and the surface other than the slope 80.
- the generation unit 54 Based on the start position and end position estimated in step S203, the generation unit 54 generates an input/output screen in which a start position bar and an end position bar are superimposed on the composite image, similar to the input/output screen 2000 shown in Figures 17 and 18 (step S204).
- the generation unit 54 can generate an input/output screen in which multiple combinations of start position bars and end position bars are superimposed on the composite image.
- the generating unit 54 generates a map screen in which an image indicating the start position and an image indicating the end position are superimposed on the map data, similar to the map information generated in step S78 of FIG. 23, based on the start position and end position estimated in step S203 (step S205).
- the generating unit 54 can generate a map screen in which multiple combinations of images indicating the start position and images indicating the end position are superimposed.
- the communication unit 51 transmits to the evaluation device 3 detection data display screen information indicating the detection data display screen generated in step S202, input/output screen information indicating the input/output screen generated in step S204, and map screen information indicating the map screen generated in step S205 (step S206).
- the communication unit 51 can also transmit this information to the data acquisition device 9, the communication terminal 1100, and the communication terminal 1200.
- FIG. 29(b) shows the processing in the evaluation device 3.
- the processing in the data acquisition device 9, the communication terminal 1100, and the communication terminal 1200 is similar.
- the communication unit 31 receives the detection data display screen information, the input/output screen information, and the map screen information transmitted from the data management device 5 (step S211).
- the display control unit 33 causes the display 306 to display the detection data display screen indicated in the detection data display screen information received in step S211 (step S212).
- the display control unit 33 causes the display 306 to display the input/output screen indicated in the input/output screen information received in step S211 so as to include the detection data selected in step S213 (step S214).
- the display control unit 33 causes the display 306 to display the divided image group including the detection data.
- the input/output screen displayed in step S214 is similar to the input/output screen 2000 shown in FIG. 17, but in FIG. 17, when the user operates the start position designation button 2402 and the end position designation button 2404, the display control unit 33 displays the start position bar 250S and the end position bar 250G at any position on the composite image 2500, whereas in the input/output screen displayed in step S214, the display control unit 33 displays the start position bar 250S and the end position bar 250G at the start position and end position estimated in step S203 of FIG. 29(a) on the composite image 2500.
- start position bar 250S is an example of a first marker that indicates the estimated position of the boundary between one end of the slope 80 and something other than the slope 80
- end position bar 250G is an example of a second marker that indicates the estimated position of the boundary between the other end of the slope 80 and something other than the slope 80.
- the display control unit 33 causes the display 306 to display the map screen indicated in the map screen information received in step S211, including the detection data selected in step S213 (step S215).
- FIG. 30 shows an example of a detection data display screen in the modified example shown in FIG. 28.
- FIG. 30 shows the detection data display screen 3000 displayed on the display 306 of the evaluation device 3 in step S212 of the flowchart shown in FIG. 29, but the same is true for the detection data display screens displayed on the displays of the data acquisition device 9, communication terminal 1100, and communication terminal 1200.
- the detection data display screen 3000 is an example of a type display screen.
- the display control unit 33 of the evaluation device 3 causes the display 306 to display the detection data display screen 3000, which includes text information 3100A-3100D indicating the multiple detection data detected in step S201 of FIG. 29(a) and image information 3200A-3200D.
- the text information 3100A-3100D includes text information relating to the type of slope and the construction method.
- the text information 3100A-3100D is an example of type information.
- the reception unit 32 of the evaluation device 3 receives a selection operation for the detected data pointed to, as shown in step S213 of FIG. 29(b).
- the display control unit 33 may switch the display of the detection data display screen 3000 to the input/output screen 2000 shown in FIG. 17, or may display it in a separate window.
- FIG. 31 shows an example of a map screen in the modified example shown in FIG. 28.
- FIG. 31 shows the map screen 490 displayed on the display 306 of the evaluation device 3 in step S215 of the flowchart shown in FIG. 29, but the same is true for the map screens displayed on the displays of the data acquisition device 9, the communication terminal 1100, and the communication terminal 1200.
- the display control unit 33 causes the display 306 to display a map screen 490 including a photography path 492 including a photography start position 492a and a photography end position 492b, a start position 491a of the slope 80 in the moving direction of the mobile body 6, and an end position 491b of the slope 80 in the moving direction of the mobile body 6.
- the start position 491a is an example of one end of the slope 80
- the end position 491b is an example of the other end of the slope 80.
- the imaging path 492 corresponds to the imaging position of the composite image described in FIG. 16 etc., and includes the imaging position of the detection data selected in step S213 of FIG. 29(b).
- start position 491a and the end position 491b correspond to the start position and the end position estimated in step S203 of FIG. 29(a).
- the display control unit 33 may switch the display of the map screen 490 to the input/output screen 2000 shown in FIG. 17, or may display it in a separate window.
- Fig. 32 is a diagram showing an example of a state in which a slope condition is inspected using a mobile body system according to Modification 1.
- the mobile body system 60 according to Modification 1 is a system in which a data acquisition device 9 is fixed to a pole installed on the upper surface of a mobile body 6 to enable photography at high altitudes.
- the imaging device 7 of the above-mentioned embodiment is low in height from the ground, and it is difficult to photograph the berms on retaining walls, berms on crenellations, or berms on sprayed mortar as shown in FIG. 32. Furthermore, the berms of current road earthwork structures are not covered as shown in FIG. 32, and there is a risk of dead leaves and the like accumulating and clogging the waterway, which requires regular cleaning.
- the mobile body system 60 according to variant 1 which is capable of imaging from a high place, even in cases where it is difficult for a person to climb a slope to check the degree of clogging of the waterway, for example, it is possible to check by imaging processing associated with the traveling movement of the mobile body 6, and therefore inspection efficiency can be significantly improved.
- ⁇ Variation 2 ⁇ 33 is a diagram showing an example of a state in which a slope condition is inspected using a mobile body system according to Modification 2.
- the mobile body system 60 (60a, 60b) according to Modification 2 is a system that uses a drone equipped with a data acquisition device 9 as an example of a mobile body 6 to photograph an embankment slope at a high place or below the roadside that cannot be photographed even by the pole-mounted photographing device of Modification 1.
- the drone as the mobile body 6 is equipped with not only the imaging device 7 but also a data acquisition device 9 equipped with sensor devices such as a distance sensor 8a, a GNSS sensor 8b, or an angle sensor 8c, making it possible to evaluate the condition of high places and embankments that could not be evaluated by a vehicle as the mobile body 6.
- embankments and high places are places where it is difficult for humans to go and visually inspect them up close, so it is desirable to photograph them with a drone like that of variant 2.
- the slopes of embankments and high places are often covered with a lot of vegetation such as trees and grass. For this reason, it is preferable for the data acquisition device 9 to be equipped with an imaging device 7 capable of taking wide-angle images.
- step S123 of FIG. 25(a) it is also desirable for the drone to travel along a path that does not deviate as much as possible from the path planned in step S122 when taking photographs.
- Fig. 34 is a diagram showing an example of inspecting a slope condition using the mobile body system according to the modified example 3. As shown in Fig. 34, a slope has a complex structure, unlike a tunnel or a bridge, which are structures on a road.
- the slope may be undulating rather than flat (e.g., an earthwork structure with mortar sprayed onto a cliff), covered with vegetation, or covered with wire mesh.
- the mobile system 60 (60a, 60b, 60c) of the third modification is equipped with a sensor device 8 that is a spectral camera, an infrared camera, or an expanded depth of field camera (EDof (Expanded Depth of Field) camera) capable of acquiring wavelength information in order to distinguish between objects such as plants and wire mesh and the shape of the slope.
- a sensor device 8 that is a spectral camera, an infrared camera, or an expanded depth of field camera (EDof (Expanded Depth of Field) camera) capable of acquiring wavelength information in order to distinguish between objects such as plants and wire mesh and the shape of the slope.
- EDof Expanded Depth of Field
- the mobile system 60 according to the third modification is configured to be configured not only as a tool for distinguishing the shape of the slope, but also by mounting a lighting device on the data acquisition device 9 so that the slope can be photographed under various conditions such as weather and sunlight.
- the lighting device is preferably a line lighting device that illuminates an area corresponding to the range photographed by the photographing device 7, or a time-sharing lighting device synchronized with the photographing device 7 and the sensor device 8.
- the evaluation target data generating unit 35 of the evaluation device 3 has image processing functions such as a camera shake correction function, a focal depth correction function (blur correction function), a distortion correction function, or a contrast enhancement function so as not to miss even small abnormalities. It is also preferable that the evaluation target data generating unit 35 has a function to remove noise that conceals abnormalities on earthwork structures such as grass, moss, or wire mesh, or a function to distinguish between shadows of grass, etc. and abnormalities such as cracks. In this way, by using the mobile system 60 according to the third modification, the condition inspection system 1 can accurately evaluate the condition of slopes even in places with complex structures or where grass, moss, wire mesh, etc. are present.
- a data management device 5 includes a generation unit 54 that generates an input/output screen 2000 that displays a composite image 2500 including the boundary between the slope 80 and other surfaces in the direction of movement of the moving body 6, by connecting together each of the captured images pn captured by a photographing device 7 installed on a moving body 6, the captured image being divided into a plurality of photographing areas dn along the direction of movement of the moving body 6, the boundary being between the slope 80 and other surfaces in the direction of movement of the moving body 6.
- the data management device 5 is an example of an information processing device
- the slope 80 is an example of an object
- the input/output screen 2000 is an example of a display screen
- the generation unit 54 is an example of a generation means.
- the generation unit 54 generates the input/output screen 2000 so that the composite image 2500 includes boundaries of a plurality of slopes 80 at different positions in the movement direction of the moving body 66 with respect to areas other than the slopes 80 .
- the generation unit 54 generates the input/output screen 2000 so that the length of the composite image 2500 in the direction of movement of the moving body 66 corresponding to the distance traveled by the moving body 66 differs depending on the resolution of the display 306 or the like on which the input/output screen 2000 is displayed.
- the generation unit 54 generates an input/output screen 2000 that displays a start position bar 250S and an end position bar 250G, which are examples of markers indicating the estimated positions of the boundary, superimposed on a composite image 2500.
- the generating unit 54 generates an input/output screen 2000 that displays, on one screen or one line, a first marker indicating an estimated position of the boundary at one end of the slope 80 and a second marker indicating an estimated position of the boundary at the other end of the slope 80.
- the start position bar 250S is an example of the first marker
- the end position bar 250G is an example of the second marker.
- the generation unit 54 generates a detection data display screen 3000 that displays text information 3100A to 3100D indicating the estimated type of the slope 80.
- the text information 3100A to 3100D is an example of type information
- the detection data display screen 3000 is an example of a type display screen. This allows the user to confirm the estimated type of the slope 80.
- the display control unit 33 of the evaluation device 3 causes the display 306 to display a captured image of the slope 80 corresponding to the selected text information 3100A-3100D from the composite image 2500, based on a selection operation for selecting the text information 3100A-3100D displayed on the display 306 or the image information 3200A-3200D corresponding to the text information 3100A-3100D.
- the data management device 5 includes a setting unit 55 that sets a partial image 255 corresponding to the partial area, based on a determination operation on the specific position determination button 2400 that determines to determine the determination of the partial area in the composite image 2500.
- the setting unit 55 is an example of a setting means.
- the generation unit 54 generates the input/output screen 2000 so as to display a plurality of divided images 250A1 to Am obtained by dividing the composite image 2500 side by side.
- the generation unit 54 generates the input/output screen 2000 so as to display the composite image 2500 at a lower resolution than each of the captured images pn captured in multiple shooting areas dn stored in the acquired data management DB 5001.
- the target area 70 includes a first target area and a second target area that are different ranges in a direction intersecting the moving direction of the moving body 66
- the generation unit 54 generates an input/output screen 2000 including at least one of a first composite image and a second composite image, the first composite image being obtained by stitching together first captured images pn obtained by dividing a first target area into a plurality of first captured images dn along the movement direction of the moving body 66, and a second composite image being stitched together second captured images pn obtained by dividing a second target area into a plurality of second captured images dn along the movement direction of the moving body 66.
- the data management device 5 is provided with a setting unit 55 that sets a first partial image corresponding to a partial area in the first composite image based on a first decision operation that decides to identify a partial area in the first composite image, and sets a second partial image corresponding to a partial area in the second composite image.
- the setting unit 55 sets an integrated partial image by joining the first partial image and the second partial image together.
- the setting unit 55 sets position information corresponding to a part of the area based on a confirmation operation.
- the setting unit 55 sets a specific point group corresponding to a part of the three-dimensional point groups corresponding to the multiple shooting regions dn based on a decision operation.
- An information processing method executes a generation step of generating an input/output screen 2000 that displays a composite image 2500 including the boundary between the slope 80 and surfaces other than the slope 80 in the direction of movement of the mobile body 66, by connecting together each of the captured images pn captured by an imaging device 7 installed on the mobile body 66, the target area 70 including the slope 80 and surfaces other than the slope 80 along the direction of movement of the mobile body 66.
- An information processing method includes a photographing step in which an imaging device 7 installed on a moving body 66 photographs a target area 70 including a slope 80 and areas other than the slope 80, dividing the target area 70 into multiple photographing areas dn along the direction of movement of the moving body 66, and a generation step in which each of the photographed images pn photographed in the multiple photographing areas dn is stitched together to generate an input/output screen 2000 that displays a composite image 2500 including the boundary between the slope 80 and areas other than the slope 80 in the direction of movement of the moving body 66.
- a program according to an embodiment of the present invention causes a computer to execute the information processing method according to the fourteenth or fifteenth aspect.
- a condition inspection system 1 comprises a mobile body system 60 having a mobile body 66 and an imaging device 7 installed on the mobile body 66, and a data management device 5 for processing images captured by the mobile body system 60.
- the mobile body system 60 uses the imaging device 7 to capture images of a target area 70 including a slope 80 and areas other than the slope 80, dividing the target area 70 into multiple imaging areas dn along the direction of movement of the mobile body 66.
- the data management device 5 comprises a generation unit 54 that connects together each of the captured images pn captured in the multiple imaging areas dn to generate an input/output screen 2000 that displays a composite image 2500 including the boundary between the slope 80 and areas other than the slope 80 in the direction of movement of the mobile body 66.
- the status inspection system 1 is an example of an information processing system
- the mobile system 60 is an example of an imaging system.
- the evaluation device 3, communication terminal 1100, or 1200 is further provided which is capable of communicating with the data management device 5, and the data management device 5 is further provided with a communication unit 51 which transmits input/output screen 2000 information indicating the input/output screen 2000 to the terminal device, and the evaluation device 3, communication terminal 1100, or 1200 is provided with a communication unit 31, 1101, or 1201 which receives the input/output screen 2000 information transmitted from the data management device 5, and a display control unit 33, 1103, or 1203 which displays the input/output screen 2000 on a display 306 or the like.
- the "processing circuit" in the present embodiment includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and devices such as an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), an SOC (System on a chip), a GPU (Graphics Processing Unit), and a conventional circuit module designed to execute each function described above.
- ASIC Application Specific Integrated Circuit
- DSP digital signal processor
- FPGA field programmable gate array
- SOC System on a chip
- GPU Graphics Processing Unit
- machine learning is a technology that allows a computer to acquire human-like learning capabilities, and refers to a technology in which a computer autonomously generates algorithms necessary for judgments such as data identification from learning data that is previously imported, and applies this to new data to make predictions.
- the learning method for machine learning may be any of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning, or may be a combination of these learning methods, and any learning method for machine learning may be used.
- the various tables in the embodiments described above may be generated using image processing techniques.
- image processing techniques include edge detection, line detection, and binarization.
- audio conversion techniques such as Fourier transform may be used.
- Status inspection system (an example of an information processing system) 3.
- Evaluation device (an example of a communication device) 4
- Evaluation system 5
- Data management device (an example of an information processing device) 6
- Moving object Photographing device
- Photographed image data (luminance image)
- Sensor device 8A
- Range-finding image data (three-dimensional point cloud)
- Distance sensor an example of a three-dimensional sensor
- GNSS sensor GNSS sensor
- Angle sensor an example of a three-dimensional sensor
- Data acquisition device (an example of a communication terminal) 92 Calculation unit 93
- Shooting device control unit (an example of an angle changing unit) 96
- Sensor data acquisition unit (an example of a distance information acquisition unit or a position information acquisition unit) 31
- Communication unit (an example of a receiving means) 32
- Reception unit (an example of an operation reception means) 33
- Display control unit (an example of a display control means) 35
- Evaluation target data generation unit an example of evaluation target data generation means)
- Detection unit (an example of a detection means)
- Report generation unit (an example of an evaluation information generation means)
- Communication unit (an example of a transmission means)
- Determination unit (an example of a position generating means)
- Generation unit (an example of an image generation means) 55
- Setting unit (an example of a setting means) 59
- Memory/read unit (an example of a memory control means) 60
- Mobile system (an example of a photography system) 71 to 73: Shooting device 70:
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Architecture (AREA)
- Civil Engineering (AREA)
- Structural Engineering (AREA)
- Processing Or Creating Images (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
Description
●システムの概略
まず、図1および図2を用いて、状態検査システムの概略について説明する。図1は、実施形態に係る状態検査システムの全体構成の一例を示す図である。図1に示されている状態検査システム1は、情報処理システムの一例であり、移動体システム60によって取得された各種データを用いて、道路土工構造物の状態の検査を行うためのシステムである。道路土工構造物とは、道路を建設するために構築する土砂や岩石等の地盤材料を主材料として構成される構造物およびそれらに附帯する構造物の総称であり、切土・斜面安定施設、盛土、カルバートおよびこれらに類するものをいう。以下、道路土工構造物を法面と称する。
次に、図4および図5を用いて、状態検査システム1を構成する各装置のハードウエア構成について説明する。なお、図4および図5に示されているハードウエア構成は、必要に応じて構成要素が追加または削除されてもよい。
図4は、データ取得装置のハードウエア構成の一例を示す図である。データ取得装置9は、図1に示されているような撮影装置7およびセンサ装置8とともに、データ取得装置9の処理または動作を制御するコントローラ900を備える。
図5は、評価装置のハードウエア構成の一例を示す図である。評価装置3の各ハードウエア構成は、300番台の符号で示されている。図5に示されているように、評価装置3は、コンピュータによって構築されており、図5に示されているように、CPU301、ROM302、RAM303、HD304、HDDコントローラ305、ディスプレイ306、外部機器接続I/F308、ネットワークI/F309、バスライン310、キーボード311、ポインティングデバイス312、DVD-RWドライブ314、およびメディアI/F316を備えている。
図5は、データ管理装置のハードウエア構成の一例を示す図である。データ管理装置5の各ハードウエア構成は、括弧内の500番台の符号で示されている。図5に示されているように、データ管理装置5は、コンピュータによって構築されており、図5に示されているように、評価装置3と同様の構成を備えているため、各ハードウエア構成の説明を省略する。なお、通信端末1100.1200も、コンピュータによって構築され、評価装置3と同様の構成を備えているが、各ハードウエア構成の説明は省略する。
次に、図6を用いて、実施形態に係る状態検査システムの機能構成について説明する。図6は、第1の実施形態に係る状態検査システムの機能構成の一例を示す図である。なお、図6には、図1に示されている装置のうち、後述の処理または動作に関連しているものが示されている。
まず、図6を用いて、データ取得装置9の機能構成について説明する。データ取得装置9は、通信部91、算出部92、撮影装置制御部93、センサ装置制御部94、撮影画像データ取得部95、センサデータ取得部96、時刻データ取得部97、要求受付部98および記憶・読出部99を有している。これら各部は、図4に示されている各構成要素のいずれかが、HD914からRAM913上に展開されたデータ取得装置用のプログラムに従ったCPU911からの命令によって動作することで実現される機能または手段である。また、データ取得装置9は、図4に示されているROM912およびHD914によって構築される記憶部9000を有している。また、図4に示されるデータ取得装置9に接続される外部PC930は、受付部および表示制御部を有する。
続いて、図6を用いて、評価装置3の機能構成について説明する。評価装置3は、通信部31、受付部32、表示制御部33、判断部34、評価対象データ生成部35、検出部36、地図データ管理部37、レポート生成部38および記憶・読出部39を有している。これら各部は、図5に示されている各構成要素のいずれかが、HD304からRAM303上に展開され評価装置用のプログラムに従ったCPU301からの命令によって動作することで実現される機能または手段である。また、評価装置3は、図5に示されているROM302およびHD304によって構築される記憶部3000を有している。
続いて、図6を用いて、データ管理装置5の機能構成について説明する。データ管理装置5は、通信部51、判断部52、データ管理部53および記憶・読出部59を有している。これら各部は、図5に示されている各構成要素のいずれかが、HD504からRAM503上に展開されデータ管理装置用のプログラムに従ったCPU501からの命令によって動作することで実現される機能または手段である。また、データ管理装置5は、図5に示されているROM502およびHD504によって構築される記憶部5000を有している。
○端末装置の機能構成○
図7および図8は、状態種別管理テーブルの一例を示す概念図である。状態種別管理テーブルは、法面の状態種別を検出するための教師データを管理するためのテーブルである。記憶部3000には、図7および図8に示されているような状態種別管理テーブルによって構成されている状態種別管理DB3001が構築されている。この状態種別管理テーブルでは、種別Noごとに、状態種別を示す種別名、教師画像、および備考欄が関連づけられて管理されている。
図9(A)は、取得データ管理テーブルの一例を示す概念図である。取得データ管理テーブルは、データ取得装置9によって取得された各種取得データを管理するためのテーブルである。記憶部5000には、図9(A)に示されているような取得データ管理テーブルによって構成されている取得データ管理DB5001が構築されている。この取得データ管理テーブルでは、フォルダごとに、撮影画像データ、センサデータおよび取得時刻が関連づけられて管理されている。
図9(B)は、処理データ管理テーブルの一例を示す概念図である。処理データ管理テーブルは、評価装置3によって処理された各種処理データを管理するためのテーブルである。記憶部5000には、図9(B)に示されているような処理データ管理テーブルによって構成されている処理データ管理DB5003が構築されている。この処理データ管理テーブルでは、フォルダごとに、評価対象データ、評価データ、測位データおよびコメントが関連づけられて管理されている。
○データ取得処理○
次に、図14を用いて、移動体システム60を用いたデータ取得処理について説明する。法面状態の点検作業者は、移動体6に搭乗して道路上の存在する法面の撮影を行い、取得したデータをデータ管理装置5にアップロードする。以下、詳細に説明する。
○評価対象データの生成
図15は、評価対象データの生成処理の一例を示すシーケンス図である。
図22は、法面状態の評価結果であるレポートの生成処理の一例を示すシーケンス図である。
図25は、状態検査システムの表示画面における操作の説明図である。図25は、図24に示したシーケンス図のステップS95において、評価装置3のディスプレイ306に表示される入出力画面2000を示すが、データ取得装置9、通信端末1100および通信端末1200の夫々のディスプレイに表示される入出力画面2000についても同様である。
表示制御部33は、ステップS211で受信した検出データ表示画面情報に示される検出データ表示画面をディスプレイ306に表示させる(ステップS212)。
○変形例1○
次に、図32乃至図34を用いて、移動体システム60の変形例について説明する。まず、図32は、変形例1に係る移動体システムを用いて法面状態を検査する様子の一例を示す図である。変形例1に係る移動体システム60は、高所の撮影を可能にするため、データ取得装置9が移動体6の上面に設置したポールに固定されているシステムである。
図33は、変形例2に係る移動体システムを用いて法面状態を点検する様子の一例を示す図である。変形例2に係る移動体システム60(60a,60b)は、例えば、変形例1のポール付き撮影装置でも撮影できないような高所または道路脇より下の盛土法面の撮影を行うために、移動体6の一例として、データ取得装置9を搭載したドローンを用いるシステムである。
図34は、変形例3に係る移動体システムを用いて法面状態を点検する様子の一例を示す図である。図34に示されているように、法面は、道路上の構造物であるトンネルや橋梁とは異なり、複雑な構造を持つ。
[第1態様]
本発明の一実施形態に係るデータ管理装置5は、移動体6に設置された撮影装置7により、法面80および法面80以外を含む対象領域70を移動体6の移動方向に沿って複数の撮影領域dnに分けて撮影したそれぞれの撮影画像pnをつなぎ合わせて、移動体6の移動方向における法面80と法面80以外の境界を含む合成画像2500を表示する入出力画面2000を生成する生成部54を備える。
第1態様において、生成部54は、合成画像2500が、移動体66の移動方向における異なる位置の複数の法面80の法面80以外に対する境界を含むように、入出力画面2000を生成する。
第1態様または第1態様の2において、生成部54は、入出力画面2000が表示されるディスプレイ306等の解像度に応じて、移動体66の移動距離に対応する移動体66の移動方向における合成画像2500の長さが異なるように、入出力画面2000を生成する。
第1態様において、生成部54は、境界の推定位置を示すマーカーの例である開始位置バー250Sおよび終了位置バー250Gを合成画像2500に重畳させて表示する入出力画面2000を生成する。
第2態様において、生成部54は、法面80の一端の境界の推定される位置を示す第1のマーカーと、法面80他端の境界の推定される位置を示す第2のマーカーを一画面または一ラインで表示する入出力画面2000を生成する。開始位置バー250Sは第1のマーカーの一例であり、終了位置バー250Gは第2のマーカーの一例である。
第1態様~第3態様の何れかにおいて、生成部54は、法面80の推定される種別を示すテキスト情報3100A~3100Dを表示する検出データ表示画面3000を生成する。テキスト情報3100A~3100Dは種別情報の一例であり、検出データ表示画面3000は種別表示画面の一例である。これにより、ユーザは、法面80の推定される種別を確認することができる。
第4態様において、評価装置3の表示制御部33は、ディスプレイ306に表示されたテキスト情報3100A~3100Dまたはテキスト情報3100A~3100Dに対応する画像情報3200A~3200Dを選択する選択操作に基づき、合成画像2500のうち選択されたテキスト情報3100A~3100Dに対応する法面80を撮影した撮影画像を、ディスプレイ306に表示させる。
第1態様~第5態様の何れかにおいて、データ管理装置5は、合成画像2500における一部の領域を特定することを決定する特定位置決定ボタン2400に対する決定操作に基づき、一部の領域に対応する部分画像255を設定する設定部55を備える。設定部55は、設定手段の一例である。
第1態様~第6態様の何れかにおいて、生成部54は、合成画像2500を分割した複数の分割画像250A1~Amのそれぞれの分割画像を並べて表示するように、入出力画面2000を生成する。
第1態様~第7態様の何れかにおいて、生成部54は、取得データ管理DB5001に記憶された複数の撮影領域dnに分けて撮影されたそれぞれの撮影画像pnよりも低い解像度で、合成画像2500を表示するように、入出力画面2000を生成する。
第1態様~第8態様の何れかにおいて、対象領域70は、移動体66の移動方向と交差する方向において異なる範囲である第1の対象領域および第2の対象領域を含み、
第1の対象領域を移動体66の移動方向に沿って複数の第1の撮影領域dnに分けて撮影したそれぞれの第1の撮影画像pnをつなぎ合わせた第1の合成画像、および第2の対象領域を移動体66の移動方向に沿って複数の第2の撮影領域dnに分けて撮影したそれぞれの第2の撮影画像pnをつなぎ合わせた第2の合成画像について、生成部54は、第1の合成画像および第2の合成画像の少なくとも一方を含む入出力画面2000を生成する。
第9態様において、データ管理装置5は、第1の合成画像における一部の領域を特定することを決定する第1の決定操作に基づき、第1の合成画像における一部の領域に対応する第1の部分画像を設定するとともに、第2の合成画像における一部の領域に対応する第2の部分画像を設定する設定部55を備える。
第10態様において、設定部55は、第1の部分画像と、第2の部分画像をつなぎ合わせた統合部分画像を設定する。
第6態様において、設定部55は、決定操作に基づき、一部の領域に対応する位置情報を設定する。
第6態様または第12態様において、設定部55は、決定操作に基づき、複数の撮影領域dnに対応する三次元点群について、一部の領域に対応する特定点群を設定する。
本発明の一実施形態に係る情報処理方法は、移動体66に設置された撮影装置7により、法面80および法面80以外を含む対象領域70を移動体66の移動方向に沿って複数の撮影領域dnに分けて撮影したそれぞれの撮影画像pnをつなぎ合わせて、移動体6の移動方向における法面80と法面80以外の境界を含む合成画像2500を表示する入出力画面2000を生成する生成ステップを実行する。
本発明の一実施形態に係る情報処理方法は、移動体66に設置された撮影装置7により、法面80および法面80以外を含む対象領域70を移動体66の移動方向に沿って複数の撮影領域dnに分けて撮影する撮影ステップと、複数の撮影領域dnに分けて撮影したそれぞれの撮影画像pnをつなぎ合わせて、移動体6の移動方向における法面80と法面80以外の境界を含む合成画像2500を表示する入出力画面2000を生成する生成ステップを実行する。
本発明の一実施形態に係るプログラムは、コンピュータに、第14態様または第15態様記載の情報処理方法を実行させる。
本発明の一実施形態に係る状態検査システム1は、移動体66および移動体66に設置された撮影装置7を備えた移動体システム60と、移動体システム60で撮影された画像を処理するデータ管理装置5と、を備え、移動体システム60は、撮影装置7により、法面80および法面80以外を含む対象領域70を移動体66の移動方向に沿って複数の撮影領域dnに分けて撮影し、データ管理装置5は、複数の撮影領域dnに分けて撮影されたそれぞれの撮影画像pnをつなぎ合わせて、移動体6の移動方向における法面80と法面80以外の境界を含む合成画像2500を表示する入出力画面2000を生成する生成部54を備える。
第17態様において、データ管理装置5と通信可能な評価装置3、通信端末1100、または1200をさらに備え、データ管理装置5は、端末装置に対して、入出力画面2000を示す入出力画面2000情報を送信する通信部51をさらに備え、評価装置3、通信端末1100、または1200は、データ管理装置5から送信された、入出力画面2000情報を受信する通信部31、1101、1201と、入出力画面2000をディスプレイ306等に表示する表示制御部33、1103、1203と、を備える。
上記で説明した実施形態の各機能は、一または複数の処理回路によって実現することが可能である。ここで、本実施形態における「処理回路」とは、電子回路により実装されるプロセッサのようにソフトウエアによって各機能を実行するようプログラミングされたプロセッサや、上記で説明した各機能を実行するよう設計されたASIC(Application Specific Integrated Circuit)、DSP(digital signal processor)、FPGA(field programmable gate array)、SOC(System on a chip)、GPU(Graphics Processing Unit)および従来の回路モジュール等のデバイスを含むものとする。
3 評価装置(通信装置の一例)
4 評価システム
5 データ管理装置(情報処理装置の一例)
6 移動体
7 撮影装置
7A 撮影画像データ(輝度画像)
8 センサ装置
8A 測距画像データ(三次元点群)
8a 距離センサ(三次元センサの一例)
8b GNSSセンサ
8c 角度センサ(三次元センサの一例)
9 データ取得装置(通信端末の一例)
92 算出部
93 撮影装置制御部(角度変更部の一例)
96 センサデータ取得部(距離情報取得部、位置情報取得部の一例)
31 通信部(受信手段の一例)
32 受付部(操作受付手段の一例)
33 表示制御部(表示制御手段の一例)
35 評価対象データ生成部(評価対象データ生成手段の一例)
36 検出部(検出手段の一例)
38 レポート生成部(評価情報生成手段の一例)
51 通信部(送信手段の一例)
52 判断部(位置生成手段の一例)
54 生成部(画像生成手段の一例)
55 設定部(設定手段の一例)
59 記憶・読出部(記憶制御手段の一例)
60 移動体システム(撮影システムの一例)
71~73 撮影装置
70 撮影範囲(対象領域)
80 法面
D 撮影装置から斜面までの距離
H 移動体に対する前記撮影装置の高さ
d11、d1n、d1x 撮影領域
701~703 対象領域
Dk 小段の奥行
Hk 小段の高さ
1100 通信端末
1200 通信端末
2000 入出力画面(表示画面の一例)
2010 特定受付画面
2020 決定受付画面
2100 表面画像
2110 表面位置画像(特定位置識別画像の一例)
2150 表面表示画像
2160 他の画像
2170 他の位置画像
2180 他の表示画像
2200 断面画像
2210 断面位置画像(特定点群識別画像の一例)
2250 断面表示画像
2300 ポインタ
2400 特定位置決定ボタン
2402 開始位置指定ボタン
2404 終了位置指定ボタン
2406 縮小ボタン
2408 拡大ボタン
2409 画面切替ボタン
2410 変状確認ボタン
2420 変状予兆確認ボタン
2430 正面図解析ボタン
2440 正面図比較ボタン
2450 断面図解析ボタン
2460 断面図比較ボタン
2500 合成画像
250A、250B 分割画像群
250A1~Am 分割画像
250S 開始位置バー(第1のマーカーの一例)
250G 終了位置バー(第2のマーカーの一例)
2550 統合部分画像
255∪ 上部部分画像
255M 中部部分画像
255L 下部部分画像
3000 検出データ表示画面(種別表示画面の一例)
3100A~3100D テキスト情報(種別情報の一例)
3200A~3200D 画像情報
490 地図画面
491a 移動体6の移動方向における法面80の開始位置(法面80の一端の一例)
491b 移動体6の移動方向における法面80の終了位置(法面80の他端の一例)
492 撮影経路
492a 撮影開始位置
492b 撮影終了位置
Claims (18)
- 移動体に設置された撮影装置により、対象物および対象物以外を含む対象領域を前記移動体の移動方向に沿って複数の撮影領域に分けて撮影したそれぞれの撮影画像をつなぎ合わせて、前記移動体の移動方向における前記対象物と前記対象物以外の境界を含む合成画像を表示する表示画面を生成する生成手段を備えた情報処理装置。
- 前記生成手段は、前記境界の推定位置を示すマーカーを前記合成画像に重畳させて表示する前記表示画面を生成する請求項1記載の情報処理装置。
- 前記生成手段は、前記対象部の一端の前記境界の推定される位置を示す第1のマーカーと、前記対象部の他端の前記境界の推定される位置を示す第2のマーカーを一画面または一ラインで表示する前記表示画面を生成する請求項2記載の情報処理装置。
- 前記生成手段は、前記対象物の推定される種別を示す種別情報を表示する種別表示画面を生成する請求項1記載の情報処理装置。
- 前記種別情報または前記種別情報に対応する画像情報を選択する選択操作に基づき、前記合成画像のうち前記選択された種別情報または画像情報に対応する前記対象物を撮影した撮影画像が、表示部に表示される請求項4記載の情報処理装置。
- 前記合成画像における一部の領域を特定することを決定する決定操作に基づき、前記一部の領域に対応する部分画像を設定する設定手段を備えた請求項1記載の情報処理装置。
- 前記生成手段は、前記合成画像を分割した複数の分割画像のそれぞれの分割画像を並べて表示するように、前記表示画面を生成する請求項1記載の情報処理装置。
- 前記生成手段は、記憶手段に記憶された前記複数の撮影領域に分けて撮影されたそれぞれの撮影画像よりも低い解像度で、前記合成画像を表示するように、前記表示画面を生成する請求項1記載の情報処理装置。
- 前記対象領域は、前記移動体の移動方向と交差する方向において異なる範囲である第1の対象領域および第2の対象領域を含み、
前記第1の対象領域を前記移動体の移動方向に沿って複数の第1の撮影領域に分けて撮影したそれぞれの第1の撮影画像をつなぎ合わせた第1の合成画像、および前記第2の対象領域を前記移動体の移動方向に沿って複数の第2の撮影領域に分けて撮影したそれぞれの第2の撮影画像をつなぎ合わせた第2の合成画像について、
前記生成手段は、
前記第1の合成画像および前記第2の合成画像の少なくとも一方を含む前記表示画面を生成する請求項1記載の情報処理装置。 - 前記第1の合成画像における一部の領域を特定することを決定する第1の決定操作に基づき、前記第1の合成画像における前記一部の領域に対応する第1の部分画像を設定するとともに、前記第2の合成画像における前記一部の領域に対応する第2の部分画像を設定する設定手段を備えた請求項9記載の情報処理装置。
- 前記設定手段は、
前記第1の部分画像と、第2の部分画像をつなぎ合わせた統合部分画像を設定する請求項10記載の情報処理装置。 - 前記設定手段は、前記決定操作に基づき、前記一部の領域に対応する位置情報を設定する請求項6記載の情報処理装置。
- 前記設定手段は、前記決定操作に基づき、前記複数の撮影領域に対応する三次元点群について、前記一部の領域に対応する特定点群を設定する請求項6または12記載の情報処理装置。
- 移動体に設置された撮影装置により、対象物および対象物以外を含む対象領域を前記移動体の移動方向に沿って複数の撮影領域に分けて撮影したそれぞれの撮影画像をつなぎ合わせて、前記移動体の移動方向における前記対象物と前記対象物以外の境界を含む合成画像を表示する表示画面を生成する生成ステップを実行する情報処理方法。
- 移動体に設置された撮影装置により、対象物および対象物以外を含む対象領域を前記移動体の移動方向に沿って複数の撮影領域に分けて撮影する撮影ステップと、
前記複数の撮影領域に分けて撮影したそれぞれの撮影画像をつなぎ合わせて、前記移動体の移動方向における前記対象物と前記対象物以外の境界を含む合成画像を表示する表示画面を生成する生成ステップと、を実行する情報処理方法。 - コンピュータに、請求項14または15記載の情報処理方法を実行させるプログラム。
- 移動体および前記移動体に設置された撮影装置を備えた撮影システムと、前記撮影システムで撮影された画像を処理する情報処理装置と、を備えた情報処理システムであって、
前記撮影システムは、
前記撮影装置により、対象物および対象物以外を含む対象領域を前記移動体の移動方向に沿って複数の撮影領域に分けて撮影し、
前記情報処理装置は、
前記複数の撮影領域に分けて撮影されたそれぞれの撮影画像をつなぎ合わせて、前記移動体の移動方向における前記対象物と前記対象物以外の境界を含む合成画像を表示する表示画面を生成する生成手段を備える情報処理システム。 - 前記情報処理装置と通信可能な端末装置をさらに備え、
前記情報処理装置は、
前記端末装置に対して、前記表示画面を示す表示画面情報を送信する送信手段をさらに備え、
前記端末装置は、
前記情報処理装置から送信された、前記表示画面情報を受信する受信手段と、
前記表示画面を表示部に表示する表示制御手段と、
を備える請求項17記載の情報処理システム。
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| AU2023354806A AU2023354806A1 (en) | 2022-09-26 | 2023-09-05 | Information processing device, information processing method, program, and information processing system |
| US19/075,281 US20250209574A1 (en) | 2022-09-26 | 2025-03-10 | Information processing device, information processing method, computer-readable medium, and information processing system |
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2022152354 | 2022-09-26 | ||
| JP2022-152354 | 2022-09-26 | ||
| JP2023-124018 | 2023-07-31 | ||
| JP2023124018A JP2024047548A (ja) | 2022-09-26 | 2023-07-31 | 情報処理装置、情報処理方法、プログラムおよび情報処理システム |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US19/075,281 Continuation US20250209574A1 (en) | 2022-09-26 | 2025-03-10 | Information processing device, information processing method, computer-readable medium, and information processing system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2024070532A1 true WO2024070532A1 (ja) | 2024-04-04 |
Family
ID=90477348
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/JP2023/032361 Ceased WO2024070532A1 (ja) | 2022-09-26 | 2023-09-05 | 情報処理装置、情報処理方法、プログラムおよび情報処理システム |
Country Status (3)
| Country | Link |
|---|---|
| US (1) | US20250209574A1 (ja) |
| AU (1) | AU2023354806A1 (ja) |
| WO (1) | WO2024070532A1 (ja) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US12601121B2 (en) * | 2023-08-01 | 2026-04-14 | Caterpillar Paving Products Inc. | Control of a work machine using ground surface work records |
| CN121053065B (zh) * | 2025-07-24 | 2026-04-17 | 北京科技大学 | 基于视觉分割与去模糊增强的结构多目标变形监测方法 |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140019301A1 (en) * | 2004-08-31 | 2014-01-16 | Mvp Portfolio, Llc | Internet-accessible drive-by street view system and method |
| US20150081215A1 (en) * | 2000-10-06 | 2015-03-19 | Vederi, Llc | System and method for creating, storing and utilizing images of a geographical location |
| JP2019061667A (ja) * | 2017-09-26 | 2019-04-18 | 株式会社リコー | 診断処理装置、診断システム、入力方法、及びプログラム |
| US20190180150A1 (en) * | 2017-12-13 | 2019-06-13 | Bossa Nova Robotics Ip, Inc. | Color Haar Classifier for Retail Shelf Label Detection |
| JP2020144862A (ja) * | 2019-03-01 | 2020-09-10 | 株式会社スカイマティクス | 石礫検出システム、石礫検出方法及びプログラム |
| JP2021148606A (ja) * | 2020-03-19 | 2021-09-27 | 株式会社リコー | 評価システム、状態検査システム、評価方法およびプログラム |
| JP2022030458A (ja) * | 2020-08-07 | 2022-02-18 | 株式会社リコー | 表示装置、表示システム、表示制御方法およびプログラム |
| JP2022076750A (ja) * | 2020-11-10 | 2022-05-20 | キヤノン株式会社 | 情報処理装置、情報処理システム、および情報処理方法 |
-
2023
- 2023-09-05 WO PCT/JP2023/032361 patent/WO2024070532A1/ja not_active Ceased
- 2023-09-05 AU AU2023354806A patent/AU2023354806A1/en active Pending
-
2025
- 2025-03-10 US US19/075,281 patent/US20250209574A1/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150081215A1 (en) * | 2000-10-06 | 2015-03-19 | Vederi, Llc | System and method for creating, storing and utilizing images of a geographical location |
| US20140019301A1 (en) * | 2004-08-31 | 2014-01-16 | Mvp Portfolio, Llc | Internet-accessible drive-by street view system and method |
| JP2019061667A (ja) * | 2017-09-26 | 2019-04-18 | 株式会社リコー | 診断処理装置、診断システム、入力方法、及びプログラム |
| US20190180150A1 (en) * | 2017-12-13 | 2019-06-13 | Bossa Nova Robotics Ip, Inc. | Color Haar Classifier for Retail Shelf Label Detection |
| JP2020144862A (ja) * | 2019-03-01 | 2020-09-10 | 株式会社スカイマティクス | 石礫検出システム、石礫検出方法及びプログラム |
| JP2021148606A (ja) * | 2020-03-19 | 2021-09-27 | 株式会社リコー | 評価システム、状態検査システム、評価方法およびプログラム |
| JP2022030458A (ja) * | 2020-08-07 | 2022-02-18 | 株式会社リコー | 表示装置、表示システム、表示制御方法およびプログラム |
| JP2022076750A (ja) * | 2020-11-10 | 2022-05-20 | キヤノン株式会社 | 情報処理装置、情報処理システム、および情報処理方法 |
Also Published As
| Publication number | Publication date |
|---|---|
| US20250209574A1 (en) | 2025-06-26 |
| AU2023354806A1 (en) | 2025-03-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7155321B2 (ja) | ひび割れ解析データ編集装置、ひび割れ解析データ編集方法、及び、ひび割れ解析データの編集プログラム | |
| JP6779698B2 (ja) | 舗装ひび割れ解析装置、舗装ひび割れ解析方法及び舗装ひび割れ解析プログラム | |
| JP2024050847A (ja) | 評価装置、状態検査システム、評価方法およびプログラム | |
| US20250209574A1 (en) | Information processing device, information processing method, computer-readable medium, and information processing system | |
| US12284451B2 (en) | Image capturing device, data acquisition unit, image capturing system, and image capturing method | |
| JP2025166083A (ja) | 評価システム、評価方法およびプログラム | |
| JP2026035822A (ja) | 情報処理装置、情報処理システム、情報処理方法およびプログラム | |
| Basnet et al. | Close range photogrammetry for dynamically tracking drifted snow deposition | |
| US12526517B2 (en) | Method for capturing image, method for processing image, image capturing system, and information processing system | |
| US12524907B2 (en) | Information processing apparatus, information processing system, information processing method, and non-transitory computer-executable medium | |
| JP2024047548A (ja) | 情報処理装置、情報処理方法、プログラムおよび情報処理システム | |
| JP7235159B1 (ja) | 情報処理装置、情報処理システム、情報処理方法およびプログラム | |
| JP2024018910A (ja) | 撮影方法、情報処理方法、撮影システムおよび情報処理システム | |
| JP2024106209A (ja) | 撮影方法、プログラム、撮影システムおよび情報処理装置 | |
| JP7739106B2 (ja) | 情報処理装置、情報処理方法及びプログラム | |
| JP2025069514A (ja) | 撮像システム及び撮像方法 | |
| Adjidjonu et al. | Optimal UAS parameters for aerial mapping and modeling | |
| JP2025133629A (ja) | 情報処理システム、情報処理装置、方法およびプログラム | |
| WO2024084873A1 (ja) | 撮影方法及び情報処理方法 | |
| Velmurugan et al. | Monitoring and Environmental Analysis | |
| Renaud et al. | Evaluation of Unmanned Aircraft Systems for Flood Risk Management: Results of Terrain and Structure Assessments | |
| Smebye et al. | Using Gigapan and LiDAR as supporting tools when analyzing Rockfall hazard in | |
| Adam | Digital images for road surface monitoring | |
| CN116071670A (zh) | 一种基于深度学习的铁路坡面排水结构劣化识别方法 | |
| 손승우 | Spatial data collection methodology for estimating large-scale waste quantity based on Unmanned Aerial Systems and Terrestrial Laser Scanning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 23871772 Country of ref document: EP Kind code of ref document: A1 |
|
| WWE | Wipo information: entry into national phase |
Ref document number: AU2023354806 Country of ref document: AU |
|
| ENP | Entry into the national phase |
Ref document number: 2023354806 Country of ref document: AU Date of ref document: 20230905 Kind code of ref document: A |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 23871772 Country of ref document: EP Kind code of ref document: A1 |