WO2021010038A1 - 情報表示装置、方法及びプログラム - Google Patents

情報表示装置、方法及びプログラム Download PDF

Info

Publication number
WO2021010038A1
WO2021010038A1 PCT/JP2020/021598 JP2020021598W WO2021010038A1 WO 2021010038 A1 WO2021010038 A1 WO 2021010038A1 JP 2020021598 W JP2020021598 W JP 2020021598W WO 2021010038 A1 WO2021010038 A1 WO 2021010038A1
Authority
WO
WIPO (PCT)
Prior art keywords
display
unit
dimensional model
information
management
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/021598
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
與那覇　誠
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujifilm Corp
Original Assignee
Fujifilm Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujifilm Corp filed Critical Fujifilm Corp
Priority to JP2021532715A priority Critical patent/JP7413383B2/ja
Publication of WO2021010038A1 publication Critical patent/WO2021010038A1/ja
Priority to US17/572,704 priority patent/US12283035B2/en
Anticipated expiration legal-status Critical
Priority to JP2023189514A priority patent/JP7557600B2/ja
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—Three-dimensional [3D] image rendering
    • G06T15/04—Texture mapping
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/20—Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G01N2021/8854—Grading and classifying of flaws
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
    • G01N21/9515—Objects of complex shape, e.g. examined with use of a surface follower device
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/24—Indexing scheme for image data processing or generation, in general involving graphical user interfaces [GUIs]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2219/00—Indexing scheme for manipulating 3D models or images for computer graphics
    • G06T2219/20—Indexing scheme for editing of 3D models
    • G06T2219/2016—Rotation, translation, scaling

Definitions

  • the present invention relates to an information display device, a method and a program, and particularly to a technique for supporting inspection of a structure.
  • Structures such as bridges need to be inspected regularly in order to maintain and repair the structures.
  • the inspection of the structure is carried out in units of the management division of the structure, and the degree of damage is evaluated and the appearance property is recorded for each management division according to the inspection evaluation standard set by the national government and local governments.
  • Patent Document 1 an image processing method has been proposed that makes it possible to easily list information associated with a position specified on a three-dimensional model of a target object such as a structure.
  • Patent Document 1 generates a three-dimensional model of a target object from a plurality of captured images (two-dimensional images) obtained by photographing the target object, and generates a three-dimensional mesh that approximates the three-dimensional model. Then, a two-dimensional development view (panoramic image) of each partial plane of the generated three-dimensional mesh is generated.
  • a panoramic image selected by the user as a viewing target among the panoramic images for each partial plane of the three-dimensional mesh is displayed in the panoramic image display area of the display unit, and the panoramic image is specified by the user.
  • An inspection image (photographed image) corresponding to the designated position is displayed in the inspection image display area of the display unit, and contents such as a crack image are superimposed and displayed on the inspection image as an annotation.
  • the image processing method described in Patent Document 1 generates a panoramic image of each partial plane of the three-dimensional mesh from a three-dimensional mesh representing the surface of an object object such as a structure, and appropriately selects from the panoramic images of each partial plane.
  • the panoramic image is displayed in the panoramic image display area of the display unit, and the captured image, which is an inspection image corresponding to the position specified on the panoramic image, is displayed in the inspection image display area of the display unit and is displayed three-dimensionally. It does not display the inspection image corresponding to the position specified on the 3D model.
  • the inspection image is a photographed image or an annotation such as a cracked image superimposed on the photographed image, and is not management information for each management category of the structure.
  • the present invention has been made in view of such circumstances, and an information display device, a method, and a program capable of easily confirming management information for each management category of a structure by using a three-dimensional model of the structure.
  • the purpose is to provide.
  • the information display device is stored in a storage unit that stores a three-dimensional model of the structure and management information for each management category of the structure, and a storage unit.
  • the first operation unit that accepts a view operation including at least enlargement of the 3D model to be displayed on the display unit, and the 3D model stored in the storage unit are read out and displayed on the display unit.
  • a first display control unit that displays and at least enlarges the 3D model based on the view operation, and a second operation that receives position information indicating the position on the 3D model that is enlarged and displayed on the display unit according to the user operation.
  • the management category identification unit that specifies the 3D position on the 3D model based on the department and the received position information and specifies the management category including the specified 3D position, and the management information corresponding to the specified management category. Is provided from the storage unit, and a second display control unit is provided which displays the read management information on the display unit.
  • a desired position of the structure can be obtained while looking down at the structure. Can be easily specified.
  • the desired position of the structure is specified, the three-dimensional position on the three-dimensional model corresponding to the position is specified, and the management division of the structure including the specified three-dimensional position is specified.
  • the management category of the structure is specified, the desired management information can be easily confirmed by reading the management information corresponding to the specified management category from the storage unit and displaying it on the display unit.
  • the management information includes an ortho image corresponding to the management classification of the structure, an ortho image on which damage information is superimposed, a damage diagram, a damage quantity table, a repair diagram, and a repair quantity. It is preferably 1 or more in the table.
  • the management classification is the classification of the inspection unit of the parts and members constituting the structure.
  • the first operation unit accepts a view operation for enlarging, reducing, translating, or rotating the three-dimensional model to be displayed on the display unit, and the first display control unit. Is preferably to enlarge, reduce, translate, or rotate the three-dimensional model to be displayed on the display unit based on the view operation received by the first operation unit.
  • the three-dimensional model is represented by a three-dimensional point group consisting of three-dimensional information of a large number of points on the surface of the structure, and is based on the three-dimensional point group. It is a collection of polygonal polygons representing the surface of the structure, or a texture-mapped image of the structure photographed on the polygonal polygons.
  • the first display control unit superimposes an index on the position indicated by the position information received by the second operation unit on the three-dimensional model enlarged and displayed on the display unit. It is preferable to display it.
  • the first display control unit displays the three-dimensional model in the first display area of the display unit
  • the second display control unit displays the second display area of the display unit. It is preferable to display the management information on the screen.
  • the information display device may include a display switching unit that switches between the display of the three-dimensional model by the first display control unit and the display of management information by the second display control unit. preferable.
  • the information display method includes a step of preparing a storage unit for storing a three-dimensional model of the structure and management information for each management division of the structure, and a three-dimensional storage unit stored in the storage unit.
  • the first operation unit accepts a view operation including at least enlargement of the three-dimensional model to be displayed on the display unit, and the first display control unit stores the three-dimensional model in the storage unit.
  • the first display step of reading out and displaying on the display unit the first display step of at least enlarging and displaying the three-dimensional model based on the view operation received by the first operation unit, and the second operation unit are user operations.
  • the step of specifying the three-dimensional position and specifying the management division including the specified three-dimensional position, and the second display control unit reads the management information corresponding to the specified management division from the storage unit and reads the read management information. Includes a second display step to be displayed on the display unit.
  • the management information includes an ortho image corresponding to the management classification of the structure, an ortho image on which damage information is superimposed, a damage diagram, a damage quantity table, a repair diagram, and repair. It is preferably 1 or more in the quantity table.
  • the management classification is a classification of inspection units of parts and members constituting the structure.
  • the step of accepting the view operation accepts the view operation of enlarging, reducing, translating, or rotating the three-dimensional model to be displayed on the display unit, and the first display step. Is preferably to enlarge, reduce, translate, or rotate the three-dimensional model to be displayed on the display unit based on the view operation received by the first operation unit.
  • the three-dimensional model is represented by a three-dimensional point group consisting of three-dimensional information of a large number of points on the surface of the structure, and is based on the three-dimensional point group. It is a collection of polygonal polygons representing the surface of the structure, or a texture-mapped image of the structure photographed on the polygonal polygons.
  • the invention according to still another aspect is an information display program installed in a computer that can access a storage unit that stores a three-dimensional model of the structure and management information for each management division of the structure, and is stored in the storage unit.
  • the function of accepting the view operation including at least enlargement of the 3D model to be displayed on the display unit and the function of reading the 3D model stored in the storage unit and displaying it on the display unit.
  • the function of specifying the 3D position on the 3D model based on the above and specifying the management category including the specified 3D position, and the management information corresponding to the specified management category are read from the storage unit and the read management information. It is an information display program that realizes a function to display on the display unit and a computer.
  • the invention according to still another aspect is a case where a storage unit that stores a three-dimensional model of the structure and management information for each management category of the structure and a three-dimensional model stored in the storage unit are displayed on the display unit.
  • the position information indicating the position on the 3D model enlarged and displayed on the display unit according to the user operation is received, the 3D position on the 3D model is specified based on the received position information, and the specified 3D position is specified.
  • the present invention by displaying a three-dimensional model of a structure on a display unit and performing a view operation for magnifying and displaying at least the three-dimensional model, it is easy to obtain a desired position of the structure while looking down at the structure.
  • the management division of the structure including the 3D position on the 3D model corresponding to the position is specified, and the management division corresponding to the specified management division is specified.
  • FIG. 1 is a block diagram showing an example of the hardware configuration of the information display device according to the present invention.
  • FIG. 2 is a flowchart showing an embodiment of the information display method according to the present invention.
  • FIG. 3 is a diagram showing an example of a screen of a display unit on which a three-dimensional model showing a panoramic view of the bridge is displayed.
  • FIG. 4 is a diagram showing an example of a screen of a display unit displaying a three-dimensional model of a bridge that has been three-dimensionally moved by a view operation such as enlargement.
  • FIG. 5 is a diagram showing an example of an ortho image corresponding to a coffer, which is one of the management divisions of a bridge.
  • FIG. 6 is a diagram showing an example of a damage diagram corresponding to a coffer.
  • FIG. 7 is a diagram showing an example of an ortho image on which a damage diagram corresponding to a coffer is superimposed.
  • FIG. 8 is a chart showing an example of the damage quantity table corresponding to the coffer.
  • FIG. 9 is a diagram showing an example of a repair diagram corresponding to the coffer.
  • FIG. 10 is a chart showing an example of the repair quantity table corresponding to the coffer.
  • FIG. 11 is a diagram showing an example of a screen of the display unit in which the three-dimensional model and the management information are displayed at the same time.
  • FIG. 1 is a block diagram showing an example of the hardware configuration of the information display device according to the present invention.
  • the information display device 10 shown in FIG. 1 a personal computer or a workstation can be used.
  • the information display device 10 of this example mainly includes an image acquisition unit 12, an image database 14, a storage unit 16, an operation unit 18, a CPU (Central Processing Unit) 20, a RAM (Random Access Memory) 22, and a ROM. It is composed of (Read Only Memory) 24 and a display control unit 26.
  • the image acquisition unit 12 corresponds to an input / output interface, and in this example, acquires a photographed image or the like of a structure to be inspected.
  • the structures to be inspected include, for example, structures such as bridges and tunnels.
  • the image acquired by the image acquisition unit 12 is, for example, a large number of images (photographed image group) obtained by photographing a structure with a drone (unmanned aerial vehicle) equipped with a camera. It is preferable that the captured image group covers the entire structure, and the adjacent captured images have overlapping images within a range of about 80%.
  • the captured image group acquired by the image acquisition unit 12 is stored in the image database 14.
  • the storage unit 16 is a storage unit composed of a hard disk device, a flash memory, etc., and includes an operating system, an information display program, a three-dimensional model of a structure, management information for each management division of the structure, and a three-dimensional model. Stores information associated with management information.
  • the three-dimensional model extracts feature points between the captured images that overlap each other in the captured image group stored in the image database 14, and based on the extracted feature points, the position of the camera mounted on the drone and the position of the camera mounted on the drone.
  • Structure from Motion which tracks the movement of many feature points from a group of captured images in which the shooting position of the camera is moved by the drone, and simultaneously estimates the three-dimensional structure (Structure) and camera posture (Motion) of the structure.
  • SfM three-dimensional structure
  • bundle adjustment has been developed, and it has become possible to output with high accuracy.
  • the absolute scale cannot be obtained by the SfM method, for example, the absolute scale (three-dimensional position) can be obtained by instructing a known size (distance between two points, etc.) of the structure. You can ask.
  • the three-dimensional model is represented by a three-dimensional point cloud of a large number of points on the surface of the structure, and is a structure consisting of a collection of polygonal polygons (for example, a triangular patch) based on the three-dimensional point cloud. It is conceivable that the surface of the object is represented, or a photographed image (texture) obtained by photographing a structure on a polygonal polygon is texture-mapped. In the three-dimensional model of the structure of this example, it is assumed that the captured image is texture-mapped to the polygon of the polygon.
  • FIG. 4 shows an example of the screen 30A of the display unit 30 on which (a part of) the three-dimensional model of the bridge 1 is displayed.
  • the bridge 1 displayed on the screen 30A is provided with the main girder 2 passed between the piers 7 and the cross girder 3 provided in the direction orthogonal to the main girder 2 and connecting the main girders, wind, earthquake, etc. It is composed of various members including an anti-tilt structure 4 and a horizontal structure 5 that connect the main girders 2 to each other in order to resist the lateral load of the above.
  • a floor slab 6 for vehicles and the like is placed on the upper part of the main girder and the like.
  • the floor slab 6 is generally made of reinforced concrete.
  • the basic unit of the floor slab 6 is usually a rectangular coffer defined by the main girder 2 and the cross girder 3, and when inspecting damage to the floor slab (cracks, concrete peeling, etc.), the coffer It is done in units of intervals.
  • each coffer of the floor slab is one of the management divisions of the inspection unit of the parts and members that make up the structure (bridge).
  • the parts / members that make up the structure that can be the management division of the bridge are the parts / members that make up the structure (main girder 2, horizontal girder 3, anti-tilt structure 4). , Horizontal structure 5, bridge pier 7 (column / wall, beam, corner / joint), etc.
  • the management information for each management category of the structure stored in the storage unit 16 includes an ortho image corresponding to the management category of the structure, an ortho image on which damage information is superimposed, a damage diagram, a damage quantity table, and a repair diagram. Alternatively, a repair quantity table or the like can be considered. These management information were created when the structure was inspected or repaired, and the details will be described later.
  • the operation unit 18 includes a UI (User Interface) such as a keyboard and a mouse that are connected to the computer by wire or wirelessly.
  • a UI User Interface
  • the keyboard, mouse, and the like function as an operation unit that accepts normal operation input of the computer, and also receives a view operation including at least enlargement of the three-dimensional model displayed on the screen of the display unit 30. It functions as a unit, and also functions as a second operation unit that receives position information indicating a position on a three-dimensional model enlarged and displayed on the display unit 30 according to a user operation.
  • the CPU 20 receives inputs from the first operation unit and the second operation unit, and performs various processes. Specifically, the CPU 20 reads various programs stored in the storage unit 16 or the ROM 24 or the like, controls each unit in an integrated manner, and executes various processes such as a process of generating a three-dimensional model and management information. Further, the CPU 20 functions as a management division specifying unit that specifies the management division of the structure as described later.
  • the RAM 22 is used as a work area of the CPU 20, and is used as a storage unit for temporarily storing the read program and various data.
  • the display control unit 26 is composed of a GPU (Graphics Processing Unit) or the like, creates display data to be displayed on the display unit 30, and outputs the display data to the display unit 30.
  • the display control unit 26 reads out the three-dimensional model stored in the storage unit 16 and causes the display unit 30 to display an image showing the three-dimensional model, and based on the view operation of the three-dimensional model by the operation unit 18, 3 It functions as a first display control unit that at least enlarges the dimensional model. Further, it functions as a second display control unit that reads out the management information corresponding to the management division specified by the management division specifying unit (CPU 20) from the storage unit 16 and displays the read management information on the display unit 30.
  • the display control unit 26 may be realized by the CPU 20 instead of the GPU described above.
  • Various monitors such as a liquid crystal monitor that can be connected to a computer are used for the display unit 30, and various information such as a three-dimensional model and management information are displayed and operated by display data input from the display control unit 26. Used as part of the user interface with section 18.
  • the CPU 20 reads the program stored in the storage unit 16 or the ROM 24 by inputting an instruction from the operation unit 18, and executes the program, so that various types of information display devices 10 can be operated according to the operation by the operation unit 18. Information is displayed on the display unit 30.
  • FIG. 2 is a flowchart showing an embodiment of the information display method according to the present invention.
  • FIG. 2 the information display method according to the present invention will be described while explaining the operation of each part of the information display device 10 shown in FIG.
  • the storage unit 16 of the information display device 10 stores a three-dimensional model of the structure (bridge in this example) and management information for each management category of the structure.
  • the storage unit 16 is prepared in advance.
  • the display control unit 26 of the information display device 10 first reads out the three-dimensional model stored in the storage unit 16 and displays the three-dimensional model on the display unit 30 (step S10, first display step).
  • the display control unit 26 first displays the three-dimensional model on the display unit 30
  • the display control unit 26 shows a panoramic view of the bridge so that the entire bridge 1 which is a structure can be grasped as shown in FIG.
  • the dimensional model is displayed on the screen 30A of the display unit 30.
  • the CPU 20 or the display control unit 26 determines whether or not a view operation including at least enlargement has been performed on the three-dimensional model displayed on the screen 30A by the user (step S12).
  • the view operation is an operation for enlarging, reducing, translating, or rotating a three-dimensional model expressed in 3D (three dimensions) on the screen 30A, and the operation unit 18 functioning as the first operation unit is displayed.
  • a 3D mouse is suitable as the first operation unit that accepts the user's view operation, but a normal mouse or the like can also be used.
  • step S12 When it is determined in step S12 that the view operation has been performed (in the case of "Yes"), the CPU 20 or the display control unit 26 either causes the view operation to enlarge or reduce the three-dimensional model, or translates the three-dimensional model. It is determined whether the operation is to make the operation or the operation to rotate (step S14, step S16).
  • the display control unit 26 functioning as the first display control unit expands the three-dimensional model in response to an instruction for enlargement or reduction by the view operation.
  • display data to be reduced is created and output to the display unit 30 (step S18).
  • the display control unit 26 creates display data for moving the three-dimensional model in parallel in response to an instruction for translation by the view operation.
  • step S20 Output to the display unit 30 (step S20), and when the operation unit 18 performs a view operation for rotating and moving the three-dimensional model, the display unit rotates and moves the three-dimensional model in response to an instruction for rotational movement by the view operation. Data is created and output to the display unit 30 (step S22).
  • step S12 when it is determined that the view operation has not been performed (in the case of "No"), or when the processing of steps S18, S20, and S22 is completed, the transition to step S23 is performed.
  • step S23 the CPU 20 determines whether or not there is an instruction input for the end of the information display from the operation unit 18, and if there is no instruction input for the end (in the case of "No"), the process proceeds to step S24. If there is an end instruction input (in the case of "Yes"), the process related to this information display is terminated.
  • the CPU 20 determines whether or not the position information indicating the position on the three-dimensional model enlarged and displayed on the display unit 30 has been received in response to the user operation on the operation unit 18 functioning as the second operation unit. (Step S24).
  • step S10 If the position information indicating the position on the 3D model is not accepted (in the case of "No"), the process proceeds to step S10, and the display control unit 26 continues to display the 3D model on the display unit 30.
  • the view operation is performed, and the display data for enlarging or reducing the three-dimensional model or the display data for rotating the three-dimensional model or the display data for rotating the three-dimensional model in steps S18, S20 or S22 is obtained.
  • a three-dimensional model that has been enlarged / reduced, parallel-moved, or rotated (three-dimensionally moved) is displayed on the display unit 30 based on the latest display data.
  • FIG. 4 is a diagram showing an example of the screen 30A of the display unit 30 displaying a three-dimensional model of a bridge that has been three-dimensionally moved by a view operation such as enlargement.
  • the three-dimensional model of the bridge 1 to be displayed on the screen 30A of the display unit 30 by the user's view operation is enlarged and moved as shown in FIG. 4 from the three-dimensional model showing the entire view of the bridge 1 shown in FIG. And it is possible to transition to a rotated 3D model.
  • the view operation is performed in order to make it easier to see the points requiring inspection while grasping the space of the entire bridge with the three-dimensional model.
  • 32 is a cursor indicating an input position of the display unit 30 on the screen 30A, and the cursor 32 can be moved on the screen 30A by operating the operation unit 18 (pointing device such as a mouse).
  • operation unit 18 pointing device such as a mouse
  • the user wants to confirm the management information of the desired management division of the bridge, the user moves the three-dimensional model three-dimensionally while grasping the entire bridge in space with the three-dimensional model, and displays the desired management division on the screen 30A of the display unit 30. Search with. Then, on the screen 30A of the display unit 30, the cursor 32 is moved within the desired management category, and a click operation with the mouse or an input operation with the execution key is performed. As a result, the operation unit 18 can specify the position on the three-dimensional model displayed on the screen 30A of the display unit 30, and can receive the position information indicating the designated position.
  • the position of the cursor 32 is located within a specific coffer (management division) included in the floor slab 6 of the bridge 1.
  • the display control unit 26 functioning as the first display control unit sets an index 34 on the three-dimensional model indicating that the selection instruction of the desired management category has been performed. It is preferable to superimpose the display on the position indicated by the cursor 32.
  • step S24 when it is determined in step S24 that the position information indicating the position on the three-dimensional model enlarged and displayed on the display unit 30 has been received (in the case of “Yes”), the process proceeds to step S26.
  • the management division specifying unit specifies the three-dimensional position on the three-dimensional model based on the position information of the position indicated by the cursor 32 on the screen 30A (displayed on the three-dimensional model) of the display unit 30.
  • the position information of the position indicated by the cursor 32 on the screen 30A of the display unit 30 can be acquired as the coordinate information on the screen 30A.
  • the three-dimensional model displayed on the screen 30A of the display unit 30 is enlarged, translated, and rotated by the view operation.
  • the three-dimensional position on the three-dimensional model can be specified by the information on the amount of rotational movement.
  • the management classification identification unit specifies the management classification of the bridge including the 3D position based on the specified 3D position on the 3D model.
  • each management category has three-dimensional information for each management category.
  • the 3D information for each management category is, for example, a 3D point cloud on the management category (part, member), or a feature point of the management category (for example, in the case of a coffer, four feature points at the four corners of the coffer). ) Can be three-dimensional information.
  • the management division identification unit manages the management division having the same or closest 3D information as the 3D position of the bridge including the 3D position. It can be specified as a category.
  • the display control unit 26 functioning as the second display control unit reads the management information corresponding to the specified management division from the storage unit 16, and the read management information. Is displayed on the display unit 30 (step S28, second display step).
  • the management information of the management category specified by the instructed position can be displayed on the display unit 30.
  • Desired management information can be easily confirmed.
  • the CPU 20 functioning as the display switching unit switches the function of the display control unit 26 from the first display control unit to the second display control unit when the display unit 30 displays the management information.
  • the display switching from the three-dimensional model to the management information by the display switching unit can be automatically switched when the management division of the bridge is specified and the management information corresponding to the specified management division can be displayed.
  • an ortho image corresponding to coffers can be considered.
  • FIG. 5 is a diagram showing an example of an ortho image corresponding to the coffer of the bridge.
  • the ortho image is an image obtained by projecting a photographed image of the subject (coffer) onto the surface of the coffer.
  • the ortho image of one case is an image obtained by extracting a plurality of captured images corresponding to the captured image from a group of captured images stored in the image database 14, panoramic combining the extracted plurality of captured images, and panoramic combining the images.
  • it can be created by projecting and transforming the panoramic composite image so that the image is projected normally on the surface of the space.
  • panorama composition of a plurality of captured images can be performed by image processing in which a plurality of feature points in an overlapping region between captured images that overlap each other are extracted and the extracted plurality of feature points are matched with each other.
  • the normal projection of the panoramic composite image on the coffer surface is a projection that matches the positions corresponding to the four corners of the coffer in the panoramic composite image with the three-dimensional positions of the four corners of the coffer. It can be done by conversion.
  • the 3D model of this example is a texture-mapped image of a captured image on a polygon
  • the surface properties of the bridge (coffer) can be confirmed to some extent by enlarging the 3D model. be able to.
  • small damage for example, cracks with a width of 0.1 mm
  • a damage diagram corresponding to coffers can be considered.
  • FIG. 6 is a diagram showing an example of a damage diagram corresponding to a coffer.
  • the damage diagram shown in FIG. 6 five cracks C1 to C5 and concrete peeling H1 are shown.
  • the damage diagram should be generated by manually tracing the damage such as cracks and peeling visually recognized on the ortho image, or performing image processing to automatically detect the damage from the ortho image and manually correcting it if necessary. Can be done.
  • an ortho image in which a damage diagram corresponding to coffers is superimposed can be considered.
  • FIG. 7 is a diagram showing an example of an ortho image on which a damage diagram corresponding to a coffer is superimposed.
  • the ortho image on which the damage diagram shown in FIG. 7 is superimposed can be created by superimposing the damage diagram shown in FIG. 6 on the ortho image shown in FIG.
  • the damage diagram is created with the damaged part colored in a conspicuous color such as red, and the damaged part can be easily visually recognized by superimposing the damage diagram on the ortho image.
  • a damage quantity table corresponding to coffers can be considered.
  • FIG. 8 is a chart showing an example of the damage quantity table corresponding to the coffer.
  • the damage quantity table shown in FIG. 8 has items of damage ID (identification), damage type, size (width), size (length), and size (area), and information corresponding to each item is described for each damage. Has been done.
  • a repair diagram corresponding to coffers can be considered.
  • FIG. 9 is a diagram showing an example of a repair diagram corresponding to the coffer.
  • FIG. 9 two crack repair works A1 and A2 and a cross-section repair work D1 are shown.
  • the crack repair works A1 and A2 show the repair work for the cracks C2 and C5 shown in FIG. 6, and the cross-section repair work D1 shows the repair work for the concrete peeling H1 shown in FIG.
  • the repair result of the coffer can be confirmed by displaying the repair diagram of the coffer as the management information of the coffer.
  • a repair quantity table corresponding to coffers can be considered.
  • FIG. 10 is a chart showing an example of the repair quantity table corresponding to the coffer.
  • the repair quantity table shown in FIG. 10 has items of repair ID, repair method type, size (width), size (length), and size (area), and information corresponding to each item is described for each repair location. ing.
  • the crack repair work includes a crack filling work in which concrete is cut along the crack and the repair material is filled in the part, and a crack injection work in which the repair material is injected at a predetermined injection pressure using a special jig. There is.
  • a crack filling work in which concrete is cut along the crack and the repair material is filled in the part
  • a crack injection work in which the repair material is injected at a predetermined injection pressure using a special jig. There is.
  • the repair method type which method was applied as the crack repair work is described.
  • the cross-section repair work includes a formwork injection method in which a formwork is installed and an injection material is injected, a plastering method in which the cross-section repair material is applied, and a spraying method in which the cross-section repair material is sprayed using compressed air.
  • the type of construction method describes which construction method was applied as the cross-section restoration work.
  • the management information of the first to sixth examples shown in FIGS. 5 to 10 can be displayed on the screen 30A of the display unit 30 in combination of any one or two or more, and any management information can be displayed. The user may appropriately select whether to display the information. Furthermore, the management information is not limited to the above, for example, “cause of damage”, “evaluation of damage”, “event caused by damage” (for example, water leakage, rust juice, free lime, rebar exposure, etc.). Can be used as management information.
  • the CPU 20 subsequently determines whether or not to switch the display on the display unit 30 from the management information to the three-dimensional model (step S30).
  • the determination of switching from the management information to the three-dimensional model can be performed based on the user operation on the operation unit 18. For example, when the user wants to confirm the management information of different management categories, the operation unit 18 can perform an operation of switching from the management information to the three-dimensional model.
  • step S30 when switching from the management information to the display of the three-dimensional model (in the case of "Yes"), the CPU 20 transitions to step S10.
  • the display unit 30 can display a three-dimensional model showing the entire view of the bridge.
  • the CPU 20, which functions as a display switching unit, can switch the function of the display control unit 26 from the second display control unit to the first display control unit when the display unit 30 displays the three-dimensional model.
  • the CPU 20 may transition to step S12 when switching from the management information to the display of the three-dimensional model (in the case of "Yes"). This makes it possible to display the latest 3D model when the display is switched from the 3D model to the management information, which is suitable when it is desired to confirm the management information of the management category close to the previous management category.
  • step S30 determines whether the display is not switched from the management information to the three-dimensional model (in the case of "No). If it is determined in step S30 that the display is not switched from the management information to the three-dimensional model (in the case of "No"), the process proceeds to step S32.
  • step S32 the CPU 20 determines whether or not there is an instruction input for the end of this information display from the operation unit 18, and if there is no instruction input for the end (in the case of "No"), the CPU 20 returns to step S28 and continues to manage. The information is displayed on the display unit 30. On the other hand, if there is an end instruction input (in the case of "Yes"), the process related to this information display is terminated.
  • the display of the 3D model on the display unit 30 and the display of the management information are switched based on the user operation or the like, but the 3D model and the management information are displayed on the display unit 30 at the same time. You may.
  • FIG. 11 is a diagram showing an example of the screen 30A of the display unit 30 in which the three-dimensional model and the management information are displayed at the same time.
  • the three-dimensional model is displayed in the first display area 30L on the left side of the screen 30A of the display unit 30, and the ortho image in which the damage diagram is superimposed as management information is displayed in the second display area 30R on the right side. Has been done.
  • the user can three-dimensionally move the three-dimensional model of the bridge displayed in the first display area 30L by a view operation including enlargement of the three-dimensional model.
  • the ortho image corresponding to the management division of the bridge including the designated position can be displayed in the second display area 30R. Can be done.
  • the user can compare the three-dimensional model displayed in parallel on the screen 30A of the display unit 30 with the ortho image, and can easily confirm the desired management information.
  • the three-dimensional model of the structure and the management information for each management category of the structure stored in the storage unit 16 are not limited to those generated by the information display device 10, but may be generated by an external device. In this case, the three-dimensional model generated by the external device and the management information for each management category of the structure are stored in the storage unit 16.
  • the three-dimensional model is not limited to the one generated by the SfM method by using the photographed image group obtained by photographing the structure, and can be generated by various methods.
  • the hardware that realizes the information display device according to the present invention can be configured by various processors.
  • Various processors include CPUs (Central Processing Units) and FPGAs (Field Programmable Gate Arrays), which are general-purpose processors that execute programs and function as various processing units, and whose circuit configurations can be changed after manufacturing. Includes a dedicated electric circuit, which is a processor having a circuit configuration specially designed for executing a specific process such as a programmable logic device (PLD) and an ASIC (Application Specific Integrated Circuit).
  • PLD programmable logic device
  • ASIC Application Specific Integrated Circuit
  • one processing unit may be composed of a plurality of FPGAs or a combination of a CPU and an FPGA.
  • a plurality of processing units may be configured by one processor.
  • one processor is configured by a combination of one or more CPUs and software, as represented by a computer such as a client or a server.
  • the processor functions as a plurality of processing units.
  • SoC System On Chip
  • the various processing units are configured by using one or more of the above-mentioned various processors as a hardware structure.
  • the hardware structure of these various processors is, more specifically, an electric circuit (circuitry) in which circuit elements such as semiconductor elements are combined.
  • the present invention can be used as an information display device according to the present invention by being installed in a computer that can access a storage unit that stores a three-dimensional model of the structure and management information for each management category of the structure. Includes an information display program to function and a storage medium on which this information display program is recorded.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Quality & Reliability (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • Signal Processing (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Architecture (AREA)
  • Human Computer Interaction (AREA)
  • Processing Or Creating Images (AREA)
  • Bridges Or Land Bridges (AREA)
  • User Interface Of Digital Computer (AREA)
PCT/JP2020/021598 2019-07-18 2020-06-01 情報表示装置、方法及びプログラム Ceased WO2021010038A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2021532715A JP7413383B2 (ja) 2019-07-18 2020-06-01 情報表示装置、方法及びプログラム
US17/572,704 US12283035B2 (en) 2019-07-18 2022-01-11 Information display apparatus, information display method, and information display program
JP2023189514A JP7557600B2 (ja) 2019-07-18 2023-11-06 情報表示装置、方法及びプログラム

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2019132789 2019-07-18
JP2019-132789 2019-07-18

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/572,704 Continuation US12283035B2 (en) 2019-07-18 2022-01-11 Information display apparatus, information display method, and information display program

Publications (1)

Publication Number Publication Date
WO2021010038A1 true WO2021010038A1 (ja) 2021-01-21

Family

ID=74210540

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/021598 Ceased WO2021010038A1 (ja) 2019-07-18 2020-06-01 情報表示装置、方法及びプログラム

Country Status (3)

Country Link
US (1) US12283035B2 (https=)
JP (2) JP7413383B2 (https=)
WO (1) WO2021010038A1 (https=)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115062393A (zh) * 2022-07-19 2022-09-16 北京云庐科技有限公司 一种桥梁施工监测方法、装置、电子设备及介质
JPWO2023062680A1 (https=) * 2021-10-11 2023-04-20

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7752370B1 (ja) * 2025-03-14 2025-10-10 西部マリン・サービス株式会社 水上点検装置及び水上点検方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017168077A (ja) * 2016-03-09 2017-09-21 株式会社リコー 画像処理方法、表示装置および点検システム
JP2017182695A (ja) * 2016-03-31 2017-10-05 富士通株式会社 情報処理プログラム、情報処理方法および情報処理装置
JP2018151964A (ja) * 2017-03-14 2018-09-27 朝日航洋株式会社 桟橋上部工下面の検査装置、桟橋上部工下面の検査システムおよび桟橋上部工下面の検査方法
JP2019049951A (ja) * 2017-09-12 2019-03-28 西日本旅客鉄道株式会社 構造物管理装置、構造物管理方法、及び構造物管理プログラム

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6905956B2 (ja) 2018-06-01 2021-07-21 富士フイルム株式会社 ひび割れ変化判定装置および方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017168077A (ja) * 2016-03-09 2017-09-21 株式会社リコー 画像処理方法、表示装置および点検システム
JP2017182695A (ja) * 2016-03-31 2017-10-05 富士通株式会社 情報処理プログラム、情報処理方法および情報処理装置
JP2018151964A (ja) * 2017-03-14 2018-09-27 朝日航洋株式会社 桟橋上部工下面の検査装置、桟橋上部工下面の検査システムおよび桟橋上部工下面の検査方法
JP2019049951A (ja) * 2017-09-12 2019-03-28 西日本旅客鉄道株式会社 構造物管理装置、構造物管理方法、及び構造物管理プログラム

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPWO2023062680A1 (https=) * 2021-10-11 2023-04-20
CN115062393A (zh) * 2022-07-19 2022-09-16 北京云庐科技有限公司 一种桥梁施工监测方法、装置、电子设备及介质

Also Published As

Publication number Publication date
US12283035B2 (en) 2025-04-22
JP2024012527A (ja) 2024-01-30
US20220130030A1 (en) 2022-04-28
JP7557600B2 (ja) 2024-09-27
JPWO2021010038A1 (https=) 2021-01-21
JP7413383B2 (ja) 2024-01-15

Similar Documents

Publication Publication Date Title
JP7557600B2 (ja) 情報表示装置、方法及びプログラム
JP7624491B2 (ja) 点検支援装置、方法及びプログラム
Alsadik et al. Automated camera network design for 3D modeling of cultural heritage objects
US12069224B2 (en) Image display apparatus, image display method, and image display program
JP7146013B2 (ja) サーバ装置、画像処理方法、及びプログラム
US12106460B2 (en) Information processing apparatus, information processing method, and non- transitory computer-readable storage medium
KR101875047B1 (ko) 사진측량을 이용한 3d 모델링 시스템 및 방법
US12131455B2 (en) Image display apparatus, image display method, and image display program
US20220405878A1 (en) Image processing apparatus, image processing method, and image processing program
WO2022070734A1 (ja) 構造物の点検支援装置、構造物の点検支援方法、及びプログラム
US20260011087A1 (en) Three-dimensional display device, three-dimensional display method, and three-dimensional display program
Szolomicki Application of 3D laser scanning to computer model of historic buildings
JP2017228208A (ja) 図面作成装置及び図面作成方法
WO2021014758A1 (ja) 情報表示装置、情報表示方法、及び情報表示プログラム
Piatkowska et al. Restoring the past digitally: A 3D visualization study of the Great Mill in Gdansk
Yang et al. Holographic Mixed-Reality Interfaces for Enhanced BIM-Driven Facility Management: A Case Study
JP2025180803A (ja) 配管表示システム及び配管表示方法
Silvestre et al. Edition-oriented 3D model rebuilt from photography

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20840248

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021532715

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20840248

Country of ref document: EP

Kind code of ref document: A1