JP2018066687A - Information processing device, information processing method, and information processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily obtain member information of a structure consisting of a plurality of members.SOLUTION: An acquisition unit 30A of an information processing device 12 acquires a first captured image and a second captured image for a structure differing in imaging position. An estimation unit 30B estimates motion information about each of a plurality of pixel areas constituting a background image at a background side of the first captured image on the basis of the motion information that indicates a direction and an amount of movement of a feature area included in the first captured image to a corresponding feature area included in the second captured image. A generation unit 30C moves each of pixel areas constituting the first captured image to each of corresponding pixel areas in the background image in accordance with the direction and the amount of the movement indicated by the estimated motion information, and thereby generates an estimated image in which the second captured image is estimated. An analysis unit 30D derives member information pertaining to members that constitute the structure on the basis of a differential area between the second captured image and the estimated image.SELECTED DRAWING: Figure 7

Description

  The present invention relates to an information processing apparatus, an information processing method, and an information processing program.

  There is known a method for inspecting the arrangement state of reinforcing bars in a structure including beams and columns. The inspection of the arrangement state of the reinforcing bars includes inspection of the number, diameter, position, etc. of the reinforcing bars.

  For example, Patent Document 1 discloses that a circular marker is attached to a measurement target reinforcing bar and a white board is installed on the background of the measurement target reinforcing bar. Patent Document 1 discloses that in this state, binarization processing or the like is performed on a captured image obtained by photographing a measurement target reinforcing bar to measure the diameter of the measurement target reinforcing bar.

  However, in the prior art, it is necessary to use a photographed image taken after installing fixtures such as markers and white boards, and the member information of the structure is derived using the photographed image taken without using these fixtures. It was difficult to do. That is, conventionally, it has been difficult to easily obtain member information of a structure composed of a plurality of members.

  This invention is made | formed in view of the above, Comprising: It aims at obtaining the member information of the structure which consists of multiple members easily.

  In order to solve the above-described problems and achieve the object, the present invention provides an acquisition unit that acquires a first captured image and a second captured image that are different from each other in the capturing position with respect to the structure, and the first captured image. A background image on the background side of the first photographed image is configured based on motion information indicating a moving direction and a movement amount of the feature area included in the second photographed image with respect to the corresponding feature area included in the second photographed image. An estimation unit configured to estimate the motion information of each of a plurality of pixel regions; and the motion estimated for each of the pixel regions constituting the first captured image with respect to each of the corresponding pixel regions in the background image. A generation unit that generates an estimated image obtained by estimating the second captured image by moving according to the moving direction and the moving amount indicated by the information, and the second captured image and the estimated image Based on the partial region, an analysis unit that derives the member information of the member constituting the structural body is an information processing apparatus including a.

  According to the present invention, it is possible to easily obtain member information of a structure composed of a plurality of members.

FIG. 1 is an example of a schematic diagram of an information processing system. FIG. 2 is a schematic diagram illustrating an example of a column member and a beam member constituting the structure. FIG. 3 is an explanatory diagram illustrating an example of a photographing method by the photographing unit. FIG. 4 is an example of a schematic external view of the photographing unit. FIG. 5 is an example of a hardware configuration diagram of the information processing apparatus. FIG. 6 is a block diagram illustrating an example of a functional configuration of the photographing unit. FIG. 7 is a block diagram illustrating an example of a functional configuration of the information processing apparatus. FIG. 8 is an explanatory diagram showing an example of image processing. FIG. 9 is an explanatory diagram illustrating an example of image processing. FIG. 10 is an explanatory diagram illustrating an example of image processing. FIG. 11 is a flowchart illustrating an example of the procedure of the analysis process. FIG. 12 is a schematic diagram illustrating an example of a simple marker. FIG. 13 is a schematic diagram illustrating an example of an information processing system according to a modification. FIG. 14 is a schematic diagram illustrating an example of an information processing system according to a modification. FIG. 15 is a schematic diagram illustrating an example of an information processing system according to a modification. FIG. 16 is a schematic diagram illustrating an example of an information processing system according to a modification.

  Hereinafter, embodiments of the present invention will be described.

  FIG. 1 is a schematic diagram of an information processing system 10 according to the present embodiment.

  The information processing system 10 includes an information processing device 12 and a photographing unit 14. The information processing apparatus 12 and the imaging unit 14 are connected to be communicable with each other via a communication unit 13 such as a network.

  For the communication unit 13, for example, a short-range wireless technology, a wireless communication network using a mobile communication system, the Internet, or the like is used. Examples of the short-range wireless technology include Wi-Fi (registered trademark) (Wireless Fidelity). Examples of wireless communication networks based on mobile communication systems include LTE (Long Term Evolution), NFC (Near Field Communication), WiMAX (World Wide Interoperability for Microwave Access) and the like. The communication unit 13 may be a cable that directly connects the information processing apparatus 12 and the imaging unit 14.

  The information processing system 10 derives member information of members constituting the structure 20.

  First, the structure 20 to be measured will be described. The structure 20 is configured by combining a plurality of rod-like members 21. The member 21 is, for example, a reinforcing bar, a fence, a bar, an anchor rod, or the like. The anchor rod is a structural member such as wood or steel.

  In the present embodiment, as an example, a case where the member 21 is a reinforcing bar will be described. That is, in the present embodiment, the case where the structure 20 is configured by combining a plurality of reinforcing bars will be described. In the present embodiment, a case will be described in which the structure 20 includes a column member 22 in which a plurality of reinforcing bars are combined and a beam member 24 in which a plurality of reinforcing bars are combined.

  FIG. 2 is a schematic diagram showing an example of the column member 22 and the beam member 24 that constitute the structure 20 of the present embodiment.

  FIG. 2A is a schematic diagram illustrating an example of the column member 22. The column member 22 is a long columnar member along the vertical direction X. The column member 22 includes a plurality of main bars 22A and a plurality of auxiliary bars 22B. The main bars 22 </ b> A in the column member 22 are long rod-shaped members along the vertical direction X. The auxiliary bars 22B in the column member 22 are rod-like members that are long in the horizontal direction Y intersecting the vertical direction X that is the major axis direction of the main bars 22A. The auxiliary muscle 22B is a member that supports the plurality of main muscles 22A at intervals along the horizontal direction Y.

  FIG. 2B is a schematic diagram illustrating an example of the beam member 24. The beam member 24 is a long columnar member along the horizontal direction Y. The beam member 24 includes a plurality of main bars 24A and a plurality of auxiliary bars 24B. The main bar 24A is a bar-like member that is long along the horizontal direction Y. The auxiliary muscle 24B is a rod-like member that is long in the vertical direction X. The auxiliary muscle 24 </ b> B is a member that supports the plurality of main muscles 24 </ b> A at intervals along the vertical direction X.

  Note that the vertical direction X and the horizontal direction Y may or may not coincide with the vertical direction and the horizontal direction in real space. Moreover, the vertical direction X and the horizontal direction Y should just be a direction which mutually cross | intersects, and are not limited to the orthogonal direction.

  Here, in the structure 20, the main muscle 22 </ b> A and the main muscle 24 </ b> A contribute to the strength of the structure 20. For this reason, in this embodiment, the information processing apparatus 12 will describe a case where the main muscle 22A and the main muscle 24A in the structure 20 are measured. The auxiliary muscle 22B and the auxiliary muscle 24B may also be measured.

  Returning to FIG. 1, the description will be continued. The imaging unit 14 images the structure 20 and obtains a captured image. In the present embodiment, the imaging unit 14 obtains a plurality of captured images having different imaging positions with respect to the structure 20. In the present embodiment, the extracted image 41 obtains two photographed images (referred to as a first photographed image and a second photographed image). Note that the photographing unit 14 may obtain three or more photographed images.

  FIG. 3 is an explanatory diagram illustrating an example of a photographing method by the photographing unit 14.

  FIG. 3A is an explanatory diagram illustrating an example of an imaging method when measuring the main bar 22A of the column member 22 in the structure 20. For example, when measuring the main muscle 22 </ b> A of the column member 22, the user M holds one end side of the holding member 16 that supports the imaging unit 14. The gripping member 16 is a rod-like member that is long in a predetermined direction. From the viewpoint of reducing the blind spot of the imaging unit 14, the gripping member 16 is preferably as thin as possible and a rod-shaped member. An imaging unit 14 is supported on the other end side of the gripping member 16 in the longitudinal direction.

  The user M grips one end side in the longitudinal direction of the gripping member 16 and moves the gripping member 16 in the horizontal direction Y. Thereby, the user M moves the imaging unit 14 from the outside of the structure 20 toward the inside of the structure 20 through the gap between the main muscle 22A and the auxiliary muscle 24B. During this movement, the photographing unit 14 obtains a plurality of photographed images. For this reason, in this case, the imaging unit 14 obtains a plurality of captured images at different imaging positions when the imaging unit 14 is moved in the horizontal direction Y from the outside to the inside of the structure 20.

  FIG. 3B is an explanatory diagram illustrating an example of an imaging method when measuring the main bar 24A of the beam member 24 in the structure 20. For example, when measuring the main bar 24 </ b> A of the beam member 24, the user M holds one end side in the longitudinal direction of the holding member 16 and moves the holding member 16 in the vertical direction X. Accordingly, the user M moves the imaging unit 14 in the vertical direction X from the outside of the structure 20 toward the inside of the structure 20 through the gap between the main muscle 24A and the auxiliary muscle 24B. During this movement, the photographing unit 14 obtains a plurality of photographed images. For this reason, in this case, the imaging unit 14 obtains a plurality of captured images at different imaging positions when the imaging unit 14 is moved in the vertical direction X from the outside to the inside of the structure 20.

  In addition, the moving method and moving direction of the imaging unit 14 are not limited to the above method and direction. For example, a plurality of photographed images with different photographing positions may be obtained by moving the photographing unit 14 in the circumferential direction around the structure 20 to be measured.

  The photographing unit 14 is a known photographing device that obtains image data of a photographed image by photographing. In the present embodiment, image data of a captured image will be described simply as a captured image. The photographing unit 14 is, for example, a digital camera.

  In addition, it is preferable that the imaging | photography part 14 can image | photograph a wider angle of view by one imaging | photography. For this reason, it is preferable that the imaging | photography part 14 is an omnidirectional camera provided with the convex mirror and the conical mirror, and the omnidirectional camera which can image | photograph an omnidirectional panoramic image. Among these, from the viewpoint of obtaining a wider-angle captured image and the size that can be easily inserted into the structure 20, the imaging unit 14 is an omnidirectional camera that captures an omnidirectional panoramic image. preferable.

  The omnidirectional panoramic image is a panoramic image obtained by photographing the entire celestial sphere (360 °). In the present embodiment, a case where the photographing unit 14 is an omnidirectional camera that obtains an omnidirectional panoramic image will be described as an example. By using an omnidirectional camera for the photographing unit 14, as shown in FIG. 3, a photographed image of a 360 ° photographing region P is obtained.

  FIG. 4 is an example of a schematic external view of the photographing unit 14. FIG. 4A is a side view of the photographing unit 14. 4B is a side view of the imaging unit 14 on the side opposite to FIG. 4A. FIG. 4C is a top view of the imaging unit 14.

  As shown in FIG. 4A, the photographing unit 14 has a size that a human can hold with one hand, for example. Note that the size of the photographing unit 14 is not limited to such a size.

  As shown in FIG. 4, a lens 14 </ b> A is provided on the front side (one surface side) and a lens 14 </ b> B is provided on the back side (the other surface side). The lens 14A and the lens 14B are wide-angle lenses each having an angle of view of 180 ° or more. In the present embodiment, the lens 14A and the lens 14B will be described as fish-eye lenses that are one of wide-angle lenses. In the photographing unit 14, an image is guided through each of the lens 14 </ b> A and the lens 14 </ b> B, and an image is formed on an imaging element corresponding to each of the images. Examples of the image sensor include a charge-coupled device (CCD) and a complementary MOS (CMOS) image sensor. 4B, an operation unit 14C such as a shutter button may be provided on the front side of the photographing unit 14.

  The imaging unit 14 detects an image derived from the lens 14A and the lens 14B with each imaging element, thereby imaging an object around the imaging unit 14 and obtaining two hemispherical images. An omnidirectional panoramic image is created by combining these two hemispherical images.

  Hereinafter, the omnidirectional panoramic image will be described simply as a captured image.

  Next, the hardware configuration of the information processing apparatus 12 will be described. FIG. 5 is an example of a hardware configuration diagram of the information processing apparatus 12.

  The information processing apparatus 12 includes a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory) 503, an HDD (Hard Disk Drive) 505, a media drive 507, a display 508, a network I / F 509. A keyboard 511, a mouse 512, and a CD-ROM drive 514 are connected via a bus 510. The media drive 507 reads or writes (stores) data on a recording medium 506 such as a flash memory. The CD-ROM drive 514 reads or writes various data with respect to a CD-ROM (Compact Disc Read Only Memory) 513 as an example of a removable recording medium.

  The information processing apparatus 12 may have a configuration including a touch panel in which the display 508, the keyboard 511, and the mouse 512 are integrally configured. Further, the information processing apparatus 12 may configure at least one of the display 508, the keyboard 511, and the mouse 512 as a separate body from the information processing apparatus 12.

  Next, the configuration of the photographing unit 14 will be described. FIG. 6 is a block diagram illustrating an example of a functional configuration of the imaging unit 14.

  The photographing unit 14 includes a lens 14A, a lens 14B, an image sensor 14D, an image sensor 14E, a control unit 14, a storage unit 14J, and a communication unit 14K. The image sensor 14D and the image sensor 14E are a CCD or a COMS image sensor. The image guided by the lens 14A is detected as a hemispherical image by the image sensor 14D, and is output to the control unit 14F. The image guided by the lens 14B is detected as a hemispherical image by the image sensor 14E and output to the control unit 14F.

  The control unit 14F controls the photographing unit 14. The control unit 14F includes an image composition unit 14G, a recording processing unit 14H, and a data communication processing unit 14I. Image synthesizing unit 14G synthesizes the hemispherical image received from image sensor 14D and the hemispherical image received from image sensor 14E. By this composition, the image composition unit 14G obtains a captured image as an omnidirectional panoramic image. The recording processing unit 14H stores the captured image obtained by the image combining unit 14G in the storage unit 14J. In addition, the data communication processing unit 14I outputs the captured image obtained by the image composition unit 14G to the information processing apparatus 12 via the communication unit 14K.

  Next, the functional configuration of the information processing apparatus 12 will be described. FIG. 7 is a block diagram illustrating an example of a functional configuration of the information processing apparatus 12.

  The information processing apparatus 12 includes a control unit 30, a communication unit 32, an input unit 33, a display unit 34, and a storage unit 35. The control unit 30, the communication unit 32, the input unit 33, the display unit 34, and the storage unit 35 are connected via a bus 36.

  The communication unit 32 is a communication interface for communicating with an external device. In the present embodiment, the communication unit 32 is connected to the photographing unit 14. For this reason, the control unit 30 obtains a photographed image from the photographing unit 14 via the communication unit 32.

  The input unit 33 receives an operation instruction from the user. The input unit 33 is a keyboard or a mouse. The display unit 34 is a display device that displays various images. The storage unit 35 stores various data.

  The control unit 30 controls the information processing apparatus 12. The control unit 30 is realized by the CPU 501, the ROM 502, the RAM 503, and the like (see FIG. 5). The control unit 30 may be realized by one or a plurality of circuits.

  A program for executing various processes executed by the information processing apparatus 12 according to the present embodiment is provided by being incorporated in advance in the ROM 502 or the like.

  A program for executing various processes executed by the information processing apparatus 12 according to the present embodiment is a file that can be installed in the information processing apparatus 12 or a file that can be executed, such as a CD-ROM, a flexible disk (FD). ), A CD-R, a DVD (Digital Versatile Disk), and the like may be recorded on a computer-readable recording medium and provided.

  Further, the program executed by the information processing apparatus 12 of the present embodiment may be configured to be stored by being stored on a computer connected to a network such as the Internet and downloaded via the network. In addition, a program for executing each process in the information processing apparatus 12 according to the present embodiment may be configured to be provided or distributed via a network such as the Internet.

  The control unit 30 includes an acquisition unit 30A, an estimation unit 30B, a generation unit 30C, an analysis unit 30D, and an output control unit 30E. The estimation unit 30B includes an extraction unit 30F, a calculation unit 30G, a separation unit 30H, and a motion estimation unit 30I.

  Some or all of the acquisition unit 30A, the estimation unit 30B, the generation unit 30C, the analysis unit 30D, the output control unit 30E, the extraction unit 30F, the calculation unit 30G, the separation unit 30H, and the motion estimation unit 30I are, for example, the CPU 501. You may implement | achieve by making a processing apparatus, such as (refer FIG. 5) run a program (namely, software). Further, it may be realized by hardware such as an IC (Integrated Circuit) or may be realized in combination.

  The acquiring unit 30A acquires a first captured image and a second captured image that are different in the capturing position with respect to the structure 20. The first photographed image and the second photographed image are photographed images with different photographing positions with respect to the structure 20.

  As described above, in the present embodiment, the imaging unit 14 obtains a plurality of captured images at different imaging positions when the imaging unit 14 is moved from the outside to the inside of the structure 20. Note that the different shooting positions indicate that the shooting positions with respect to the structure 20 are different as described above. The subject is the structure 20 to the last.

  The acquisition unit 30A receives from the imaging unit 14 via the communication unit 32 one of the plurality of captured images (first captured image) and the first of the plurality of captured images. One photographed image (second photographed image) having a photographing position different from the one photographed image is acquired.

  The shooting timing of the first shot image and the second shot image may be the same or different timing. When the timing is the same, the photographing unit 14 may be configured by a plurality of photographing units having different photographing positions. In the present embodiment, as an example, a case will be described in which the shooting timing of the second shot image is before the shooting timing of the first shot image. However, the shooting timing of the second shot image may be after the shooting timing of the first shot image.

  Note that a plurality of photographed images with different photographing positions with respect to the structure 20 may be stored in the storage unit 35 in advance. The control unit 30 may store the captured image captured by the imaging unit 14 in the storage unit 35. In this case, the acquisition unit 30A may acquire the captured images by reading the first captured image and the second captured image from the storage unit 35.

  8-10 is explanatory drawing which shows an example of the image process by the control part 30. FIG. In the following, a case where the captured image is a captured image having a standard aspect ratio will be described. When the captured image is an omnidirectional panoramic image, the control unit 30 may perform the same processing as described below after expanding the omnidirectional panoramic image into a cube map.

  8A and 8B are schematic diagrams illustrating an example of the captured image 40. FIG. FIG. 8A is a schematic diagram showing the second captured image 40B, and FIG. 8B is a schematic diagram showing the first captured image 40A. 8A and 8B, when measuring the main bar 22A of the column member 22, the image is taken when the imaging unit 14 is inserted inward from the outside of the structure 20 along the horizontal direction Y. The second photographed image 40B and the first photographed image 40A are shown.

  As shown in FIGS. 8B and 8A, the first photographed image 40A and the second photographed image 40B are photographed images obtained by photographing the pillar member 22 in the same structure 20 from different photographing positions. 40. In the example illustrated in FIG. 8, the second captured image 40B is a captured image 40 obtained by capturing the pillar member 22 from a photographing position farther from the pillar member 22 than the first captured image 40A. In the example illustrated in FIG. 8, the first captured image 40A is a captured image 40 obtained by capturing the pillar member 22 from a photographing position closer to the pillar member 22 than the second captured image 40B.

  Returning to FIG. 7, the description will be continued. The estimation unit 30B derives motion information for each feature region included in the first captured image 40A. The motion information is information indicating the motion of the feature region included in the first captured image 40A with respect to the corresponding feature region included in the second captured image 40B.

  Then, the estimation unit 30B, based on the motion information of each feature region included in the first captured image 40A, configures each of the plurality of pixel regions that constitute the background image on the background side in the first captured image 40A. , Generate motion information.

  The characteristic area indicates a characteristic area in the image processing of each of the first captured image 40A and the second captured image 40B. The feature region is a region indicating a specific color, a specific shape, or the like. The specific shape includes, for example, an edge. In the present embodiment, the case where the characteristic region is an edge region indicating an edge will be described as an example.

  The movement information is represented by a movement direction and a movement amount. In the present embodiment, the motion information indicates the moving direction and the moving amount of the characteristic area (edge area) included in the first captured image 40A with respect to the corresponding characteristic area (edge area) in the second captured image 40B. Show. The motion information may be referred to as edge flow. The edge flow is a vector representing the motion of the feature region (edge region) in the captured image 40.

  Details of the estimation unit 30B will be described. The estimation unit 30B includes an extraction unit 30F, a calculation unit 30G, a separation unit 30H, and a motion estimation unit 30I.

  The extraction unit 30F extracts a plurality of edge regions from each of the first captured image 40A and the second captured image 40B.

  8C and 8D are explanatory diagrams of the extracted image 41 from which the edge region E is extracted. The extraction unit 30F performs an edge extraction process of extracting each edge region E of the first captured image 40A and the second captured image 40B. Thereby, the extraction unit 30F derives an extraction image 41 (first extraction image 41A, second extraction image 41B).

  That is, the extraction unit 30F derives a first extracted image 41A obtained by extracting the edge region Ea included in the first captured image 40A from the first captured image 40A (see FIG. 8B) (FIG. 8). 8 (D)). Similarly, the extraction unit 30F derives a second extracted image 41B obtained by extracting the edge region Eb included in the second captured image 40B from the second captured image 40B (see FIG. 8A) (see FIG. 8A). (See FIG. 8C).

  The extraction unit 30F may extract the edge region E from each of the first captured image 40A and the second captured image 40B using a known edge extraction method.

  The calculation unit 30G calculates the motion information for the corresponding edge region Eb included in the second captured image 40B for the plurality of edge regions Ea included in the first captured image 40A.

  FIG. 8E is an explanatory diagram of the calculated motion information V. The calculation unit 30G specifies an edge region Eb indicating the same edge region E in the second photographed image 40B for each of the edge regions Ea included in the first photographed image 40A. A known image processing method may be used for specifying the edge region Eb.

  Then, the calculation unit 30G calculates the motion information V for the corresponding edge region Eb in the second captured image 40B for each of the edge regions Ea included in the first captured image 40A.

  FIG. 8F is a schematic diagram showing the motion information V in an enlarged manner. In the present embodiment, the motion information V is represented by a vector indicating the moving direction and the moving amount of the edge region Ea included in the first captured image 40A with respect to the corresponding edge region Eb in the second captured image 40B. The

  The extraction unit 30F calculates the motion information V from the corresponding edge region Eb in the second extracted image 41B for each of all the edge regions Ea included in the first extracted image 41A. Thereby, the extraction unit 30F obtains a motion information imparted image 42A shown in FIG.

  The motion information imparted image 42A is obtained by imparting motion information V to each of the edge regions Ea included in the first captured image 40A.

  For example, the extraction unit 30F calculates the motion information V for each of the edge regions Ea using a known MRF (Markov Random Field).

  Specifically, the MRF formula is represented by the following formula (1).

  In the formula (1), NCC () indicates a calculation result of normalized cross correlation. In the formula (1), λ is 0.005. In Expression (1), N indicates four pixel regions G that are other pixel regions G around the pixel region G to be processed and include the edge region E.

In the formula (1), S, C _L and ε are represented by the following formulas (2) to (4), respectively.

  Returning to FIG. 7, the description will be continued. The separation unit 30H separates the motion information V calculated for each edge region Ea included in the first captured image 40A into a foreground image on the foreground side and a background image on the background side.

  The foreground side indicates a side closer to the photographing unit 14 than the depth threshold in the photographed image 40. On the other hand, the background side indicates a side farther from the photographing unit 14 than the depth threshold in the photographed image 40. The depth threshold may be determined in advance.

  In the background image, the motion information V of the edge region E located on the foreground side of the depth threshold among the motion information V of the edge region E calculated by the calculation unit 30G is a position corresponding to the edge region E in the captured image 40. It is the image arranged in.

  In the foreground image, the motion information V of the edge region E located on the background side of the depth threshold among the motion information V of the edge region E calculated by the calculation unit 30G is a position corresponding to the edge region E in the captured image 40. It is the image arranged in.

  Specifically, the separation unit 30H converts the motion information V included in the motion information added image 42 illustrated in FIG. 8E into motion information V1 indicating a motion amount equal to or greater than a predetermined motion threshold and less than the motion threshold. It is classified into motion information V2 indicating the amount of motion. Then, the separation unit 30H classifies the motion information addition image 42 into a background image 43A2 and a foreground image 43A1 (see FIGS. 9B and 9A). The motion threshold value may be determined in advance according to the depth threshold value.

  For this reason, as shown in FIG. 9B, the foreground image 43A1 is the edge region Ea to which the motion information V1 in the first captured image 40A is attached to the motion information added image 42 (see FIG. 8E). The image is obtained by extracting the motion information V1 while maintaining the position and the positional relationship.

  Further, as shown in FIG. 9A, the background image 43A2 maintains the position and positional relationship of the edge region Ea to which the motion information V2 is added with respect to the motion information added image 42 (see FIG. 8E). The motion information V2 is extracted as it is.

  Specifically, the separation unit 30H converts the motion information-added image 42 (see FIG. 8E) generated from the first captured image 40A into a plurality of planes along the depth direction in the imaging space of the captured image 40. Considered a layered configuration. Then, the separation unit 30H includes a plurality of planes constituting the motion information imparted image 42 in a plane group whose depth is closer to the front side (front side) (imaging unit 14 side) than the predetermined depth threshold value and to the back side (background) from the threshold value. Side) (in a direction away from the imaging unit 14) and a plane group. Each plane (layer) constituting the plane group is given an edge region E and motion information V located at the corresponding depth.

  Then, the separation unit 30H treats the plane group on the foreground side from the depth threshold as one surface on the foreground side. At this time, the separation unit 30H obtains a projective transformation matrix for making the plane group on the foreground side from the depth threshold as one surface using a known RANSAC (Random Sample Consensus) method. Then, among the pieces of motion information V1 of the edge region Ea included in the one surface, a group of motion information V1 of the edge region Ea having good performance is obtained by the RAMSAC method, and is set as the foreground image 43A1 (FIG. 9B). reference).

  Similarly, the separation unit 30H treats the group of planes on the background side from the depth threshold as one surface on the background side. At this time, the separation unit 30H obtains a projective transformation matrix for making the plane group on the background side from the depth threshold as one surface using a known RANSAC method. Then, among the pieces of motion information V2 of the edge region Ea included in the one surface, a group of the motion information V2 of the edge region Ea having good performance is obtained by the RAMSAC method and is set as the background image 43A2 (FIG. 9A). reference).

  Here, when shooting is performed while moving the shooting unit 14, the subject included in the obtained shot image 40 is located at a position farther from the shooting unit 14 and closer to the shooting unit 14. The amount of movement between the captured images 40 differs with a certain subject. Specifically, the amount of movement of the subject located farther from the photographing unit 14 is smaller than the amount of movement of the subject located closer to the photographing unit 14.

  For this reason, the motion threshold value used when the motion information imparted image 42 is separated into the background image 43A2 and the foreground image 43A1 can separate the subject on the background side from the depth threshold and the subject on the foreground side from the depth threshold. In this way, it may be set in advance according to the movement amount predicted in advance. That is, the motion threshold value may be adjusted as appropriate according to the magnitude of the difference between the shooting positions of the first captured image 40A and the second captured image 40B, the type of the structure 20 to be measured, and the like. Further, the motion threshold value may be changed as appropriate according to an operation instruction of the input unit 33 by the user.

  That is, in the present embodiment, the separation unit 30H includes the motion information V given to each of the edge regions Ea in the first captured image 40A (see FIG. 8B) (see FIG. 8E). The motion information V smaller than the motion threshold is referred to as motion information V2, and the motion information V greater than or equal to the motion threshold is referred to as motion information V1. Then, the separation unit 30H classifies the motion information imparted image 42 into a background image 43A2 that defines motion information V2 that is smaller than the motion threshold and a foreground image 43A1 that defines motion information V1 that is greater than or equal to the motion threshold (FIG. 9). (B), see FIG. 9A).

  Note that the movement amount of the subject at a position farther from the photographing unit 14 is closer to the photographing unit 14 by changing the moving direction and moving method of the photographing unit 14 at the time of photographing the structure 20. May be larger than the amount of movement of the subject. In this case, the separation unit 30H may set the motion information V smaller than the motion threshold as the motion information V1, and the motion information V equal to or higher than the motion threshold as the motion information V2.

  The motion estimation unit 30I estimates the motion information V3 of each of the plurality of pixel regions G constituting the background image 43A2 based on the motion information V2 for each edge region Ea in the background image 43A2.

  Here, when the magnitude of the motion information V of the foreground image 43A1 is compared with the magnitude of the motion information V of the background image 43A2, the motion information V of the background image 43A2 is usually smaller. For this reason, the motion information V of the background image 43A2 can acquire more stable information than the foreground image 43A1 in many cases. The stable information means that there is little noise with respect to the motion information V (position and size). Thus, since the motion information V of the background image 43A2 is more stable than the motion usage V of the foreground image 43A1, in the present embodiment, each of the plurality of pixel regions G constituting the background image 43A2 The motion information V3 is estimated and used to generate an estimated image to be described later (details will be described later).

  FIG. 9C is a schematic diagram illustrating an example of the background motion image 44A2. The background motion image 44A2 is obtained from the motion information V2 defined for each edge region E by the background image 43A2 (see FIG. 9A), and the motion information V (motion information V3) of each pixel region G constituting the background image 43A2. This will be explained as follows.

  The pixel area G is an area occupied by a predetermined number of pixels. In the present embodiment, the pixel area G will be described as an area occupied by one pixel.

  The motion estimation unit 30I supplements the motion information V3 for each pixel region G using the motion information V2 defined for each edge region E. By this complementing process, the motion estimation unit 30I estimates the motion information V3 of each pixel region G. A known complementing method may be used for this complementing process. A well-known complementing technique is a technique shown in the well-known obstruction-free technique, for example.

  Thereby, the motion estimation unit 30I uses all the pixel regions constituting the first captured image 40A using the motion information V2 indicated by the edge region E on the background side from the predetermined depth threshold in the first captured image 40A. G motion information V (motion information V3) is estimated. That is, the motion estimation unit 30I applies the motion information V3 corresponding to the motion information V2 of the background image 43A2 to all the pixel regions G that constitute the first captured image 40A.

  An image in which motion information V3 corresponding to the motion information V2 of the background image 43A2 is defined in all the pixel regions G constituting the first captured image 40A will be referred to as a background motion image 44A2.

  In FIG. 9C, an arrow indicating the motion information V3 is shown for a part of the pixel region G of the background motion image 44A2, but actually, for all the pixel regions G of the background motion image 44A2, Motion information V3 is defined.

  Returning to FIG. 1, the generation unit 30C will be described next. The generation unit 30C sets each of the pixel areas G constituting the first captured image 40A to the movement direction and the movement amount indicated by the motion information V3 estimated for each of the corresponding pixel areas G in the background image 43A2. Move accordingly.

  At this time, in the generation unit 30C, the movement direction indicated by the motion information V3 indicates the movement direction from the edge region Eb included in the second captured image 40B to the corresponding edge region Ea included in the first captured image 40A. When shown, each of the pixel regions G constituting the first captured image 40A is moved in the direction opposite to the moving direction indicated by the motion information V3.

  Specifically, the generation unit 30C identifies the same pair of pixel positions for each of the pixel areas G constituting the first captured image 40A and each of the pixel areas G constituting the background motion image 44A2. Then, the generation unit 30C moves the pixel region G in the first captured image 40A in the direction opposite to the movement direction indicated by the motion information V3 given to the pixel region G at the same pixel position in the background motion image 44A2. The amount of movement indicated by the information V3 is moved.

  On the other hand, in the generation unit 30C, the movement direction indicated by the motion information V3 indicates the movement direction from the edge region E included in the first captured image 40A to the corresponding edge region E included in the second captured image 40B. In this case, each pixel region G constituting the first captured image 40A may be moved in the movement direction indicated by the motion information V3 by the movement amount indicated by the movement information V3.

  In this way, the generation unit 30C adds the first captured image 40A (see FIG. 8B) to each of the pixel regions G that form the background motion image 44A2 (see FIG. 9C). Deformation is performed according to the motion information V3.

  Accordingly, the generation unit 30C generates an estimated image 45 (see FIG. 9D) in which the second captured image 40B is estimated from the first captured image 40A.

  Here, as described above, when shooting is performed while moving the shooting unit 14, the position of the subject in the obtained shot image 40 is determined by the subject farther than the shooting unit 14 and the shooting unit 14. On the other hand, the moving direction and the moving amount between the captured images 40 are different between subjects closer to each other. The background motion image 44A2 defines motion information V3 corresponding to the motion information V2 indicated in the background image 43A2 in each pixel region G of the first captured image 40A.

  Therefore, the estimated image 45 generated by transforming the first captured image 40A using the motion information V3 indicated in the background motion image 44A2 is the second captured image for the background region on the background side from the depth threshold. Although it is close to 40B, the foreground area on the foreground side from the depth threshold is a deformed image area unlike the second captured image 40B.

  Returning to FIG. The analysis unit 30D is a member that constitutes the structure 20 based on a difference area between the second captured image 40B and the estimated image 45 obtained by estimating the second captured image 40B from the first captured image 40A. Deriving information.

  FIG. 10A is the same as the second captured image 40B of FIG. FIG. 10B is the same as the estimated image 45 in FIG.

  The analysis unit 30D calculates a difference area 49 between the second captured image 40B and the estimated image 45. FIG. 10C is an example of the difference extraction image 46 showing the difference area 49. The difference extraction image 46 is an image showing that the difference on the foreground side is larger than the difference on the background side. A member on the foreground side having a large difference, that is, a large movement amount is easily represented as a difference region 49 in a state in which the shape of the member is maintained.

  Therefore, the analysis unit 30D analyzes the difference area 49 to derive member information 52 indicating the member 21 (in this embodiment, the main muscle 22A or the main muscle 24A).

  The member information 52 includes at least one of the number of members 21 (the main reinforcement 22A or the main reinforcement 24A), the position of the member 21, and the diameter of the member 21. The position of the member 21 indicates a relative position with respect to the other member 21. In the present embodiment, the member information 52 includes at least the number of members 21 (the main muscle 22A or the main muscle 24A).

  The analysis unit 30D may analyze the difference area 49 by a known image processing method to obtain the member information 52.

  In the present embodiment, the analysis unit 30D analyzes the difference region 49 to derive member information 52 of the member 21 arranged on the foreground side in the structure 20.

  For example, the analysis unit 30D performs a probabilistic Hough transform on the difference extraction image 46 (see FIG. 10C) showing the difference region 49, thereby specifying the silhouette of the rod-shaped member 21 to thereby determine the member 21 ( The main muscle 22A and the main muscle 24A) are detected. Then, the analysis unit 30D may calculate the member information 52 by counting the number of detected members 21.

  In the present embodiment, analysis unit 30 </ b> D obtains the number of members 21 as member information 52. FIG. 10D is a diagram illustrating an example of the analysis result 47 obtained by the analysis unit 30D. In the example illustrated in FIG. 10D, the analysis unit 30 </ b> D obtains the number “4” of the main bars 22 </ b> A of the column member 22 and the position of each of the main bars 22 </ b> A as the member information 52.

  The analysis unit 30D may perform the probabilistic Hough transform after performing a known silhouette correction process. The silhouette correction process includes a median filter, an eload / dieload process, and the like.

  In the present embodiment, the main bars 24A and the beam members 24 are photographed in a state where they are arranged extending along the vertical direction X or the horizontal direction Y. For this reason, the analysis unit 30D may derive the member information 52 after applying a bias in the vertical direction X or the horizontal direction Y by performing correction by the gravity sensor.

  Further, the analysis unit 30D may derive the member information 52 after performing the black-and-white reversal process or the noise removal process on the difference extraction image 46.

  As shown in FIG. 3B, when the measurement target is the main bar 24 </ b> A of the beam member 24, the control unit 30 takes the first image taken by moving the structure 20 in the vertical direction X. The number of main bars 24A of the beam member 24 is obtained as member information 52 by performing the same processing as described above using the captured image 40A and the second captured image 40B.

  Returning to FIG. 7, the description will be continued. The output control unit 30E performs control to output the member information 52. For example, the output control unit 30E controls the display unit 34 to display an image indicating the member information 52. In this case, an image indicating the member information 52 is displayed on the display unit 34.

  For example, the output control unit 30E transmits the member information 52 to the external device via the communication unit 32. In this case, the external device receives the member information 52 from the information processing device 12 and uses it for various processes. The output control unit 30E may store the member information 52 in the storage unit 35.

  Note that at least one of the functions of the acquisition unit 30A, the estimation unit 30B, the generation unit 30C, the analysis unit 30D, the output control unit 30E, the extraction unit 30F, the calculation unit 30G, the separation unit 30H, and the motion estimation unit 30I is used as a server device or the like. It is good also as a structure provided in the external device.

  The analysis unit 30D may compare the derived member information 52 with the design information of the structure 20 and determine whether the derived member information 52 matches the design information. In this case, the output control unit 30E may further control output of information indicating the determination result.

  Next, an example of an analysis process procedure executed by the control unit 30 of the information processing apparatus 12 will be described.

  FIG. 11 is a flowchart illustrating an example of the analysis processing procedure executed by the control unit 30 of the information processing apparatus 12.

  First, the acquisition unit 30A acquires the first captured image 40A and the second captured image 40B (step S100). Next, the extraction unit 30F extracts the edge region E from each of the first captured image 40A and the second captured image 40B (step S102).

  Next, the calculation unit 30G calculates the motion information V for the corresponding edge region Eb included in the second captured image 40B for the plurality of edge regions Ea included in the first captured image 40A (step S104). .

  Next, the separation unit 30H separates the motion information V calculated for each edge region Ea included in the first captured image 40A into the background image 43A2 and the foreground image 43A1 (step S106).

  Next, the motion estimation unit 30I estimates the motion information V3 of each of the plurality of pixel regions G constituting the background image 43A2 based on the motion information V2 for each edge region Ea in the background image 43A2 (step S108). .

  Next, the generating unit 30C moves each of the pixel regions G constituting the first captured image 40A according to the motion information V3 estimated for each of the corresponding pixel regions G in the background image 43A2 ( Step S110). In step S110, the generation unit 30C moves the first captured image 40A (see FIG. 8B) to each of the pixel regions G constituting the background motion image 44A2 (see FIG. 9C). Deformation is performed according to the information V3. Through the processing in step S110, the generation unit 30C generates an estimated image 45 (see FIG. 9D) obtained by estimating the second captured image 40B from the first captured image 40A.

  Next, the analysis unit 30D specifies a difference area 49 between the second captured image 40B and the estimated image 45 (step S112).

  Next, the analysis unit 30D derives member information 52 indicating the member 21 (in this embodiment, the main muscle 22A or the main muscle 24A) by analyzing the difference region 49 (step S114).

  The output control unit 30E stores the member information 52 derived in step S114 in the storage unit 35 (step S116). Next, the output control unit 30E outputs the member information 52 derived in step S114 to the display unit 34 (step S118). Then, this routine ends.

  As described above, the information processing apparatus 12 according to the present embodiment includes the acquisition unit 30A, the estimation unit 30B, the generation unit 30C, and the analysis unit 30D. The acquisition unit 30A acquires the first captured image 40A and the second captured image 40B that are different in the capturing position with respect to the structure 20. The estimation unit 30B moves the feature region (edge region E) included in the first photographed image 40A with respect to the corresponding feature region (edge region E) included in the second photographed image 40B. Based on the information V, the motion information V3 of each of the plurality of pixel regions G constituting the background-side background image 43A2 in the first captured image 40A is estimated. The generation unit 30C sets each of the pixel areas G constituting the first captured image 40A to the movement direction and the movement amount indicated by the motion information V3 estimated for each of the corresponding pixel areas G in the background image 43A2. The estimated image 45 which estimated the 2nd picked-up image 40B is produced | generated by moving according to it. The analysis unit 30 </ b> D derives member information 52 of the members 21 constituting the structure 20 based on the difference area 49 between the second captured image 40 </ b> B and the estimated image 45.

  Here, as described above, when shooting is performed while moving the shooting unit 14, the position of the subject in the obtained shot image 40 is determined by the subject farther than the shooting unit 14 and the shooting unit 14. On the other hand, the moving direction and the moving amount between the captured images 40 are different between subjects closer to each other. The background motion image 44A2 is obtained by adding motion information V3 corresponding to the motion information V2 shown in the background image 43A2 to each of the pixel regions G of the first captured image 40A.

  Therefore, the estimated image 45 generated by transforming the first captured image 40A using the motion information V3 indicated in the background motion image 44A2 is close to the second captured image 40B in the background region on the background side. However, unlike the second captured image 40B, the foreground area on the foreground side is deformed.

  In the present embodiment, the information processing apparatus 12 derives the member information 52 of the member 21 constituting the structure 20 based on the difference area 49 between the second captured image 40B and the estimated image 45. . For this reason, in the information processing apparatus 12 of this Embodiment, the member information 52 of the member 21 can be derived | led-out easily, without using fixtures, such as a white board.

  Therefore, the information processing apparatus 12 according to the present embodiment can easily obtain the member information 52 of the structure 20 including the plurality of members 21.

  The estimation unit 30B includes an extraction unit 30F, a calculation unit 30G, a separation unit 30H, and a motion estimation unit 30I. The extraction unit 30F extracts a plurality of feature regions E from each of the first captured image 40A and the second captured image 40B. The calculation unit 30G calculates the motion information V (motion information V1, motion information V2) for the corresponding feature area E included in the second captured image 40B for the plurality of feature areas E included in the first captured image 40A. To do. The separation unit 30H separates the motion information V calculated for each edge region E included in the first captured image 40A into the foreground image 43A1 on the foreground side and the background image 43A2 on the background side.

  The motion estimation unit 30I estimates the motion information V3 of each of the plurality of pixel regions G constituting the background image 43A2 based on the motion information V2 for each feature region (edge region E) in the background image 43A2.

  In addition, the analysis unit 30D analyzes the difference area 49 and obtains member information 52 of members arranged on the foreground side in the structure 20.

  Further, the analysis unit 30D obtains member information 52 including at least one of the number of members 21, the position of the member 21, and the diameter of the member 21.

  Further, the first photographed image 40A and the second photographed image 40B are photographed at different photographing positions, which are photographed when the photographing unit 14 is moved toward the outside of the structure 20 or the inside of the structure 20. Image 40.

  The first captured image 40A and the second captured image 40B are omnidirectional panoramic images.

  The feature area is an edge area E indicating an edge.

  In addition, the information processing method of the present embodiment obtains the first captured image 40A and the second captured image 40B, which have different imaging positions with respect to the structure 20 to be measured, and the first captured image 40A. Based on the motion information V indicating the moving direction and the moving amount of the included feature region (edge region E) with respect to the corresponding feature region included in the second captured image 40B, the background on the background side in the first captured image 40A The step of estimating the motion information V of each of the plurality of pixel regions G constituting the image 43A2 and the pixel region G constituting the first photographed image 40A are assigned to each of the corresponding pixel regions G in the background image 43A2. On the other hand, an estimated image 45 obtained by estimating the second captured image 40B is moved by moving according to the moving direction and the moving amount indicated by the motion information V estimated for Comprising a step of forming, on the basis of the differential area 49 between the estimated image 45 and the second captured image 40B, deriving the member information 52 of member 21 constituting the structure 20.

  In addition, the information processing program according to the present embodiment acquires the first captured image 40A and the second captured image 40B, which have different shooting positions with respect to the structure 20 to be measured, and the first captured image 40A. Based on the motion information V indicating the moving direction and the moving amount of the included feature region (edge region E) with respect to the corresponding feature region included in the second captured image 40B, the background on the background side in the first captured image 40A The step of estimating the motion information V of each of the plurality of pixel regions G constituting the image 43A2 and the pixel region G constituting the first photographed image 40A are assigned to each of the corresponding pixel regions G in the background image 43A2. An estimated image obtained by estimating the second captured image 40B by moving according to the moving direction and the moving amount indicated by the motion information V estimated for the second moving image V 5 and a step of deriving the member information 52 of the member 21 constituting the structure 20 based on the difference area 49 between the second captured image 40B and the estimated image 45. This is an information processing program.

  In the above embodiment, the description has been given on the assumption that the photographed image 40 is photographed in a photographing environment with sufficient ambient light. However, in some cases, the above processing is performed using a captured image 40 that is captured in an environment where the ambient light necessary for image processing cannot be obtained. In this case, the information processing apparatus 12 may perform a process similar to the above after performing a known light amount correction process such as setting the parameter at the time of extraction of the edge region E as a parameter specialized for the dark environment. .

<Modification 1>
In the above embodiment, the case where the first captured image 40A and the second captured image 40B are the captured images 40 obtained by capturing the structure 20 has been described. However, you may use the picked-up image 40 which image | photographed the structure 20 provided with the simple marker used as a mark as the 1st picked-up image 40A and the 2nd picked-up image 40B.

  FIG. 12 is a schematic diagram illustrating an example of the simple marker 60. The simple marker 60 may be any shape, color, and size that can be easily identified by image processing when the simple marker 60 is captured as the captured image 40. For example, the simple marker 60 may be any of the simple marker 60A, the simple marker 60B, and the simple marker 60C.

  The information processing apparatus 12 features the simple marker 60 by using the captured image 40 obtained by capturing the structure 20 with the simple marker 60 attached to the member 21 as the first captured image 40A and the second captured image 40B. It can be used as one of the areas. In addition, what is necessary is just to use well-known methods, such as the circle detection by Hough transformation, for the detection of the simple marker 60 in the picked-up image 40.

  Further, the information processing apparatus 12 uses the ratio between the size of the simple marker 60 included in the captured image 40 and the size of the actual simple marker 60 to determine the diameter and size of the member 21 included in the captured image 40. The actual size may be calculated. Further, the information processing apparatus 12 may calculate the interval between the members 21 using the ratio.

  Further, the simple marker 60 may be given only to the member 21 existing on the background side when the captured image 40 is captured. Then, the information processing apparatus 12 may determine that the region to which the simple marker 60 included in the captured image 40 is added is the member 21 existing on the background side and use it for deriving the member information 52.

  For this reason, in this modification, the information processing apparatus 12 can derive the member information 52 with higher accuracy than the above embodiment.

<Modification 2>
In the above-described embodiment, the information processing system 10 has been described as an example in which the information processing apparatus 12 and the photographing unit 14 are configured. However, the configuration of the information processing system 10 is not limited to the above configuration.

  13 to 16 are schematic diagrams illustrating an example of the information processing system 10A to the information processing system 10D according to the present modification.

  As illustrated in FIG. 13, for example, the information processing system 10 </ b> A may have a configuration in which the cloud server 71 and the imaging unit 14 are connected via the communication unit 13. The cloud server 71 is obtained by mounting the information processing apparatus 12 on the cloud server 71 installed on the cloud 70.

  As illustrated in FIG. 14, the information processing system 10 </ b> B may have a configuration in which the information processing apparatus 12 and the mobile terminal 72 are connected via the communication unit 13. The portable terminal 72 is a portable terminal device. The portable terminal 72 is, for example, a smart phone, a tablet, a game machine, or the like. In this case, what is necessary is just to set it as the structure which mounted the imaging | photography part 14 in the portable terminal 72. FIG.

  Further, as illustrated in FIG. 15, the information processing system 10 </ b> C may include a mobile terminal 74. The portable terminal 74 may be configured to include the information processing device 12 and the photographing unit 14.

  Further, as illustrated in FIG. 16, the information processing system 10 </ b> D may have a configuration in which a portable terminal 76 and a photographing unit 75 are connected via a communication unit 13. The portable terminal 76 only needs to have a configuration in which the information processing apparatus 12 is mounted. The photographing unit 75 includes the photographing unit 14 and is a device that can photograph a panoramic image.

  Note that a program for executing the above-described processing executed by the information processing apparatus 12 is provided by being incorporated in advance in NV-RAM, ROM, or other nonvolatile storage medium. A program for executing the above processing executed by the information processing apparatus 12 is a file in an installable or executable format, and is a CD-ROM, flexible disk (FD), CD-R, DVD (Digital It is also possible to record and provide in a computer-readable recording medium such as Versatile Disk).

  In addition, the program for executing the above-described processing executed by the information processing apparatus 12 is stored on a computer connected to a network such as the Internet, and is provided or distributed by being downloaded via the network. May be.

  Moreover, you may comprise so that the program for performing the said process performed with the information processing apparatus 12 may be previously incorporated in ROM etc. and provided.

  Further, the program for executing the above-described processing executed by the information processing apparatus 12 has a module configuration including the above-described units. As actual hardware, a CPU (processor) downloads the program from a storage medium. By reading and executing, each unit is loaded on the main memory, and each unit is generated on the main memory.

  Note that various types of information stored in the storage unit 35 of the information processing apparatus 12 may be stored in an external device. In this case, the external device and the information processing device 12 may be configured to be connected via a network or the like.

  In addition, although this Embodiment and the modification were demonstrated above, the said embodiment and modification are shown as an example, and are not intending limiting the range of invention. The above-described novel embodiments and modifications can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. The above-described embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

12 information processing device 20 structure 21 member 30A acquisition unit 30B estimation unit 30C generation unit 30D analysis unit 30E output control unit 30F extraction unit 30G calculation unit 30H separation unit 30I motion estimation unit

Japanese Patent No. 5412092

Claims (9)

An acquisition unit for acquiring a first photographed image and a second photographed image having different photographing positions relative to the structure;
A background side in the first photographed image based on motion information indicating a movement direction and a movement amount of the feature area included in the first photographed image with respect to the corresponding feature area included in the second photographed image. An estimation unit for estimating the motion information of each of a plurality of pixel regions constituting the background image of
By moving each of the pixel areas constituting the first captured image according to the movement direction and the movement amount indicated by the movement information estimated for each of the corresponding pixel areas in the background image, A generating unit that generates an estimated image obtained by estimating the second captured image;
An analysis unit for deriving member information of members constituting the structure based on a difference area between the second captured image and the estimated image;
An information processing apparatus comprising: The estimation unit includes
An extraction unit that extracts a plurality of the feature regions from each of the first photographed image and the second photographed image;
A calculation unit that calculates the motion information for the corresponding feature region included in the second captured image with respect to the plurality of feature regions included in the first captured image;
The motion information calculated for each of the feature regions included in the first photographed image includes a foreground image (mapping the motion information at a corresponding position) and a background image (giving the motion information A separation unit that separates the background image;
A motion estimation unit configured to estimate the motion information of each of a plurality of pixel regions constituting the background image based on the motion information for each of the feature regions in the background image;
The information processing apparatus according to claim 1, comprising: The analysis unit
The information processing apparatus according to claim 1, wherein the difference area is analyzed to derive the member information of the member arranged on a foreground side in the structure. The analysis unit derives the member information including at least one of the number of the members, the position of the members, and the diameter of the members.
The information processing apparatus according to any one of claims 1 to 3.   The first photographed image and the second photographed image are photographed images having different photographing positions, which are photographed when the photographing unit is moved from the outside of the structure toward the inside of the structure. The information processing apparatus according to any one of claims 1 to 4.   The information processing apparatus according to claim 1, wherein the first photographed image and the second photographed image are omnidirectional panoramic images.   The information processing apparatus according to claim 1, wherein the feature area is an edge area indicating an edge. Obtaining a first photographed image and a second photographed image having different photographing positions with respect to the structure;
A background side in the first photographed image based on motion information indicating a movement direction and a movement amount of the feature area included in the first photographed image with respect to the corresponding feature area included in the second photographed image. Estimating the motion information of each of a plurality of pixel regions constituting the background image of
By moving each of the pixel areas constituting the first captured image according to the movement direction and the movement amount indicated by the movement information estimated for each of the corresponding pixel areas in the background image, Generating an estimated image obtained by estimating the second captured image;
Deriving member information of members constituting the structure based on a difference area between the second captured image and the estimated image;
An information processing method including: Obtaining a first photographed image and a second photographed image having different photographing positions with respect to the structure;
A background side in the first photographed image based on motion information indicating a movement direction and a movement amount of the feature area included in the first photographed image with respect to the corresponding feature area included in the second photographed image. Estimating the motion information of each of a plurality of pixel regions constituting the background image of
By moving each of the pixel areas constituting the first captured image according to the movement direction and the movement amount indicated by the movement information estimated for each of the corresponding pixel areas in the background image, Generating an estimated image obtained by estimating the second captured image;
Deriving member information of members constituting the structure based on a difference area between the second captured image and the estimated image;
Processing program for causing a computer to execute.