WO2018180551A1

WO2018180551A1 - Image processing device, image processing method and program

Info

Publication number: WO2018180551A1
Application number: PCT/JP2018/010278
Authority: WO
Inventors: 與那覇　誠
Original assignee: 富士フイルム株式会社
Priority date: 2017-03-30
Filing date: 2018-03-15
Publication date: 2018-10-04
Also published as: JPWO2018180551A1; JP6779368B2

Abstract

The present invention has a purpose of providing an image processing device, an image processing method and a program which can perform displaying, through which the presence of an imaging omission and the location of the imaging omission can be checked without a calculation cost in a short time. The image processing device (400) is provided with: an image input unit (401); a corresponding point information acquisition unit (403); a reference divided image specifying unit (405); a matrix calculation unit (407) which calculates a projection transformation matrix for a reference divided image; a first projection transformation unit (411) which performs projection transformation on contour location information, which indicates a contour of a divided image to be transformed, by means of the calculated projection transformation matrix; a synthesis information generation unit (413) which generates, on the basis of the projection-transformed contour location information, contour location information on a synthesized image in which the reference divided image and the divided image to be transformed after the projection transformation are synthesized; and a display unit (415).

Description

Image processing apparatus, image processing method, and program

The present invention relates to an image processing apparatus, an image processing method, and a program, and more particularly to a technique for suppressing the occurrence of missing shots when performing divided shooting.

Conventionally, there has been known a technique of performing divided shooting on one shooting target object to acquire a divided image, and combining the acquired divided images to obtain a combined image in which an entire image of the shooting target is reflected.

For example, when detecting and inspecting damage to structures such as bridges, inspection is often performed in units of gaps, and in that case, a plurality of divided images are obtained by dividing and photographing floor slabs in the area of the gap. Then, a composite image in which the floor slab of the entire space is reflected is generated by the obtained divided image.

Also, for example, Patent Document 1 describes a technique intended to prevent spillage (shooting omission) when shooting an image for three-dimensional measurement. Specifically, in Patent Document 1, each feature point for which the three-dimensional coordinates are obtained is written in the coordinate space of the photographing range image that represents the photographing range of the measurement object in the three-dimensional space, and the coordinate space of the photographing range image is written. Among the feature points written in, three or more feature points surrounding the shooting area are connected and divided for each shooting area, and the shooting range image is formed as a wire frame image, and the shooting range image has an insufficient shooting area. Techniques for determining whether or not are described.

JP 2010-256253 A

However, in the technique described in Patent Document 1, since a shooting range image is generated as a wire frame image based on feature points obtained in three-dimensional coordinates, it takes time to generate a shooting range image. That is, each feature point for which three-dimensional coordinates are obtained is written in a three-dimensional coordinate space, and a wire frame image is generated by connecting three or more feature points surrounding the imaging region among the feature points. A frame image is an image having a large amount of information, and it takes time to synthesize a wire frame image.

For example, when performing a detection inspection for damage to a structure such as a bridge, a composite image is generated using a divided image obtained by dividing and shooting a floor slab in a space as described above. Since this is an image having a large amount of information, it takes time to generate a composite image by combining the divided images.

Here, if you can confirm the spillage in a short time, you can easily check the spillover even at the inspection (shooting) site. When you return to the office from the inspection site and generate a composite image It is possible to suppress the occurrence of a situation where it is necessary to discover a spillage and go to the inspection site again. In other words, if it is possible to confirm the spillage in a short time, the inspection work can proceed efficiently.

The present invention has been made in view of such circumstances, and an object of the present invention is to provide a display capable of confirming the presence / absence of a spill and the position of the spill over in a short time without incurring calculation costs. An image processing apparatus, an image processing method, and a program are provided.

In order to achieve the above object, an image processing apparatus according to an aspect of the present invention includes an image input unit to which a plurality of divided images obtained by subjecting an imaging target to division imaging are input, and a plurality of divided images. A corresponding point information acquisition unit that searches for corresponding points and acquires corresponding point information about the corresponding points, a reference divided image specifying unit that specifies a reference divided image that is a reference for projective transformation among a plurality of divided images, and corresponding points Based on the information and the reference divided image, a matrix calculation unit that calculates a projective transformation matrix for the reference divided image of the converted divided image other than the reference divided image among the plurality of divided images, and a converted projection matrix by the calculated projective transformation matrix When the first projective conversion unit that performs projective transformation on the contour position information indicating the contour of the divided image and the reference divided image and the converted divided image after the projective transformation are synthesized based on the projective transformed contour position information Comprising the synthesis information generation unit for generating a contour position information of the composite image, on the basis of the contour position information of the composite image, and a display unit for displaying the contour of the composite image.

According to this aspect, the contour position information indicating the contour of the transformed divided image is projectively converted by the first projective transformation unit using the calculated projective transformation matrix, and the contour position subjected to the projective transformation by the synthesis information generating unit. Based on the information, the outline position information of the synthesized image when the reference divided image and the converted divided image after projective transformation are synthesized is generated, and the outline of the synthesized image is displayed based on the outline position information of the synthesized image. Is done. That is, this aspect displays the contour of the composite image based on the contour position information of the divided image that has undergone the projective transformation. Thereby, this aspect can provide a user with a display that can confirm the presence or absence of a spill and the position of the spill over in a short time without incurring calculation costs.

Preferably, the image processing apparatus includes a second projective conversion unit that performs projective conversion on a part of the converted divided images based on the projective conversion matrix, and the display unit is the one converted by the second projective conversion unit. At least one of the converted divided image and the reference divided image is displayed on the outline of the composite image.

According to this aspect, the second projective transform unit performs projective transformation on a part of the converted divided image based on the projective transformation matrix, and the display unit converts the partial projective image converted by the second projective transform unit. At least one of the converted divided image and the reference divided image is displayed on the contour of the synthesized image. Thus, according to the present aspect, since some of the divided images are also displayed on the display unit in accordance with the contour of the composite image, it is possible to provide a display that can more accurately confirm the presence / absence of the shooting and the position of the shooting failure. it can.

Preferably, the second projective transformation unit performs projective transformation on the transformed divided image corresponding to the corner of the composite image.

According to this aspect, the second projective conversion unit performs projective conversion on the converted divided image corresponding to the corner of the synthesized image, and the divided image is also displayed on the display unit according to the outline of the synthesized image. Can more accurately confirm the presence / absence of a spill and the position of the spill.

Preferably, the second projective transformation unit performs projective transformation on the transformed divided image corresponding to the four corners of the composite image.

According to this aspect, the second projective conversion unit performs projective conversion on the converted divided image corresponding to the four corners of the composite image, and the split image is also displayed on the display unit in accordance with the contour of the composite image. It is possible to provide a display that can more accurately confirm the presence / absence of a shooting failure and the position of the shooting failure.

Preferably, the second projective transformation unit performs projective transformation on the transformed divided image corresponding to the side of the composite image.

According to this aspect, the second projective conversion unit performs projective conversion on the converted divided image corresponding to the side of the combined image, and the divided image is also displayed on the display unit according to the outline of the combined image. It is possible to provide a display capable of accurately confirming the presence / absence of a spill and the position of the spill.

Preferably, the second projective transformation unit performs projective transformation on the transformed divided image corresponding to the four sides of the composite image.

According to this aspect, the second projective conversion unit performs projective conversion on the converted divided image corresponding to the four sides of the composite image, and the split image is also displayed on the display unit in accordance with the contour of the composite image. It is possible to provide a display that can more accurately confirm the presence / absence of a shooting failure and the position of the shooting failure.

Preferably, the image processing apparatus includes a first designation receiving unit that receives a designation of a position in the contour of the composite image displayed on the display unit, and the second projective conversion unit is received by the first designation receiving unit. Projective transformation is performed on the transformed divided image corresponding to the selected position.

According to this aspect, the first designation accepting unit accepts designation of the position in the outline of the composite image displayed on the display unit, and the second projective transformation unit accepts the first designation accepting unit. The transformed divided image corresponding to the selected position is projectively transformed. In this aspect, the divided image corresponding to the position received by the first designation receiving unit is displayed on the display unit in accordance with the contour of the synthesized image. Thereby, this aspect can provide the display which can confirm the presence or absence of a shooting spill and the position of a shooting spill more accurately.

Preferably, the image processing apparatus performs projective conversion on a part of the reduced images based on the reduced image generation unit that generates a reduced image with a reduced resolution of the divided image input to the image input unit and the projective transformation matrix. 3 projection conversion units, and the display unit displays a part of the reduced image converted by the third projection conversion unit on the outline of the composite image.

According to this aspect, the reduced image generation unit generates a reduced image in which the resolution of the divided image input to the image input unit is reduced, and the third projective conversion unit generates a reduced image based on the projective transformation matrix. Projective transformation is performed. As a result, since the amount of information of the reduced image is smaller than that of the divided image, the projection conversion of the reduced image can be performed in a shorter time than the projection conversion of the divided image. A display capable of confirming the position can be provided to the user.

Preferably, the image processing apparatus includes a reduced image generation unit that generates a reduced image with a reduced resolution of the divided image input to the image input unit, and a third projection that performs projective conversion on the reduced image based on the projection conversion matrix. A conversion unit, and a second designation receiving unit that receives designation of the reduced image displayed on the display unit, and the display unit corresponds to the reduced image designated by the second designation receiving unit as the reduced image. Then, the image is switched to the converted divided image or the reference divided image that has undergone projective conversion by the second projective conversion unit.

According to this aspect, the reduced image generation unit generates a reduced image in which the resolution of the divided image input to the image input unit is reduced, and the third projective conversion unit generates a reduced image based on the projective transformation matrix. Projection conversion is performed, and the second designation accepting unit accepts designation of a reduced image displayed on the display unit, and a divided image corresponding to the reduced image is displayed. As a result, this mode can display the specified location in the divided image when the presence or absence of the shot or the position of the shot is not easily understood in the reduced image display. A display that can confirm the position of the spill can be provided.

Preferably, the reference divided image specifying unit specifies the reference divided image based on designation of the reference divided image or information on the input order of the divided images to be the reference divided image.

According to this aspect, since the reference divided image is specified based on the designation of the reference divided image or the information related to the input order of the divided images to be the reference divided image, the reference divided image suitable as the reference divided image is specified. Appropriate composite image contour position information can be acquired.

An image processing apparatus according to another aspect of the present invention reduces the resolution of an image input unit to which a plurality of divided images obtained by dividing and shooting an imaging target are input, and the divided image input to the image input unit. A reduced image generation unit that generates reduced images, a corresponding point information acquisition unit that searches for corresponding points between the reduced images and acquires corresponding point information about the corresponding points, and is a reference for projective transformation among a plurality of reduced images Based on the reference reduced image specifying unit for specifying the reference reduced image, the corresponding point information, and the reference reduced image, a projection transformation matrix for the reference reduced image of the converted reduced image other than the reference reduced image among the plurality of reduced images is calculated. Based on the matrix calculation unit, the third projective transformation unit that performs projective transformation on the contour position information indicating the contour of the transformed reduced image by the calculated projective transformation matrix, and the reference reduced image based on the projective transformed contour position information Oh A composite information generating unit that generates contour position information of the composite image when the converted reduced image after projective conversion is combined, and a display unit that displays the contour of the composite image based on the contour position information of the composite image; .

According to this aspect, the contour position information indicating the contour of the converted reduced image is projectively converted by the first projective transformation unit using the calculated projective transformation matrix, and the contour position subjected to the projective transformation by the composite information generating unit. Based on the information, the outline position information of the synthesized image when the reference reduced image and the converted reduced image after projective transformation are synthesized is generated, and the outline of the synthesized image is displayed based on the outline position information of the synthesized image. Is done. That is, this aspect displays the contour of the composite image based on the contour position information of the reduced image subjected to the projective transformation. Thereby, this aspect can provide a user with a display that can confirm the presence or absence of a spill and the position of the spill over in a short time without incurring calculation costs.

According to another aspect of the present invention, there is provided an image processing method including a step of inputting a plurality of divided images obtained by subjecting a subject to a divided photographing, and searching for corresponding points between the plurality of divided images, A step of acquiring point information, a step of identifying a reference divided image serving as a reference for projective transformation among the plurality of divided images, and a reference divided image of the plurality of divided images based on the corresponding point information and the reference divided image A step of calculating a projective transformation matrix with respect to a reference divided image of a non-converted divided image, a step of performing projective transformation of contour position information indicating an outline of the converted divided image by the calculated projective transformation matrix, and a projective transformation Based on the contour position information, a step of generating the contour position information of the synthesized image when the reference divided image and the transformed divided image after the projective transformation are synthesized; Based on Guo position information, and displaying the contour of the composite image.

An image processing method according to another aspect of the present invention includes a step of inputting a plurality of divided images obtained by dividing and photographing an object to be imaged, a step of generating a reduced image with a reduced resolution of the divided image, and a reduction Searching for corresponding points between images to obtain corresponding point information about the corresponding points; identifying a reference reduced image as a reference for projective transformation among a plurality of reduced images; corresponding point information and reference reduced image Based on the above, a step of calculating a projection transformation matrix for a reference reduced image of a converted reduced image other than the reference reduced image among a plurality of reduced images, and a contour indicating an outline of the converted reduced image by the calculated projective transformation matrix A step of projective conversion of position information, and a composite image obtained by combining the reference reduced image and the converted reduced image after the projective conversion based on the contour position information subjected to the projective conversion. Comprising the steps of generating a contour position information, based on the contour position information of the composite image, and displaying the contour of the composite image.

The program according to another aspect of the present invention includes a step in which a plurality of divided images obtained by subjecting a subject to division photographing are input, and corresponding point information on the corresponding points by searching for corresponding points between the plurality of divided images. Obtaining a reference divided image serving as a reference for projective transformation among the plurality of divided images, and other than the reference divided image among the plurality of divided images based on the corresponding point information and the reference divided image A step of calculating a projective transformation matrix of the converted divided image with respect to a reference divided image, a step of performing projective transformation of contour position information indicating a contour of the converted divided image by the calculated projective transformation matrix, and a contour position subjected to the projective transformation Generating the contour position information of the composite image when combining the reference divided image and the converted divided image after the projective transformation based on the information; Based on the position information, to execute a step of displaying the contour of the composite image, the process including the computer.

The program according to another aspect of the present invention includes a step of inputting a plurality of divided images obtained by subjecting a subject to division photographing, a step of generating a reduced image with a reduced resolution of the divided images, and a reduction image between A step of searching corresponding points and acquiring corresponding point information on the corresponding points, a step of identifying a reference reduced image as a reference for projective transformation among a plurality of reduced images, and the corresponding point information and the reference reduced image Calculating a projection transformation matrix for the reference reduced image of the converted reduced image other than the reference reduced image among the plurality of reduced images, and contour position information indicating an outline of the reduced reduced image by the calculated projection transformation matrix And a combined image obtained by combining the reference reduced image and the converted reduced image after the projective transformation based on the projective transformed contour position information. Generating a contour position information, based on the contour position information of the composite image, and executes a step of displaying the contour of the composite image, the process including the computer.

According to the present invention, the contour position information indicating the contour of the converted divided image is projectively converted by the first projective transformation unit using the calculated projective transformation matrix, and the contour position subjected to the projective transformation by the synthesis information generating unit. Based on the information, the outline position information of the synthesized image when the reference divided image and the converted divided image after projective transformation are synthesized is generated, and the outline of the synthesized image is displayed based on the outline position information of the synthesized image. As a result, it is possible to provide the user with a display that allows the user to confirm the presence / absence of shooting and the position of shooting failure without incurring calculation costs in a short time.

It is a perspective view which shows the structure of a bridge. It is a conceptual diagram which shows an imaging | photography system. It is the figure which showed notionally the photography position of a mobile. It is a block diagram which shows the system configuration example of a computer. It is a figure which shows the synthetic | combination image in which the spilling has occurred. It is a figure which shows the synthetic | combination image in which the spilling has occurred. It is a figure which shows the function structural example of an image processing apparatus. It is a figure explaining the projective transformation of outline position information. It is a figure which shows the outline of a synthesized image. It is a figure which shows the outline of a synthesized image. It is a flowchart which shows operation | movement of an image processing apparatus. It is a figure which shows a synthesized image. It is a figure which shows the outline of a synthesized image. It is a figure which shows the function structural example of an image processing apparatus. It is a figure which shows the outline of a synthesized image. It is a figure which shows the function structural example of an image processing apparatus. It is a figure which shows the function structural example of an image processing apparatus. It is a figure which shows the function structural example of an image processing apparatus.

Hereinafter, preferred embodiments of an image processing apparatus, an image processing method, and a program according to the present invention will be described with reference to the accompanying drawings.

FIG. 1 is a perspective view showing the structure of a bridge, which is one of the structures to be photographed, and is a perspective view of the bridge as viewed from below. The object to be imaged in the present invention is not particularly limited. For example, the present invention is applied to the case of synthesizing a photographed image that is used when a detection inspection for damage to a structure is performed. Specifically, when the damage detection inspection of the floor slab 6 (FIG. 1) is performed in units of spaces, the floor slab 6 in the area of the space 502 (FIG. 5) is divided and the captured images obtained by the division are displayed. The present invention is applied when synthesizing.

The bridge 1 shown in FIG. 1 has a main girder 2, a cross girder 3, a counter-tilt 4 and a horizontal 5 which are connected by bolts, rivets or welding. In addition, a floor slab 6 for driving a vehicle or the like is provided on the upper portion of the main beam 2 or the like. The floor slab 6 is generally made of reinforced concrete.

The main girder 2 is a member that is passed between the abutment or the pier and supports the load of the vehicle on the floor slab 6. The cross beam 3 is a member that connects the main beams 2 in order to support the load by the plurality of main beams 2. The anti-tilt frame 4 and the horizontal frame 5 are members for connecting the main girders 2 to each other in order to resist lateral loads of wind and earthquake, respectively.

FIG. 2 is a conceptual diagram showing a photographing system that obtains divided images by separately photographing the floor slab 6.

The imaging system 500 includes a moving body 100 and a computer 300. That is, in the photographing system 500, the divided images acquired by the moving body 100 are transmitted to the computer 300, and the computer 300 is provided with the image processing device 400 (FIG. 7).

The moving body 100 includes a camera 101 and moves and takes an image with the camera 101 under the control of the computer 300. The moving body 100 is not particularly limited as long as it can move and shoot under the control of the computer 300. For example, the moving body 100 includes a device called a traveling robot, a small helicopter, a multicopter, a drone, or UAV (Unmanned Aerial Vehicles).

The computer 300 and the mobile unit 100 can communicate with each other, and the computer 300 can remotely control the movement of the mobile unit 100 and the imaging control by the camera 101. The captured image captured by the moving body 100 is displayed on the touch panel display 302 that also serves as the display unit 326 (FIG. 4) and the input unit 328 (FIG. 4) of the computer 300.

FIG. 3 is a diagram conceptually showing the shooting position of the moving body 100. FIG. 3 is a conceptual diagram of the floor slab 6 viewed from the X-axis direction in FIG. The camera 101 is provided in the moving body 100 as shown in FIG. 2 and is moved by the movement of the moving body 100. In addition, the mobile body 100 is abbreviate | omitted in FIG. Further, the vehicle 10 travels on the floor slab 6, and a photographed image from below the floor slab 6 is acquired and a detection inspection for damage to the floor slab 6 is performed.

The moving body 100 is moved to each shooting position under the control of the computer 300 and performs shooting to acquire a shot image. When the inspection is performed in units of spaces of the floor slab 6, a divided image of the floor slab 6 is acquired by divided photographing.

FIG. 4 is a block diagram illustrating a system configuration example of the computer 300.

As shown in FIG. 4, a computer 300 includes a CPU (Central Processing Unit) 310 that controls the overall operation of the computer 300. The CPU 310 is connected to a main memory 314, a nonvolatile memory 316, a mobile device via a system bus 312. A communication unit 318, a wireless LAN communication unit 320, a short-range wireless communication unit 322, a wired communication unit 324, a display unit 326, an input unit 328, a key input unit 330, an audio processing unit 332, an image processing unit 334, and the like are connected. Is done.

The CPU 310 reads out an operation program (OS (Operating System) and application program that operates on the OS), standard data, and the like stored in the non-volatile memory 316, expands them in the main memory 314, and performs this operation. By executing the program, it functions as a control unit that controls the operation of the entire computer.

The main memory 314 is composed of, for example, a RAM (Random Access Memory) and functions as a work memory for the CPU 310.

The non-volatile memory 316 is composed of, for example, a flash EEPROM (EEPROM: Electrically Erasable Programmable Read Only Memory), and stores the above-described operation program and various fixed data. The nonvolatile memory 316 functions as a storage unit of the computer 300 and stores various data.

The mobile communication unit 318 is a third-generation mobile communication system compliant with the IMT-2000 standard (International Mobile Telecommunication-2000) and a fourth-generation mobile communication system compliant with the IMT-Advanced standard (International Mobile Telecommunications-Advanced). Based on this, data is transmitted / received to / from the nearest base station (not shown) via the antenna 318A.

The wireless LAN communication unit 320 uses a predetermined wireless LAN communication standard (for example, IEEE802.11a / b / g / n standard) between the wireless LAN access point and an external device capable of wireless LAN communication via the antenna 320A. ) To perform wireless LAN communication.

The short-range wireless communication unit 322 transmits / receives data to / from other Bluetooth (registered trademark) equipment within the range of, for example, class 2 (within a radius of about 10 m) via the antenna 322A.

The wired communication unit 324 performs communication according to a predetermined communication standard with an external device connected by a cable via the external connection terminal 306. For example, USB (Universal Serial Bus) communication is performed.

The display unit 326 includes a color LCD (liquid crystal display) panel that constitutes a display portion of the touch panel display 302 and a driving circuit thereof, and displays various images.

The input unit 328 constitutes a touch panel portion of the touch panel display 302. The input unit 328 is configured integrally with the color LCD panel using a transparent electrode.

The key input unit 330 includes a plurality of operation buttons provided on the casing of the computer 300 and a drive circuit thereof.

The audio processing unit 332 converts the digital audio data provided via the system bus 312 into an analog signal and outputs it from the speaker 304.

The image processing unit 334 digitizes an analog image signal output from the built-in camera 305 including a photographing lens and an image sensor, performs necessary signal processing, and outputs it.

Next, an example of shooting spills will be described. In particular, the example described below is an example of spilling that may occur when the damage detection inspection of the floor slab 6 is performed in units of spaces.

FIG. 5 is a diagram showing a composite image 501 in which type 1 shooting has been missed, and FIG. 6 is a diagram showing a composite image 501 in which type 2 shooting has been missed. The composite image 501 shown in FIGS. 5 and 6 is obtained by projecting and combining a plurality of divided images obtained by dividing and photographing the floor slab 6. That is, the divided images obtained by photographing the floor slab 6 are projected and combined to form a composite image 501.

In the composite image 501 shown in FIG. That is, the floor slab 6 in the area 502 is supposed to be captured without omission in the composite image 501, but there is no image corresponding to the floor slab 6 at the location of the spilled 503. When combining divided images, a margin is provided for each image to be combined, and the images are combined in the margin area. Therefore, it is necessary to take a divided image including an estimate of the margin area. However, if the estimate is incorrect, for example, a shooting spill 503 occurs.

In the composite image 501 shown in FIG. That is, the floor slab 6 in the area 502 is supposed to be captured without omission in the composite image 501, but there is no image corresponding to the floor slab 6 at the location of the shot spill 507. The shooting spill 507 is a so-called hollow and the composite image 501 is an image having a hollow.

Here, the composite image 501 is shown in FIGS. 5 and 6, and the user can confirm the occurrence of the overshoot 503 and the overshoot 507 by checking the composite image 501. However, since the composite image 501 is composed of divided images, each of the divided images must be projected and changed, and generation of the composite image 501 requires time and calculation cost. For example, in the case of performing a detection inspection for damage to the bridge 1, considering that a composite image 501 composed of divided images is generated for a total of 800 spaces, the composite image 501 is used for confirmation of overshooting. If it is generated at the shooting site, the efficiency of the inspection work is greatly reduced.

Therefore, the present invention proposes a display capable of confirming the presence / absence of a spill and the position of the spill over in a short time without incurring calculation costs.

<First Embodiment>
The first embodiment of the present invention will be described. FIG. 7 is a diagram illustrating a functional configuration example of the image processing apparatus 400 according to the present embodiment.

The image processing apparatus 400 includes an image input unit 401, a corresponding point information acquisition unit 403, a reference divided image specification unit 405, a matrix calculation unit 407, a projective conversion unit 409, a composite information generation unit 413, and a display unit 415.

The image input unit 401 inputs a plurality of obtained divided images by subjecting the subject to division photography. For example, the divided image of the floor slab 6 acquired by the moving body 100 is input to the image input unit 401 every time a divided image is acquired or every time a single space 502 is captured. The image input unit 401 is realized by the mobile communication unit 318, the wireless LAN communication unit 320, or the short-range wireless communication unit 322 of the computer 300, for example.

Corresponding point information acquisition unit 403 searches for corresponding points between a plurality of divided images and acquires corresponding point information related to the corresponding points. Specifically, the corresponding point information acquisition unit 403 extracts feature points in each divided image, and acquires information on the corresponding points between the divided images at the extracted feature points. The information regarding the corresponding point is, for example, the coordinates of the corresponding point. Note that the corresponding point information acquisition unit 403 can acquire information on corresponding points between corresponding divided images by a known method. For example, as a method for acquiring information about corresponding points, SIFT (Scale-invariant feature transform) feature, SURF (Speed-Upped Robust Feature) ) Features and AKAZE (Accelerated KAZE) features are known.

The reference divided image specifying unit 405 specifies a reference divided image serving as a reference for projective transformation among the plurality of divided images. Since the reference divided image is necessary for the subsequent projective transformation and generation of the contour position information of the composite image, at least one reference divided image needs to be specified. The specified reference divided image becomes a reference for projective transformation and affects the contour position information of the composite image. That is, depending on the specified reference divided image, the contour position information of the composite image facing the subject may be generated, or the contour position information of the composite image having tilt may be generated. In order to check for missed shots, the shape of the composite image and whether or not it is facing is not a decisive problem, and therefore, the reference divided image may be specified randomly (randomly).

In addition, the reference divided image specifying unit 405 selects a reference divided image based on information regarding designation of a reference divided image serving as a reference for projective transformation among a plurality of divided images or input order of divided images serving as a reference divided image. You may specify. For example, when the user selects a divided image displayed on the touch panel display 302 of the computer 300, the reference divided image specifying unit 405 receives the user's specification and receives the reference divided image specification. In addition, the reference divided image specifying unit 405 specifies the reference divided image by receiving information on the input order of the divided images to be the reference divided images stored in the main memory 314 of the computer 300. For example, the information related to the input order of the divided images to be the reference divided images is the divided image input first in the shooting for each case as the reference divided image, or is input in the middle order of the input divided images. The divided image is used as a reference divided image. The reference divided image specifying unit 405 is realized by the CPU 310 of the computer 300, for example.

The matrix calculation unit 407 calculates a projective transformation matrix for the reference divided image of the converted divided image other than the reference divided image among the plurality of divided images based on the corresponding point information and the reference divided image. That is, the matrix calculation unit 407 performs projective transformation so as to match the reference divided image based on the corresponding point information of the converted divided image adjacent to the reference divided image specified by the reference divided image specifying unit 405, and sequentially converts the converted divided image. A projection transformation matrix of the transformed divided image is calculated based on the corresponding point information of the image.

The projective conversion unit 409 includes a first projective conversion unit 411 that performs projective conversion of the contour position information. The first projective transformation unit 411 performs projective transformation on the contour position information indicating the contour of the transformed divided image using the calculated projective transformation matrix. That is, the projective transformation is performed on the contour position information indicating the contour of each transformed divided image by the projection transformation matrix for each transformed divided image calculated by the matrix calculation unit 407. The projection conversion unit 409 is realized by the CPU 310 of the computer 300, for example.

FIG. 8 is a diagram for explaining the projective conversion of the contour position information of the converted divided image performed by the first projective conversion unit 411.

FIG. 8A is a diagram showing the contour position information of the transformed divided image D1 before projective transformation. Specifically, the coordinates E1, F1, G1, and H1 of the four corners of the converted divided image D1 are shown as the contour position information.

FIG. 8B is a diagram showing the contour position information of the converted divided image D2 after projective transformation. The coordinates E1, F1, G1, and H1 indicating the contour position information of the divided image D1 to be converted are projectively transformed by the projective transformation matrix to be transformed into coordinates E2, F2, G2, and H2. Coordinates E2, F2, G2, and H2 indicate the contour of the transformed divided image D2 after projective transformation.

Here, the first projective transformation unit 411 performs projective transformation on the coordinates E1, F1, G1, and H1 indicating the contour position information of the transformed divided image D1, and all the pixels constituting the transformed divided image D1. Is not projective transformed. That is, the first projective transformation unit 411 performs projective transformation only on the contour of the transformed divided image D1, and calculates the contour position information of the transformed divided image D2 after the projective transformation. Thus, by causing the first projective transformation unit 411 to perform projective transformation of only the outline of the divided image, the time required for the projective transformation can be shortened, and the calculation cost for the projective transformation can be suppressed. In the example shown in FIG. 8, the coordinates of the four corners of the converted divided image are shown as the contour position information, but the present invention is not limited to this. Other forms of information may be used as long as the information indicates the contour of the converted divided image.

Returning to FIG. 7, the composite information generation unit 413 combines the reference divided image and the converted divided image after the projective conversion based on the contour position information subjected to the projective conversion, and outputs the contour position information of the composite image 501. Is generated. That is, the composite information generation unit 413 combines the contour position information of the converted divided image converted by the first projective conversion unit 411, thereby synthesizing the contour position of the composite image 501 obtained by combining the reference divided image and the converted divided image. Calculate information. The composite information generation unit 413 is realized by the CPU 310 of the computer 300, for example.

The display unit 415 displays the contour 505 of the composite image 501 based on the contour position information of the composite image 501. That is, the display unit 415 displays the contour 505 of the composite image 501 on the touch panel display 302 of the computer 300 based on the contour position information of the composite image 501 generated by the composite information generation unit 413. By displaying the outline 505 of the composite image 501 on the touch panel display 302, the user can confirm the presence / absence of shooting and the position of shooting failure. The display unit 415 is realized by the display unit 326 of the computer 300, for example.

9 and 10 are diagrams showing the contour 505 of the composite image 501 displayed on the touch panel display 302 by the display unit 415 based on the contour information of the composite image 501 generated by the composite information generation unit 413. 9 is a display of the contour 505 of the composite image 501 with respect to the composite image 501 having the spillover 503 described in FIG. 5, and FIG. 10 is a display of the contour 505 of the composite image 501 having the spillover 507 described in FIG. It is.

For comparison, FIG. 9A shows a composite image 501 obtained by synthesizing a divided image (a reference divided image and a converted divided image after projective transformation).

On the other hand, FIG. 9B shows a contour 505 of the composite image 501 based on the contour information of the composite image 501 generated by the composite information generation unit 413. Specifically, as described with reference to FIG. 8, the first projective transformation unit 411 performs projective transformation on the contour position information of each converted divided image, and the composite information generation unit 413 generates the contour position information of the composite image 501. The outline 505 of the composite image 501 is shown on the display unit 415.

For example, the user confirms the outline 505 of the composite image 501 after acquiring a plurality of divided images for the floor slab 6 of one space 502. Then, since the user has a substantially rectangular outline 505 of the composite image 501 for the floor slab 6 of the space 502, the user can confirm the display of the contour 505 corresponding to the overshoot 503, thereby It can be seen that spillage has occurred in the floor slab 6.

For comparison, FIG. 10A shows a composite image 501 obtained by normally combining a split image (a reference split image and a converted split image after projective transformation).

On the other hand, FIG. 10B shows the contour 505 of the composite image 501 based on the contour information of the composite image 501 generated by the composite information generation unit 413 as in FIG. 9B.

The user confirms the outline 505 of the composite image 501 after acquiring a plurality of divided images for the floor slab 6 of one space 502. Then, since the contour 505 of the composite image 501 for the floor slab 6 of the space 502 does not become a hollow like the contour 505 corresponding to the shooting spill 507, the user does not spill the floor slab 6 of the space 502. You can see that it has occurred.

FIG. 11 is a flowchart showing the operation of the image processing apparatus 400.

First, a divided image is taken by the moving body 100 for the floor slab 6 of the space 502. When the shooting of the divided images is completed for the floor slab 6 of the space 502, the divided images are transmitted to the computer 300. Then, a plurality of divided images obtained by dividing and shooting the shooting target are input to the image input unit 401 (step S10).

Next, the corresponding point information acquisition unit 403 searches for corresponding points between the plurality of input divided images, and acquires corresponding point information regarding the searched corresponding points (step S11). Then, the reference divided image specifying unit 405 specifies the reference divided image (step S12). For example, the designation of the reference divided image is input by the user via the input unit 328, and information regarding the input order of the divided images to be the reference divided image stored in the main memory 314 of the computer 300 is received, for example.

Next, the matrix calculation unit 407 calculates a projective transformation matrix for the reference divided image of the converted divided image other than the reference divided image among the plurality of divided images based on the corresponding point information and the reference divided image (step S13). ). Then, the first projective transformation unit 411 of the projective transformation unit 409 performs projective transformation on the contour position information indicating the contour 505 of the transformed divided image using the calculated projective transformation matrix (step S14). Thereafter, the composition information generation unit 413 generates contour position information of the composite image when the reference divided image and the converted divided image after the projective conversion are combined based on the contour position information subjected to the projective conversion ( Step S15). Then, the outline 505 of the composite image 501 is displayed on the display unit 415 (step S16).

In the above-described embodiment, the hardware structure of a processing unit (processing unit) that executes various processes is the following various processors. For various processors, the circuit configuration can be changed after manufacturing a CPU (Central Processing Unit) or FPGA (Field Programmable Gate Array) that is a general-purpose processor that functions as various processing units by executing software (programs). Includes dedicated logic circuits such as programmable logic devices (Programmable Logic Devices: PLDs) and ASICs (Application Specific Specific Integrated Circuits) that have specially designed circuit configurations to execute specific processing. It is.

One processing unit may be configured by one of these various processors, or may be configured by two or more processors of the same type or different types (for example, a plurality of FPGAs or a combination of CPU and FPGA). May be. Further, the plurality of processing units may be configured by one processor. As an example of configuring a plurality of processing units with one processor, first, as represented by a computer such as a client or server, one processor is configured with a combination of one or more CPUs and software. There is a form in which the processor functions as a plurality of processing units. Second, as represented by a system-on-chip (SoC), a form of using a processor that realizes the functions of the entire system including a plurality of processing units with a single IC (integrated circuit) chip. is there. As described above, various processing units are configured using one or more of the various processors as a hardware structure.

Further, the hardware structure of these various processors is more specifically an electric circuit (circuitry) in which circuit elements such as semiconductor elements are combined.

The above-described configurations and functions can be appropriately realized by arbitrary hardware, software, or a combination of both. For example, a program that causes the computer 300 to execute the above-described processing steps (processing procedure), a computer-readable recording medium (non-temporary recording medium) that records such a program, or a computer 300 that can install such a program The present invention can also be applied to the above.

<Second Embodiment>
Next, a second embodiment of the present invention will be described. First, an example of shooting spill that is solved in the present embodiment will be described.

FIG. 12 is a diagram showing a composite image 501 in which type 3 shooting has been missed.

In the composite image 501 shown in FIG. Note that the composite image 501 shown in FIG. 12 is obtained by projecting and combining a plurality of divided images obtained by dividing and photographing the floor slab 6. That is, the divided images obtained by photographing the floor slab 6 are projected and combined to form a composite image 501.

In the composite image 501 shown in FIG. In other words, the floor slab 6 in the area 502 of the space 502 should be shown without omission in the composite image 501, but the image of the floor slab 6 corresponding to the end of the space 502 is displayed at the location of the overshoot 509. Lack.

FIG. 13 is a diagram showing a contour 505 of the composite image 501 displayed on the touch panel display 302 by the display unit 415 based on the contour information of the composite image 501 generated by the composite information generation unit 413. FIG. 13 is a display of the outline 505 of the composite image 501 for the composite image 501 having the spillover 509 described in FIG.

For comparison, FIG. 13A shows a composite image 501 obtained by normally combining a split image (a reference split image and a converted split image after projective transformation).

On the other hand, FIG. 13B shows a contour 505 of the composite image 501 based on the contour information of the composite image 501 generated by the composite information generation unit 413. Here, in the case of a shooting spill 509 such as type 3, the user may confirm that there is a shooting spill by simply checking the outline 505 of the composite image 501 as described in the first embodiment. Have difficulty. That is, as shown in FIG. 13, it is difficult to determine whether the floor slab 6 of the space 502 can be photographed without omission only by the outline 505 of the composite image 501 displayed on the display unit 415.

In the present embodiment, a divided image (a reference divided image or a converted divided image obtained by projective transformation) is also displayed together with the outline 505 of the composite image 501 so that a shooting failure such as type 3 can also be confirmed.

FIG. 14 is a diagram illustrating a functional configuration example of the image processing apparatus 400 according to the present embodiment. The parts already described with reference to FIG.

The image processing apparatus 400 includes an image input unit 401, a corresponding point information acquisition unit 403, a reference divided image specification unit 405, a matrix calculation unit 407, a projective conversion unit 409, a composite information generation unit 413, a display unit 415, and a first designation reception. Part 419 is provided. In addition, the part already demonstrated in FIG. 7 attaches the same code | symbol, and abbreviate | omits description.

The projection conversion unit 409 includes a first projection conversion unit 411 and a second projection conversion unit 417. The projection conversion unit 409 includes a plurality of functions (a first projection conversion unit 411 and a second projection conversion unit 417). However, the projection conversion unit 409 may be realized by one projection conversion unit 409, or each function may be provided. Hardware (or software, or a combination of both) may be assigned.

The second projective transformation unit 417 performs projective transformation on a part of the converted divided images based on the projective transformation matrix. That is, the second projective transformation unit 417 performs projective transformation on a part of the transformed divided images based on the projective transformation matrix, and generates a transformed divided image that has undergone projective transformation.

Specific examples of the converted divided image to which the second projective conversion unit 417 performs projective conversion include a converted divided image corresponding to the corner of the composite image 501 and a converted divided image corresponding to the four corners of the composite image 501. It is done. As another specific example of the converted divided image for which the second projective conversion unit 417 performs the projective conversion, a converted divided image corresponding to the side of the composite image 501 and a converted split corresponding to the four sides of the composite image 501 are shown. An image is given.

The composition information generation unit 413 synthesizes the contour position information subjected to the projective conversion by the first projective conversion unit 411 and the converted divided image subjected to the projective conversion by the second projective conversion unit 417. That is, the contour position information of the synthesized image 501 is generated, and the transformed divided image that has undergone projective transformation by the second projective transformation unit 417 is synthesized so as to match the contour 505 of the synthesized image 501.

The display unit 415 displays the contour position information of the composite image 501 generated by the composite information generation unit 413 and the converted divided image or the reference divided image after projective conversion. That is, the display unit 415 synthesizes and displays at least one of the converted divided image and the reference divided image on the contour 505 indicated by the contour position information of the synthesized image 501.

The first designation accepting unit 419 accepts designation of a position in the contour 505 of the composite image 501 displayed on the display unit 415. Specifically, the first designation receiving unit 419 receives the designation of the position of the composite image 501 when the contour 505 of the composite image 501 displayed on the touch panel display 302 of the computer 300 is designated by the user. Then, when the designation is received by the first designation receiving unit 419, the second projective conversion unit 417 performs projective transformation on the converted divided image corresponding to the position received by the first designation receiving unit 419. May be. In other words, the user performs display of the reference divided image displayed in accordance with the outline 505 of the composite image 501 displayed on the display unit 415 or the converted divided divided image via the first designation receiving unit 419. Also good.

FIG. 15 is a diagram showing a contour 505 of the composite image 501 displayed on the touch panel display 302 by the display unit 415 based on the contour information of the composite image 501 generated by the composite information generation unit 413.

For comparison, FIG. 15A shows an outline 505 of a composite image 501 obtained by normally combining a split image (a reference split image and a converted split image after projective transformation).

On the other hand, in FIG. 15B, the outline 505 of the composite image 501 and a part of the divided images are displayed. Specifically, the divided image 511, the divided image 513, the divided image 515, and the divided image 517 corresponding to the contour 505 of the composite image 501 and the four corners of the contour 505 of the composite image 501 are displayed in accordance with the contour 505 of the composite image 501. Has been.

After the user acquires a plurality of divided images for the floor slab 6 of one space 502, the user confirms the display of the outline 505 of the composite image 501 and a part of the divided images. It can be confirmed that the floor slab 6 is missing, and the outline 505 corresponding to the shooting spill 509 indicates that spilling has occurred for the floor slab 6 in the space 502.

<Third Embodiment>
Next, a third embodiment of the present invention will be described. In the present embodiment, a reduced image is displayed at a part of the composite image 501 corresponding to the divided image displayed in the second embodiment.

FIG. 16 is a diagram illustrating a functional configuration example of the image processing apparatus 400 according to the present embodiment. The parts already described with reference to FIG.

The image processing apparatus 400 includes an image input unit 401, a reduced image generation unit 423, a corresponding point information acquisition unit 403, a reference divided image specification unit 405, a matrix calculation unit 407, a projective conversion unit 409, a composite information generation unit 413, and a display unit. 415.

The reduced image generation unit 423 generates a reduced image in which the resolution of the divided image input to the image input unit 401 is lowered. Specifically, the reduced image generation unit 423 generates reduced images of 90% to 50%, preferably 80% to 60% of the number of pixels included in the divided image. The reduced image generation unit 423 generates a reduced image by a known technique.

The projection conversion unit 409 includes a first projection conversion unit 411 and a third projection conversion unit 421. The third projective transformation unit 421 performs projective transformation on a part of the reduced images based on the projective transformation matrix. That is, the third projective transformation unit 421 performs projective transformation on a part of the reduced images based on the projective transformation matrix, and generates a reduced image subjected to the projective transformation. The projective conversion unit 409 includes three functions (a first projective conversion unit 411 and a third projective conversion unit 421). However, the projective conversion unit 409 may be realized by one projective conversion unit 409. Hardware (or software, or a combination of both) may be assigned.

Specific examples of the reduced image for which the third projective conversion unit 421 performs the projective conversion include a reduced image corresponding to the corner of the composite image 501 and a reduced image corresponding to the four corners of the composite image 501. Other specific examples of the reduced image for which the third projective conversion unit 421 performs projective conversion include a reduced image corresponding to the side of the composite image 501 and a reduced image corresponding to the four sides of the composite image 501.

The display unit 415 displays a part of the reduced image converted by the third projective conversion unit 421 on the outline 505 of the composite image 501. In this way, by displaying the reduced image corresponding to the projection-converted reduced image or the reference divided image on the display unit 415, only the display of the outline 505 of the composite image 501 as described in the second embodiment. Then, it is possible to make the user confirm a shooting spill that is difficult to distinguish. In the present embodiment, since the reduced image is projectively transformed, processing in a shorter time and reduction in calculation cost can be realized.

<Fourth Embodiment>
Next, a fourth embodiment of the present invention will be described. In the present embodiment, the reduced image displayed in the third embodiment is specified, and the display of the specified reduced image is switched to the display of a divided image having a high resolution.

FIG. 17 is a diagram illustrating a functional configuration example of the image processing apparatus 400 of the present embodiment. The parts already described with reference to FIG.

The image processing apparatus 400 includes an image input unit 401, a reduced image generation unit 423, a corresponding point information acquisition unit 403, a reference divided image specification unit 405, a matrix calculation unit 407, a projective conversion unit 409, a composite information generation unit 413, and a display unit 415. , And a second designation receiving unit 425.

The projection conversion unit 409 includes a first projection conversion unit 411, a second projection conversion unit 417, and a third projection conversion unit 421. The projective conversion unit 409 includes three functions (a first projective conversion unit 411, a second projective conversion unit 417, and a third projective conversion unit 421), and is realized by a single projective conversion unit 409. Alternatively, hardware (or software, or a combination of both) may be assigned to each function.

The second projective transformation unit 417 performs projective transformation on the transformed divided image using the projective transformation matrix when the divided image corresponding to the position received by the second designation unit is the transformed divided image. .

The second designation receiving unit 425 receives the designation of the reduced image displayed on the display unit 415. Specifically, the second designation receiving unit 425 receives the designation of the position of the composite image 501 when the user designates a reduced image of the composite image 501 displayed on the touch panel display 302 of the computer 300.

The display unit 415 switches the reduced image designated by the second designation receiving unit 425 to the converted divided image or the reference divided image corresponding to the reduced image and subjected to the projective transformation by the second projective transformation unit 417. That is, the display unit 415 switches the reduced image designated by the second designation receiving unit 425 to a divided image with higher resolution and displays it. As described above, by switching from a reduced image having a low resolution to a divided image having a high resolution in accordance with a designation from the user, it is possible to provide a display that enables more accurate confirmation of the spillage.

<Fifth Embodiment>
Next, a fifth embodiment of the present invention will be described. In the present embodiment, the input divided image is converted into a reduced image with reduced resolution, and the outline 505 of the composite image 501 is displayed based on the converted reduced image as described in the first embodiment. . In the present embodiment, the processing is performed based on the reduced image, so that the processing time can be further shortened.

FIG. 18 is a diagram illustrating a functional configuration example of the image processing apparatus 400 according to the present embodiment. In addition, the part already demonstrated in FIG. 7 attaches | subjects the same code | symbol, and abbreviate | omits description.

The image processing apparatus 400 includes an image input unit 401, a reduced image generation unit 423, a corresponding point information acquisition unit 403, a reference reduced image specification unit 427, a matrix calculation unit 407, a projective conversion unit 409, a composite information generation unit 413, and a display unit. 415.

Corresponding point information acquisition unit 403 searches for corresponding points between the reduced images and acquires corresponding point information regarding the corresponding points.

The reference reduced image specifying unit 427 specifies a reference reduced image serving as a reference for projective transformation among the plurality of reduced images.

The matrix calculation unit 407 calculates a projective transformation matrix for the reference reduced image of the converted reduced image other than the reference reduced image among the plurality of reduced images based on the corresponding point information and the reference reduced image.

The third projective transformation unit 421 performs projective transformation on the contour position information indicating the contour of the transformed reduced image using the calculated projective transformation matrix.

The composite image generation unit 413 generates the contour position information of the composite image 501 when the reference reduced image and the converted reduced image after the projective conversion are combined based on the projective converted contour position information.

The display unit 415 displays the contour 505 of the composite image 501 based on the contour position information of the composite image 501.

The examples of the present invention have been described above, but the present invention is not limited to the above-described embodiments, and it goes without saying that various modifications can be made without departing from the spirit of the present invention.

DESCRIPTION OF SYMBOLS 1 Bridge 2 Main girder 3 Horizontal girder 4 Opposite structure 5 Horizontal structure 6 Floor slab 10 Vehicle 100 Mobile body 101 Camera 300 Computer 302 Touch panel display 304 Speaker 305 Built-in camera 306 External connection terminal 310 CPU
312 System bus 314 Main memory 316 Non-volatile memory 318 Mobile communication unit 318A Antenna 320 Wireless LAN communication unit 320A Antenna 322 Short-range wireless communication unit 322A Antenna 324 Wired communication unit 326 Display unit 328 Input unit 330 Key input unit 332 Audio processing unit 334 Image processing unit 400 Image processing device 401 Image input unit 403 Corresponding point information acquisition unit 405 Reference divided image specification unit 407 Matrix calculation unit 409 Projection conversion unit 411 First projection conversion unit 413 Composite information generation unit 415 Display unit 417 Second Projection conversion unit 419 First designation reception unit 421 Third projection conversion unit 423 Reduced image generation unit 425 Second designation reception unit 427 Reference reduced image specification unit 500 Shooting system 501 Composite image 502 Space 505 Outline 511 Divided image 5 13 divided image 515 divided image 517 divided image S10-S16 image processing steps

Claims

An image input unit for inputting a plurality of divided images obtained by dividing and shooting the shooting target;
A corresponding point information acquisition unit that searches for corresponding points between the plurality of divided images and acquires corresponding point information about the corresponding points;
A reference divided image specifying unit for specifying a reference divided image serving as a reference for projective transformation among the plurality of divided images;
A matrix calculation unit that calculates a projective transformation matrix for the reference divided image of the converted divided image other than the reference divided image among the plurality of divided images based on the corresponding point information and the reference divided image;
A first projective transformation unit that performs projective transformation on contour position information indicating a contour of the transformed divided image, based on the calculated projective transformation matrix;
Based on the contour-converted contour position information, a composite information generation unit that generates contour position information of a composite image when the reference split image and the conversion-converted split image after the projective conversion are combined;
Based on the contour position information of the composite image, a display unit that displays the contour of the composite image;
An image processing apparatus.
Based on the projective transformation matrix, comprising a second projective transformation unit for projective transformation of a part of the transformed divided images,
The image according to claim 1, wherein the display unit displays at least one of the partial converted divided image and the reference divided image converted by the second projective conversion unit on an outline of the composite image. Processing equipment.
The image processing apparatus according to claim 2, wherein the second projective conversion unit performs projective conversion on the converted divided image corresponding to a corner of the composite image.
The image processing apparatus according to claim 3, wherein the second projective conversion unit performs projective conversion on the converted divided image corresponding to four corners of the composite image.
The image processing apparatus according to any one of claims 2 to 4, wherein the second projective conversion unit performs projective conversion on the converted divided image corresponding to a side of the composite image.
6. The image processing apparatus according to claim 5, wherein the second projective conversion unit performs projective conversion on the converted divided image corresponding to four sides of the composite image.
A first designation accepting unit that accepts designation of a position in the outline of the composite image displayed on the display unit;
The image processing apparatus according to claim 2, wherein the second projective conversion unit performs projective conversion on the converted divided image corresponding to the position received by the first designation receiving unit. .
A reduced image generation unit that generates a reduced image with a reduced resolution of the divided image input to the image input unit;
A third projective transformation unit that performs projective transformation on a part of the reduced image based on the projective transformation matrix;
The image processing apparatus according to claim 1, wherein the display unit displays the part of the reduced image converted by the third projective conversion unit on an outline of the composite image.
A reduced image generation unit that generates a reduced image with a reduced resolution of the divided image input to the image input unit;
A third projective transformation unit for projective transformation of the reduced image based on the projective transformation matrix;
A second designation accepting unit for accepting designation of the reduced image displayed on the display unit;
With
The display unit includes the converted divided image or the reference divided image obtained by performing projective conversion on the reduced image designated by the second designation receiving unit and corresponding to the reduced image by the second projective transformation unit. The image processing apparatus according to claim 2, wherein the image processing apparatus is switched to an image.
The reference divided image specifying unit specifies the reference divided image based on designation of the reference divided image or information on an input order of the divided images to be the reference divided image. The image processing apparatus according to item.
An image input unit for inputting a plurality of divided images obtained by dividing and shooting the shooting target;
A reduced image generation unit that generates a reduced image with a reduced resolution of the divided image input to the image input unit;
A corresponding point information acquisition unit that searches for corresponding points between the reduced images and acquires corresponding point information about the corresponding points;
A reference reduced image specifying unit for specifying a reference reduced image serving as a reference for projective transformation among the plurality of reduced images;
A matrix calculation unit that calculates a projective transformation matrix for the reference reduced image of the converted reduced image other than the reference reduced image among the plurality of reduced images based on the corresponding point information and the reference reduced image;
A third projective transformation unit that performs projective transformation on contour position information indicating the contour of the transformed reduced image by the calculated projective transformation matrix;
Based on the contour-converted contour position information, a composite information generating unit that generates contour position information of a composite image when the reference reduced image and the converted reduced image after the projective conversion are combined;
Based on the contour position information of the composite image, a display unit that displays the contour of the composite image;
An image processing apparatus.
A step of inputting a plurality of divided images obtained by dividing and shooting the shooting target;
Searching for corresponding points between the plurality of divided images to obtain corresponding point information on the corresponding points;
Identifying a reference divided image as a reference for projective transformation among the plurality of divided images;
Calculating a projective transformation matrix for the reference divided image of the converted divided image other than the reference divided image among the plurality of divided images based on the corresponding point information and the reference divided image;
Projective transformation of contour position information indicating the contour of the transformed divided image by the calculated projective transformation matrix;
Generating outline position information of a synthesized image when the reference divided image and the transformed divided image after the projection transformation are synthesized based on the outline position information obtained by the projective transformation;
Displaying the contour of the composite image based on the contour position information of the composite image;
An image processing method including:
A step of inputting a plurality of divided images obtained by dividing and shooting the shooting target;
Generating a reduced image with reduced resolution of the divided image;
Searching for corresponding points between the reduced images to obtain corresponding point information about the corresponding points;
Identifying a reference reduced image serving as a reference for projective transformation among the plurality of reduced images;
Calculating a projective transformation matrix for the reference reduced image of the converted reduced image other than the reference reduced image among the plurality of reduced images based on the corresponding point information and the reference reduced image;
A projective transformation of contour position information indicating a contour of the transformed reduced image by the calculated projective transformation matrix;
Generating contour position information of a composite image when the reference reduced image and the converted reduced image after the projective conversion are combined based on the projective-converted contour position information;
Displaying the contour of the composite image based on the contour position information of the composite image;
An image processing method including:
A step of inputting a plurality of divided images obtained by dividing and shooting the shooting target;
Searching for corresponding points between the plurality of divided images to obtain corresponding point information on the corresponding points;
Identifying a reference divided image as a reference for projective transformation among the plurality of divided images;
Calculating a projective transformation matrix for the reference divided image of the converted divided image other than the reference divided image among the plurality of divided images based on the corresponding point information and the reference divided image;
Projective transformation of contour position information indicating the contour of the transformed divided image by the calculated projective transformation matrix;
Generating outline position information of a synthesized image when the reference divided image and the transformed divided image after the projection transformation are synthesized based on the outline position information obtained by the projective transformation;
Displaying the contour of the composite image based on the contour position information of the composite image;
A program for causing a computer to execute a process including:
A computer-readable recording medium on which the program according to claim 14 is recorded.
A step of inputting a plurality of divided images obtained by dividing and shooting the shooting target;
Generating a reduced image with reduced resolution of the divided image;
Searching for corresponding points between the reduced images to obtain corresponding point information about the corresponding points;
Identifying a reference reduced image serving as a reference for projective transformation among the plurality of reduced images;
Calculating a projective transformation matrix for the reference reduced image of the converted reduced image other than the reference reduced image among the plurality of reduced images based on the corresponding point information and the reference reduced image;
A projective transformation of contour position information indicating a contour of the transformed reduced image by the calculated projective transformation matrix;
Generating contour position information of a composite image when the reference reduced image and the converted reduced image after the projective conversion are combined based on the projective-converted contour position information;
Displaying the contour of the composite image based on the contour position information of the composite image;
A program for causing a computer to execute a process including:
The computer-readable recording medium which recorded the program of Claim 16.