JP2004342004A - Image processing device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide the technique of an image processing device which can efficiently generate predetermined data while preventing unnecessary photographing when sequential photographing is performed and predetermined data is generated based on plural pieces of the image data photographed. <P>SOLUTION: A data processing system comprises a camera and an image processing device. Image data of an object photographed by a photographer is transmitted to the image processing device. In the image processing device, modeling is performed based on the image data transmitted from the camera 2 for every photographing and plural pieces of image data already acquired, and texture image (photographed image) is mapped onto the three-dimensional model generated. In the case where the mapping is good, it is determined that photographing should be completed, and, in the case where the mapping is bad, it is determined that photographing should be continued, and this determination result is transmitted to the camera. The camera, which receives this notification of the completion of photographing, completes photographing. Due to this, image processing can be efficiently performed while preventing unnecessary photographing. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image processing apparatus capable of processing image data.
[0002]
[Prior art]
2. Description of the Related Art Some conventional image processing techniques perform modeling and generate a panoramic image based on a plurality of image data obtained by sequentially photographing a subject.
[0003]
The above-described modeling is disclosed in, for example, Patent Document 1, and describes a technique for generating a three-dimensional model from a silhouette image of an object.
[0004]
Further, generation of a panoramic image is disclosed, for example, in Patent Document 2, and describes a technique of creating a panoramic image by combining and combining a plurality of two-dimensional images.
[0005]
[Patent Document 1]
JP-A-8-105724
[Patent Document 2]
JP-A-11-205648
[0006]
[Problems to be solved by the invention]
However, in the above-described modeling and generation of a panoramic image, when a photographer sequentially shoots, it is determined whether or not sufficient image data necessary for modeling or generation of a panoramic image has been acquired at the time of the current imaging. difficult. Therefore, excessive shooting may be performed. In this case, useless shooting occurs, and processing such as modeling cannot be performed efficiently and quickly.
[0007]
The present invention has been made in view of the above-described problems, and when generating predetermined data based on a plurality of image data sequentially captured and acquired, it is possible to prevent unnecessary shooting and efficiently convert predetermined data. It is an object of the present invention to provide a technology of an image processing device that can generate the image.
[0008]
[Means for Solving the Problems]
In order to solve the above-mentioned problems, the invention according to claim 1 is an image processing apparatus capable of processing image data, wherein (a) a fixed image processing apparatus based on a plurality of image data sequentially photographed and acquired by photographing means. Processing means for generating predetermined data that satisfies the conditions of (b), and (b) when one photographing operation is performed by the photographing means, one image data acquired by the one photographing operation and one photographing operation Determining means for determining whether the predetermined data can be generated based on a plurality of image data acquired before the operation; and (c) the predetermined data can be generated by the determining means. If it is determined that there is, a notifying unit is provided for notifying the end of the shooting by the shooting unit.
[0009]
According to a second aspect of the present invention, in the image processing apparatus according to the first aspect of the present invention, the processing means performs (B-1) three-dimensional data processing based on the plurality of pieces of image data to convert the three-dimensional data. Means for generating.
[0010]
According to a third aspect of the present invention, in the image processing apparatus according to the second aspect of the present invention, the processing unit performs (b-2) a calibration process related to the three-dimensional data processing based on the plurality of image data. And means for generating unique data of the photographing means.
[0011]
According to a fourth aspect of the present invention, in the image processing apparatus according to any one of the first to third aspects, the processing means performs (b-3) an image combining process based on the plurality of pieces of image data. Means for generating panoramic image data.
[0012]
According to a fifth aspect of the present invention, there is provided a program executed by a computer incorporated in an image processing apparatus to cause the image processing apparatus to function as the image processing apparatus according to any one of the first to fourth aspects. It is.
[0013]
BEST MODE FOR CARRYING OUT THE INVENTION
<First embodiment>
<Three-dimensional data processing system>
FIG. 1 is a schematic diagram illustrating a main configuration of a data processing system 1A according to the first embodiment of the present invention.
[0014]
The data processing system 1A includes a camera 2 for photographing an object SB as a subject, and an image processing device 3A communicably connected to the camera 2 via a cable CB. In addition, the target object SB is an ornament, and its shape is constant and stable.
[0015]
The camera 2 can acquire a two-dimensional color image of the object SB, which is a three-dimensional object, using a digital camera, for example. The camera 2 serving as a photographing unit is provided with a display unit 20 such as a liquid crystal display on the back surface.
[0016]
The image processing device 3A is configured as a personal computer, for example, and can process image data. The image processing apparatus 3A includes a processing unit 30 having a box shape, an operation unit 31, and a display unit 32 configured by, for example, a CRT.
[0017]
A drive 301 for inserting the optical disc 9 is provided on the front of the processing unit 30.
[0018]
The operation unit 31 has a mouse 311 and a keyboard 312, and receives an operation input to the image processing apparatus 3A from a user.
[0019]
FIG. 2 is a diagram illustrating functional blocks of the data processing system 1A.
[0020]
The camera 2 includes an imaging unit 22 that captures an image of the target SB via the optical unit 21, and a control unit 23 that is communicably connected to these units. Further, the camera 2 includes a communication unit I / F 24 and a position / posture detection unit 25 which are connected to the control unit 23 so as to be able to transmit.
[0021]
The optical unit 21 has a lens group and a drive unit that changes the arrangement of these lens groups, and forms an optical image of the target SB on the imaging unit 22 by performing focus and zoom.
[0022]
The imaging unit 22 includes an imaging element such as a CCD, for example. Then, after an analog image signal of the object SB is acquired by the imaging element, A / D conversion is performed to generate image data of a digital signal.
[0023]
The communication I / F 24 is an interface for transmitting data with the image processing apparatus 3A via the cable CB.
[0024]
The position / posture detection unit 25 has, for example, an acceleration sensor and a gyro, and is a part that detects a three-dimensional position of the camera 2 and a posture of the camera 2 that determines a shooting direction. The position / posture information of the camera 2 is obtained by the position / posture detection unit 25.
[0025]
The control unit 23 has a CPU, and is a part that controls the operation of the camera 2 overall. Further, the control unit 23 performs control for transmitting the image data acquired by the imaging unit 22 to the image processing device 3A via the communication I / F 24.
[0026]
The image processing device 3A includes a control unit 33 that is connected to the operation unit 31 and the display unit 32 so that transmission is possible. Further, the image processing apparatus 3A includes a storage unit 34 that is connected to the control unit 33 so that transmission is possible, an input / output I / F 35, and a communication I / F 36.
[0027]
The storage unit 34 is configured as, for example, a hard disk, and stores a shooting support program DP described later.
[0028]
The input / output I / F 35 is an interface for controlling transmission and reception of data between the control unit 33 and the operation unit 31 and the display unit 32.
[0029]
The input / output I / F 35 is an interface for inputting and outputting data to and from the optical disc 9 as a recording medium via the drive 301.
[0030]
The communication I / F 36 is an interface for transmitting data with the camera 2 via the cable CB.
[0031]
The control unit 33 includes a CPU 331 and a memory 332 that function as a computer, and is a unit that controls the operation of the image processing apparatus 3A. When the control unit 33 executes the photographing support program DP, the image processing device 3A performs an operation that supports the photographing operation of the photographer and prevents unnecessary photographing (described later in detail).
[0032]
In the memory 332 of the control unit 33, program data such as the photographing support program DP recorded on the optical disk 9 can be stored via the input / output I / F 35. Thus, the stored program can be reflected on the operation of the image processing apparatus 3A.
[0033]
<Modeling and texture mapping>
Modeling in the data processing system 1A is performed by the image processing device 3A. That is, the image data acquired by the camera 2, the position / posture information of the camera 2, and the internal parameters are transmitted to the image processing apparatus 3 </ b> A, and the control unit 33 of the image processing apparatus 3 </ b> A that has received these data, for example, Shape From Shilohette A three-dimensional shape model of the object SB is generated using a method (hereinafter, referred to as “SFS method”). This modeling process will be described below.
[0034]
FIG. 3 is a diagram for explaining modeling. FIG. 3B shows a state in which the camera 2 is moved by a predetermined amount in the direction Ha (see FIG. 1) from the photographing state shown in FIG.
[0035]
First, contours Gs1 and Gs2 (thick lines) of the object SB are extracted from the images G1 and G2 obtained by photographing the object SB, and a silhouette image thereof is created. After extracting the contours Gs1 and Gs2 of the target SB based on the color difference between the object surface and the background, the silhouette image is binarized to set the inside of the contour to “1” and the outside of the contour to “0”, for example. Created as image data. That is, the silhouette image is a silhouette of the target SB viewed from different viewpoints (photographing positions) H1 and H2 in the line-of-sight directions (photographing directions) Hd1 and Hd2 with respect to the target SB, that is, an object centered on the viewpoints H1 and H2. This is a projection image of the object SB onto the two-dimensional space.
[0036]
Then, for example, by moving the camera 2 around the target SB in the direction Ha (FIG. 1), the plurality of images (silhouette images) acquired while relatively changing the shooting direction with respect to the subject include the position of the camera 2 Utilizing the posture information and the internal parameters, it can be associated with a predetermined three-dimensional coordinate system. Thus, the two-dimensional shape that is the silhouette area of the target SB can be integrated by using the above-described SFS method, and a three-dimensional shape model of the target SB, specifically, polygon data can be generated.
[0037]
The polygon data of the object SB generated as described above is mapped with a photographed image (texture image) photographed by the camera 2 such as images G1 and G2. In this mapping, attention is paid to each polygon of the three-dimensional shape model, and a texture image is mapped such that the polygon and the texture substantially face each other.
[0038]
As described above, modeling and texture mapping (three-dimensional data processing) are performed based on a plurality of pieces of image data obtained by sequentially photographing the object SB with the camera 2 to perform texture mapping. A three-dimensional model (three-dimensional data) of the SB is generated.
[0039]
In texture mapping, when the number of acquired images is not sufficient, a defect state where a texture cannot be mapped to a polygon due to an insufficient amount of information is mainly involved. More specifically, when there is a polygon for which no texture image is directly opposed, for example, when the rear surface of the object SB is not photographed, there is no texture image to be assigned to the polygon at the rear of the object SB. Will occur.
[0040]
On the other hand, when a plurality of image data have been acquired without excess or deficiency, textures are appropriately mapped on polygons generated from these image data, and good three-dimensional data can be generated. Is not efficient to get.
[0041]
From the above, it is possible to eliminate unnecessary photographing if it is possible to determine whether or not sufficient image data necessary for performing texture mapping on the three-dimensional shape model has been acquired and notify the photographer of the photographing support. Becomes possible. The photographing support operation in the data processing system 1A will be described below.
[0042]
<Operation of Data Processing System 1A>
FIG. 4 is a flowchart illustrating a basic operation in the data processing system 1A. FIG. 4A shows the processing of the camera 2, and FIG. 4B shows the processing of the image processing device 3A. The processing of the image processing apparatus 3A is performed by the control unit 33 executing the data processing program DP.
[0043]
In step S <b> 1, the camera 2 generates image data of an object SB photographed by a photographer.
[0044]
In step S2, the image data generated in step S1 is transmitted via the communication I / F 24. When transmitting the image data, the position / posture information and the internal parameters of the camera 2 detected by the position / posture detection unit 25 are also transmitted. The internal parameters (camera parameters) represent shooting conditions and the like, and are uniquely determined based on information such as the focal length and the optical axis center at the time of shooting, and are known in the camera 2 of the first embodiment. Has become.
[0045]
The image data, the position / orientation information, and the internal parameters transmitted from the camera 2 as described above are received by the image processing device 3A (step S11).
[0046]
In step S12, the above-described modeling is performed from the image data and the position / posture information received in step S11.
[0047]
In step S13, it is determined whether texture mapping can be performed favorably on the three-dimensional model generated in step S12. That is, when there is one photographing operation by the camera 2 in step S1, a certain amount of photographing operation is performed based on image data acquired by the photographing operation and a plurality of image data acquired before the photographing operation. It is determined whether or not predetermined data that satisfies the condition, that is, three-dimensional data to which texture mapping is appropriately performed can be generated. Here, as described above, the texture is mapped to the three-dimensional model, and it is determined whether or not there is an unmapped area.
[0048]
In step S13, if the texture mapping is good, the process proceeds to step S14; otherwise, the process proceeds to step S15.
[0049]
In step S14, the end of shooting is determined. Here, since the texture mapping is good in the determination in step S13, a signal indicating the end of imaging is generated.
[0050]
In step S15, it is determined whether to continue shooting. Here, since the texture mapping is defective in the determination in step S13, a signal to continue shooting is generated.
[0051]
In step S16, the determination result is transmitted to the camera 2. Specifically, the signal of the end of imaging generated in step S14 or the signal of continuation of imaging generated in step S15 is transmitted to camera 2.
[0052]
The determination result transmitted from the image processing device 3A is received by the camera 2 (step S3).
[0053]
In step S4, it is determined whether or not the result of the determination received in step S3 is the end of shooting. Here, when the notification of the end of the photographing is received from the image processing device 3A, that is, when the signal of the end of the photographing is received, for example, the display of the end of the photographing is displayed on the display unit 20 of the camera 2, and the photographing operation is ended. Then, the process ends. At this time, the texture is mapped on the three-dimensional shape model, and appropriate three-dimensional data is generated by the image processing device 3A. On the other hand, when the notification that the shooting is not completed is received, that is, when the signal of the shooting continuation is received, for example, the display of the shooting continuation is displayed on the display unit 20 of the camera 2 so that the shooting operation is continued, and the process returns to the step S1. .
[0054]
By the above-described operation of the data processing system 1A, the image processing apparatus 3A automatically determines the end of shooting and notifies the camera 2 in almost real time every time shooting is performed. It can be generated efficiently.
[0055]
<Second embodiment>
The data processing system 1B according to the second embodiment of the present invention has a configuration similar to that of the data processing system 1A according to the first embodiment shown in FIGS. The photographing support program DP is different.
[0056]
That is, the image processing apparatus 3A of the first embodiment has a shooting support program DP for performing a shooting support operation when performing modeling with the internal parameters of the camera 2 being known, but the image processing apparatus 3B of the second embodiment. Has a shooting support program DP for performing a shooting support operation for performing camera calibration (calibration) when the internal parameters of the camera 2 are unknown. This calibration will be described below.
[0057]
<Calibration>
The calibration in the data processing system 1B is described in, for example, a document “Reconstruction of camera internal variables, three-dimensional shape, and three-dimensional motion using three central projected images”, Noriko Sugimoto, Xu Go, IPSJ Research Report, CVIM 99-114. Pp. 9-16, 1999]. Hereinafter, this method will be briefly described.
[0058]
FIG. 5 is a flowchart illustrating the calibration operation.
[0059]
First, a plurality of image data obtained by photographing the object SB with the camera 2 is transmitted to the image processing device 3B. Then, in the image processing apparatus 3B, calibration is performed by executing the following steps based on the received plurality of image data.
[0060]
In step S21, feature points are associated between images. For example, eight or more feature points such as the nose tip Sp1, the ear tips Sp2, and Sp3 shown in FIG. 6 are set in each image, and their correspondence is taken.
[0061]
In step S22, an F matrix (fundamental matrix) is calculated based on the feature points associated in step S21. This F matrix is information including internal parameters of the camera 2 and external parameters representing the position and orientation of the camera 2.
[0062]
In step S23, the internal parameters of the camera 2 are calculated based on the F matrix calculated in step S22. In this case, for example, an internal function is calculated by setting an energy function (evaluation function) related to the F matrix and solving the value by the steepest descent method so that a numerical value derived from the energy function becomes small.
[0063]
In step S24, an E matrix (Essential Matrix) is calculated based on the F matrix calculated in step S22 and the internal parameters calculated in step S23. The E matrix is information including only external parameters of the camera 2.
[0064]
In step S25, external parameters of the camera 2 are calculated based on the E matrix calculated in step S24.
[0065]
By the above-described calibration processing, unknown internal parameters (unique data) of the camera 2 are generated and can be grasped. By using the internal parameters, the modeling in the first embodiment can be appropriately performed.
[0066]
In this calibration, similarly to the texture mapping of the first embodiment, if an unsatisfactory operation result is obtained, it is necessary to additionally acquire new image data and complement it. For example, in the case where feature point correspondence between images cannot be obtained in step S21, or in the case where the energy value is equal to or greater than a predetermined threshold value in step S23, it is determined that the image is defective.
[0067]
On the other hand, if a plurality of image data can be acquired without excess or deficiency, the calibration will be good, but acquiring image data more than necessary is not efficient.
[0068]
Accordingly, in the calibration, as in the first embodiment, if it is possible to determine whether or not necessary and sufficient image data has been obtained and notify the photographer of the photographing, it is possible to eliminate unnecessary photographing. It becomes possible. The photographing support operation in the data processing system 1B will be described below.
[0069]
<Operation of Data Processing System 1B>
FIG. 7 is a flowchart illustrating a basic operation in the data processing system 1B. FIG. 7A shows the processing of the camera 2, and FIG. 7B shows the processing of the image processing device 3B. The processing of the image processing apparatus 3B is performed by the control unit 33 executing the data processing program DP.
[0070]
In step S31, the camera 2 generates image data of the target SB photographed by the photographer.
[0071]
In step S32, the image data generated in step S31 is transmitted via the communication I / F 24.
[0072]
The image data transmitted from the camera 2 as described above is received by the image processing device 3B (step S41).
[0073]
In step S42, the calibration operation shown in FIG. 5 is performed from the image data received in step S41.
[0074]
In step S43, it is determined whether the calibration performed in step S42 is good. That is, when one shooting operation is performed by the camera 2 in step S31, a fixed image data is obtained based on the image data obtained by the shooting operation and the plurality of image data obtained before the shooting operation. It is determined whether predetermined data that satisfies the condition, that is, internal parameters (unique data) of the camera 2 in an appropriate numerical range can be generated. Here, as described above, when there is no correspondence between feature points between images or when the energy value is less than a predetermined threshold, it is determined that the image is not good. Here, if the calibration is good, the process proceeds to step S44, and if not, the process proceeds to step S45.
[0075]
In step S44, it is determined whether the photographing has been completed. Here, since the calibration is good in the determination in step S43, a signal indicating the end of the photographing is generated.
[0076]
In step S45, a determination is made to continue shooting. Here, since the calibration is not good in the determination in step S43, a signal to continue shooting is generated.
[0077]
In step S46, the determination result is transmitted to the camera 2. Specifically, the signal of the end of imaging generated in step S44 or the signal of continuation of imaging generated in step S45 is transmitted.
[0078]
The determination result transmitted from the image processing device 3B is received by the camera 2 (step S33).
[0079]
In step S34, it is determined whether or not the result of the determination received in step S33 is the end of shooting. Here, when the notification of the end of the photographing is received from the image processing apparatus 3B, that is, when the signal of the end of the photographing is received, for example, the end of the photographing is displayed on the display unit 20 of the camera 2 and the photographing operation is ended. Then, the process ends. If a notification indicating that the shooting has not been completed is received, that is, if a signal indicating that the shooting is to be continued is received, for example, a display indicating that the shooting is to be continued is displayed on the display unit 20 of the camera 2 and the shooting operation is continued, and the process returns to step S31. .
[0080]
Through the operation of the data processing system 1B, the image processing apparatus 3B automatically determines the end of shooting and notifies the camera 2 each time shooting is performed, so that unnecessary shooting can be prevented and calibration can be performed efficiently.
[0081]
<Third embodiment>
FIG. 8 is a schematic diagram illustrating a main configuration of a data processing system 1C according to the third embodiment of the present invention.
[0082]
The appearance of the data processing system 1C is the same as that of the data processing system 1A of the first embodiment shown in FIG.
[0083]
The functional blocks of the data processing system 1C have a configuration similar to that of the data processing system 1A of the first embodiment shown in FIG. Is different.
[0084]
That is, the image processing apparatus 3C according to the third embodiment has a photographing support program DP for performing a photographing support operation when generating a panoramic image by combining images acquired by the camera 2. The generation of the panoramic image will be briefly described below.
[0085]
<Generation of panoramic image>
In the generation of a panoramic image in the data processing system 1C, as shown in FIG. 8, for example, while changing the shooting angle of the camera 2 to the rotation direction Rt, for example, the shooting area P1 (parallel hatched portion) or the shooting area shown in FIG. P2 (parallel hatched portion) is sequentially photographed. Note that the photographing region P1 and the photographing region P2 represent images of the region to be photographed.
[0086]
Then, the end portions of the adjacent images, for example, the end portion Ep of the image of the photographing region P1 and the end portion Ep of the image of the photographing region P2 are attached to form an image. Here, based on the position / posture information of the camera 2, an image combining process in which the three-dimensional position of each image is reflected is performed.
[0087]
By repeating the above image composition, a 360-degree panoramic image Pt (cylindrical portion shown in FIG. 8) around the camera 2 can be generated.
[0088]
In the panorama image generation process, as in the first and second embodiments, new image data must be obtained when the panorama image Pt cannot be generated. More specifically, if the panorama image Pt cannot be completed, that is, if there is an unphotographed area in the middle of sequentially synthesizing the image P1 and the image P2, additional photography is required.
[0089]
On the other hand, if a plurality of image data can be acquired without excess or deficiency, the panoramic image Pt can be generated, but it is not efficient to acquire the image data more than necessary.
[0090]
As described above, even in the generation of a panoramic image, as in the first and second embodiments, if it is possible to determine whether or not necessary and sufficient image data has been obtained and notify the photographer of the photographing support, it is useless. This makes it possible to omit unnecessary shooting. The photographing support operation in the data processing system 1C will be described below.
[0091]
<Operation of Data Processing System 1C>
FIG. 9 is a flowchart illustrating a basic operation in the data processing system 1C. FIG. 9A shows the processing of the camera 2, and FIG. 9B shows the processing of the image processing device 3C. The processing of the image processing device 3C is performed by the control unit 33 executing the data processing program DP.
[0092]
In step S51, the image data photographed by the photographer is generated by the camera 2.
[0093]
In step S52, the image data generated in step S51 is transmitted via the communication I / F 24. When transmitting the image data, the position / posture information of the camera 2 detected by the position / posture detection unit 25 is also transmitted.
[0094]
The image data transmitted from the camera 2 as described above is received by the image processing device 3C (Step S61).
[0095]
In step S62, it is determined whether the image data received in step S61 can be combined to generate a panoramic image Pt of the entire circumference. That is, when one shooting operation is performed by the camera 2 in step S51, a fixed image data is obtained based on the image data obtained by the shooting operation and the plurality of image data obtained before the shooting operation. It is determined whether or not predetermined data that satisfies the condition, that is, data of the panoramic image Pt of the entire periphery can be generated.
[0096]
If it is determined in step S62 that all the areas have been photographed and that an all-around panoramic image can be generated, the process proceeds to step S63; otherwise, the process proceeds to step S64.
[0097]
In step S63, it is determined whether the photographing has been completed. Here, since it is in a state in which an all-around panoramic image can be generated by the determination in step S62, a signal indicating that shooting has been completed is generated.
[0098]
In step S64, a determination is made to continue shooting. Here, since the panoramic image of the entire circumference cannot be generated yet in the determination in step S62, a signal to continue shooting is generated.
[0099]
In the step S65, the determination result is transmitted to the camera 2. Specifically, the signal of the end of imaging generated in step S63 or the signal of continuation of imaging generated in step S64 is transmitted.
[0100]
The determination result transmitted from the image processing device 3C is received by the camera 2 (Step S53).
[0101]
In step S54, it is determined whether or not the result of the determination received in step S53 is the end of shooting. Here, when the notification of the end of the photographing is received from the image processing apparatus 3C, that is, when the signal of the end of the photographing is received, for example, the display of the end of the photographing is displayed on the display unit 20 of the camera 2, and the photographing operation is ended. Then, the process ends. Then, the image processing device 3C generates a panoramic image of the entire periphery based on the received plurality of image data.
[0102]
On the other hand, when a notification indicating that shooting is not completed is received from the image processing apparatus 3C, that is, when a signal indicating that shooting is to be continued is received, for example, a message indicating that shooting is to be continued is displayed on the display unit 20 of the camera 2 so that shooting is continued. The process returns to step S51.
[0103]
By the operation of the data processing system 1C described above, the image processing apparatus 3C automatically determines the end of shooting and notifies the camera 2 every time shooting is performed, so that useless shooting can be prevented and a panoramic image can be generated efficiently. .
[0104]
<Modification>
In each of the above embodiments, it is not essential that data communication between the camera and the image processing apparatus is performed via the cable CB, but may be performed wirelessly.
[0105]
In the cameras in the above embodiments, it is not essential to shoot a still image, but may shoot a moving image. When shooting a moving image, a still image is cut out at regular intervals from continuous image data to generate a plurality of image data.
[0106]
In each of the above embodiments, it is not essential that the image processing apparatus determines the end of shooting, and the camera may determine the end of shooting. In this case, the camera needs to be configured as a camera that can execute the above-described shooting support program DP.
[0107]
◎ In each of the above embodiments, it is not essential to move the camera to the object SB fixed in the three-dimensional space to take an image, and to fix the object SB in the three-dimensional space while rotating the object on the rotary table. The camera may be used for shooting.
[0108]
In this case, a configuration in which a plurality of image data is automatically acquired by the camera without performing the shooting operation by the photographer is also possible. However, even in such a configuration, the image processing apparatus notifies the camera of the end of shooting. By notifying, it is possible to prevent useless photographing as in manual photographing by the photographer.
[0109]
【The invention's effect】
As described above, according to the first to fifth aspects of the present invention, when a shooting operation is performed by the shooting unit, the image data obtained by the shooting operation and the plurality of data obtained before the shooting operation are obtained. It is determined whether or not predetermined data can be generated based on the image data. If it is determined that the predetermined data can be generated, the end of shooting by the shooting unit is notified. As a result, unnecessary data can be prevented and predetermined data can be efficiently generated.
[0110]
In particular, according to the second aspect of the present invention, since three-dimensional data processing is performed based on a plurality of image data to generate three-dimensional data, appropriate three-dimensional data can be efficiently generated.
[0111]
According to the third aspect of the present invention, since the calibration processing relating to the three-dimensional data processing is performed based on the plurality of image data to generate the unique data of the photographing unit, appropriate unique data can be efficiently generated.
[0112]
Further, in the invention according to the fourth aspect, since image synthesis processing is performed based on a plurality of image data to generate data of a panoramic image, an appropriate panoramic image can be efficiently generated.
[Brief description of the drawings]
FIG. 1 is a schematic diagram showing a main configuration of a data processing system 1A according to a first embodiment of the present invention.
FIG. 2 is a diagram showing functional blocks of a data processing system 1A.
FIG. 3 is a diagram for explaining modeling.
FIG. 4 is a flowchart illustrating a basic operation in the data processing system 1A.
FIG. 5 is a flowchart illustrating a calibration operation in a data processing system 1B according to a second embodiment of the present invention.
FIG. 6 is a diagram for explaining the operation of calibration.
FIG. 7 is a flowchart illustrating a basic operation in the data processing system 1B.
FIG. 8 is a schematic diagram showing a main configuration of a data processing system 1C according to a third embodiment of the present invention.
FIG. 9 is a flowchart illustrating a basic operation in the data processing system 1C.
[Explanation of symbols]
1A, 1B, 1C data processing system
2 Camera
3A, 3B, 3C image processing device
20 Display
25 Position / posture detector
33 control unit
DP shooting support program
Pt panoramic image

Claims (5)

An image processing device capable of processing image data,
(A) processing means for generating predetermined data that satisfies certain conditions based on a plurality of image data sequentially captured and obtained by the imaging means;
(B) When one photographing operation is performed by the photographing means, one image data acquired by the one photographing operation and a plurality of image data acquired before the one photographing operation are combined. Determining means for determining whether or not the predetermined data can be generated,
(C) a notifying unit for notifying the end of the image capturing by the image capturing unit when the determining unit determines that the predetermined data can be generated;
An image processing apparatus comprising: The image processing apparatus according to claim 1,
The processing means includes:
(B-1) means for performing three-dimensional data processing based on the plurality of image data to generate three-dimensional data;
An image processing apparatus comprising: The image processing device according to claim 2,
The processing means includes:
(B-2) means for performing calibration processing relating to the three-dimensional data processing based on the plurality of image data to generate unique data of the imaging means;
An image processing apparatus comprising: The image processing apparatus according to claim 1, wherein
The processing means includes:
(B-3) means for performing panorama image data by performing image synthesis processing based on the plurality of image data;
An image processing apparatus comprising: A non-transitory computer-readable storage medium storing a program executed by a computer built in an image processing apparatus to cause the image processing apparatus to function as the image processing apparatus according to claim 1.

Images