JP4642375B2 - Image processing device, image processing / editing device, image file playback device, image processing method, image processing / editing method, and image file playback method - Google Patents

Description

  The present invention relates to an image processing device, an image processing / editing device, an image file playback device, an image processing method, an image processing / editing method, and an image file playback method suitable for capturing and editing stereo images.

  There are a wide variety of proposals for a method of photographing and recording an image including three-dimensional information and reproducing and observing the image. Among them, the so-called binocular stereo system that records two images with parallax corresponding to the viewpoints of the left and right eyes and presents them to the left and right eyes is the simplest and cheapest configuration. Since it is large, it has been used from the old days to today.

  In this binocular stereo system, although there is a restriction that only one person can observe at the same time for so-called personal use, the system using stereo pair images, which is the most basic and classical method, is extremely inexpensive. Furthermore, as a method for observing a clear image, it is still widely used today.

  And as a stereo imaging device using a stereo adapter, there exists a thing indicated by patent documents 1, for example. In Patent Literature 1, a stereo adapter detection unit that detects the mounting of a stereo adapter in a system controller, and an automatic exposure (AE) control unit that calculates a photometric information necessary for exposure control by analyzing a subject image signal related to a photometric area And a photometry area setting unit for setting the photometry area as described above, and the photometry area setting unit has a function for setting different photometry areas in the normal shooting mode and the stereo shooting mode. A technique for setting an optimum photometric area for each of the mode and the stereo shooting mode is disclosed.

  As described above, the binocular stereo method employs a method of projecting two images of the same subject (hereinafter referred to as a monocular image) that are shifted from each other in accordance with the parallax on the left and right sides of one image frame. That is, in the binocular stereo method, in order to obtain a stereoscopic effect from a single image in which monocular images are arranged on the left and right (hereinafter referred to as an integrated image), the left and right monocular images in the integrated image are respectively displayed with the right eye or the left eye. Observe. By fusing two images observed with the left and right eyes as one image, an image having a stereoscopic effect can be recognized. This recognition image (hereinafter referred to as a fused image) causes each part to have a perspective according to the amount of displacement of each part of the left and right monocular images.

  By the way, as the image data handled by the electronic camera, the data format of the Exif standard is widespread. The Exif standard conforms to the JPEG standard file (JPEG file) format. That is, the image data itself is treated as JPEG standard image data, and Exif data (metadata) is arranged in the header portion of the JPEG file. Exif standard metadata is described in the TIFF format and includes information such as a shooting date and a thumbnail.

In this metadata, information about the stereo image can be described. For example, the left and right monocular images captured using a stereo adapter may actually have vignetting at the boundary or a shift in the imaging position. In order to correct this vignetting, misalignment, etc., a predetermined portion in each of the left and right regions is trimmed to set an effective monocular image range (hereinafter referred to as an image frame). Since the range of the image frame affects the sense of depth of the stereoscopic image, it is necessary to appropriately set the image frame when processing / editing the stereoscopic image. By describing information related to trimming in the metadata, it is possible to appropriately set the image frame and to give a proper sense of depth to the fused image.
JP 2002-218506 A

  By the way, it is conceivable to process / edit stereoscopic images using image processing / editing software. In this case, the processing / editing software (hereinafter referred to as 3D-compatible software) corresponding to a stereoscopic image is not always processed / edited, and the processing / editing software (hereinafter referred to as 3D non-compatible) not compatible with a stereoscopic image Processing / editing may be performed using compatible software).

  General processing / editing software supports the Exif standard. For this reason, even when a 3D non-compatible software processes and edits a stereoscopic image, image data can be processed without destroying the metadata portion. However, in the 3D non-compatible software, the metadata portion regarding the stereoscopic image cannot be edited.

  Therefore, when image data is processed / edited with 3D non-compatible software, the corresponding metadata is not edited and remains unchanged even though the stereoscopic image is processed / edited. . That is, in this case, there is a problem in that the correlation between the stereoscopic image and the metadata is shifted.

  FIG. 9 is an explanatory diagram for explaining the problem in this case. FIGS. 9A to 9D show a case where an integrated image is trimmed with 3D-compatible software, and FIGS. 9E to 9H are diagrams where the integrated image is trimmed with non-3D-compatible software, and then 3D-compatible software is used. An example of trimming is shown.

  FIGS. 9A and 9E show integrated images in which monocular images of the same subject (person) are arranged on the left and right. As described above, this integrated image is subjected to trimming for setting an effective image frame used for the fused image for vignetting and misalignment correction. In the Exif standard, information about an appropriate trimming position is described in the metadata in advance.

  The 3D-compatible software can automatically perform trimming at the optimum trimming position by using this metadata. FIG. 9B shows the optimum trimming position by a broken line. FIG. 9C shows a cut-out image by 3D-compatible software. At the time of clipping by 3D software, the metadata is automatically corrected corresponding to the image after clipping. That is, information such as the optimum trimming position and the maximum pop-out amount that determines the maximum value of the depth feeling of the image is also updated.

  Here, it is assumed that the image is further cut out by 3D-compatible software. In this case, the optimum trimming position is updated corresponding to the image of FIG. 9C, and the cut-out image by the 3D software is as shown in FIG.

  On the other hand, it is assumed that the integrated image shown in FIG. In this case, generally, it is conceivable that clipping is performed at the trimming position indicated by the one-dot chain line in FIG. FIG. 9G shows a cut-out image by 3D non-compatible software. In this case, the metadata is not corrected.

  Here, it is assumed that the 3D-compatible software cuts out from the image in FIG. In this case, the 3D-compatible software performs automatic trimming with reference to the metadata. In other words, as shown in FIG. 9H, clipping is performed at the optimum trimming position (broken line) set in the metadata. That is, in this case, the subject protrudes from the cutout range, and an appropriate integrated image cannot be obtained from the cutout image.

  Further, when the image cut out at the trimming position indicated by the broken line in FIG. 9H is reproduced and output, there is a problem that the sense of depth of the fused image is extremely large and exceeds the set maximum pop-out amount.

  The present invention provides an image processing apparatus, an image processing / editing apparatus, and an image file reproduction capable of enabling proper recognition of a fused image even when a stereoscopic image is processed by a processing / editing machine or software that does not support stereoscopic images. An object is to provide an apparatus, an image processing method, an image processing / editing method, and an image file reproduction method.

An image processing apparatus according to claim 1 of the present invention relates to stereo image data generating means for generating stereo image data based on a plurality of monocular images of the same subject obtained with a predetermined parallax, and the stereo image data Accompanying data generating means for generating accompanying data, stereo image size information generating means for generating information on the image size of the stereo image data, stereo image data generated by the stereo image data generating means, and the accompanying data generation Image file generating means for generating an image file by synthesizing the accompanying data generated by the means, and adding information relating to the image size generated by the stereo image size information generating means to the inside and outside of the accompanying data, respectively. To create an image file by converting it to the specified file format. And a file generating unit, which has,
The image processing / editing apparatus according to claim 2 of the present invention combines stereo image data based on a plurality of monocular images of the same subject obtained with a predetermined parallax and accompanying data related to the stereo image data, In addition, information relating to the image size of the stereo image data is further added to the inside and outside of the accompanying data, and image processing / editing is performed based on the stereo image data in the image file generated by conversion into a predetermined file format. Image processing / editing means, accompanying data update means for updating the accompanying data based on the contents of the image processing / editing with respect to the accompanying data before the image processing / editing is performed, and inside / outside of the accompanying data The information about the image size of the stereo image data before the image processing / editing added to each is added. Stereo image size information updating means for updating information on the respective image sizes based on the contents of image processing / editing, stereo image data processed / edited by the image processing / editing means, and the associated data update An image file generation means for generating an image file by combining the accompanying data updated by the means, and adding data relating to the image size updated by the stereo image size information updating means to the inside and outside of the accompanying data, respectively. And an image file generating means for generating an image file by converting into a predetermined file format,
An image file reproduction device according to claim 3 of the present invention combines stereo image data based on a plurality of monocular images of the same subject obtained with a predetermined parallax and accompanying data related to the stereo image data, and Information relating to the image size of the stereo image data is added to the inside and outside of the accompanying data, respectively, and playback means for playing back the image file generated by converting into the predetermined file format; and the playback means played back by the playback means Determination means for determining a match between the information about the image size in the accompanying data and the information about the image size outside the accompanying data, and when the determination means determines that the information does not match, the reproduction operation by the reproduction means is continued. Control means to prohibit.

  In the first aspect of the present invention, the accompanying data generation means generates accompanying data related to image data of a stereo image based on a plurality of monocular images of the same subject obtained with a predetermined parallax. The stereo image size information generating means sets information on the image size of the stereo image. The image file generation unit combines the image data and the accompanying data, adds information on the image size of the stereo image to the inside and outside of the accompanying data, converts the data to a predetermined file format, and generates an image file.

  According to claim 2 of the present invention, the image processing / editing means is provided with image data of a stereo image based on a plurality of monocular images of the same subject obtained with a predetermined parallax, and this image data is processed / edited To do. The accompanying data related to the image data before processing / editing is updated based on the image data after processing / editing by the accompanying data update means. In addition, the stereo image size information update unit sets information on the image size of the stereo image after processing and editing. An image file of a predetermined file format is generated based on the image data, the accompanying data, and information on the image size of the stereo image arranged inside and outside the accompanying data.

  According to a third aspect of the present invention, the determination means determines whether or not there is a correlation between the image data and the accompanying data based on the image size information of the stereo image inside and outside the accompanying data. The reproduction means performs a predetermined reproduction process using the accompanying data on the image data separated from the image file. In this case, if the determination unit determines that there is no correlation between the image data and the accompanying data, the continuation of the reproduction operation by the reproduction unit is prohibited.

  Note that the present invention relating to an apparatus is also established as an invention relating to a method.

  According to the present invention, even when a stereoscopic image is processed by a processing / editing machine or software that does not support a stereoscopic image, it is possible to enable proper recognition of a fused image.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing an electronic camera in which an image processing apparatus according to a first embodiment of the present invention is incorporated. This embodiment is applied to an image file recording apparatus.

(First embodiment)
In FIG. 1, the electronic camera includes a camera body 1, a lens unit 5 having a lens barrel, and a stereo adapter 10 for taking a stereo image. In the electronic camera, a mirror type stereo adapter 10 is detachably attached to the lens unit 5. The stereo adapter 10 has a configuration in which mirrors 11 and 12 are arranged at positions separated by a parallax, and mirrors 13 and 14 for guiding light reflected by the mirrors 11 and 12 to the camera side are arranged. ing.

  Light that has passed through the mirrors 11 and 13 and the mirrors 12 and 14 in the stereo adapter 10 is guided to the exposure control mechanism 22 and further to the half mirror 31 in the camera body 1 through the photographing lens group 21 in the lens unit 5. It is burned.

  The lens unit 5 includes a photographic lens group 21, an exposure control mechanism 22, a lens drive mechanism 23, a lens driver 24, and an exposure control driver 25.

  The photographing lens group 21 is a main imaging optical system capable of normal monocular imaging (monocular imaging) when the stereo adapter 10 is not attached, and is driven by the lens driving mechanism 23 to adjust focusing and zooming. To be done. The lens driving mechanism 23 is controlled by the lens driver 24. The exposure control mechanism 22 controls the stop and shutter device (not shown) of the photographic lens group 21. The exposure control mechanism 22 is controlled by the exposure control driver 25.

  The light guided from the lens unit 5 to the camera body 1 passes through the half mirror 31 and is guided to the CCD color imaging device 34 through the low-pass and infrared cut filter system 32 so as to be imaged. It has become. The CCD color image sensor 34 is driven and controlled by a CCD driver 35 to convert an optical image of a subject into an electrical signal. As the CCD color imaging device 34, for example, an interline type of a vertical overflow drain structure and a progressive (sequential) scanning type are employed.

  Light incident on the left-eye viewing mirror 11 of the stereo adapter 10 is imaged on a region L of an imaging surface (not shown) of the CCD color imaging device 34 via the mirror 13 and the imaging lens group 21. Similarly, the light incident on the right-eye viewing mirror 12 is focused on a region R of an imaging surface (not shown) of the CCD color imaging device 34 via the mirror 14 and the imaging lens group 21.

  The signal photoelectrically converted by the CCD color imaging device 34 is a digital process circuit for performing color signal generation processing, matrix conversion processing, and other various digital processing via a preprocessing circuit 36 including an A / D converter and the like. 39. In the digital process circuit 39, color image data is generated by processing the digitized image signal.

  An LCD display 40 is connected to the digital process circuit 39, and a memory card 42 such as a CF (Compact Flash Memory Card) or smart media is connected via a card interface (IF) 41. The LCD display unit 40 performs display based on color image data, and the memory card 42 stores color image data.

  The memory card 42 can be loaded into an external personal computer 60. The image recorded on the memory card 42 can be displayed on the personal computer 60, image processing, and the like. Furthermore, the image recorded on the memory card 42 can be printed out by a printer (not shown).

  The half mirror 31 is configured such that a part of an incident subject image is reflected, and guides reflected light to the AF sensor module 45. The AF sensor module 45 is for performing focus detection based on a light beam incident through the photographing lens group 21. The AF sensor module 45 has a pupil-division separator lens 46 and an AF sensor 47 composed of a line sensor.

  A system controller 50 constituted by a CPU or the like is for comprehensively controlling each part in the camera body 1 and the lens unit 5. The system controller 50 includes the above-described lens driver 24, exposure control driver 25, CCD driver 35, preprocess circuit 36, digital process circuit 39, AF sensor module 45, operation switch unit 52, operation A display unit 53, a nonvolatile memory (EEPROM) 51, and a stereo changeover switch (SW) 54 are connected.

  The operation switch unit 52 includes various switches such as a release switch and a shooting mode setting. The operation display unit 53 is a display unit for displaying the operation state and mode state of the camera.

  The EEPROM 51 is a memory for storing various setting information. The stereo changeover switch 54 is a changeover switch for switching modes when the stereo adapter 10 is attached to the lens unit 5. Here, the switching of the shooting mode is performed by operating the stereo switch 54, but the present invention is not limited to this. For example, a detection function may be provided in the stereo adapter 10 so that the shooting mode is automatically switched.

  The system controller 50 controls the exposure control mechanism 22 and the CCD driver 35 to drive the CCD color imaging device 34 to perform exposure (charge accumulation) and signal readout. The system controller 50 supplies the output of the CCD 34 to the digital process circuit 39 via the preprocess circuit 36 to perform various signal processing, and records it on the memory card 42 via the card interface 41.

  The strobe 57 emits flash light and is controlled by the system controller 50 via the exposure control driver 25 in the lens unit 5.

  The system controller 50 further includes an exposure control unit 50d and a photometric area setting unit 50e. The exposure control unit 50d is for calculating the exposure information necessary for exposure control by analyzing the subject image signal relating to the photometry area. The photometric area setting unit 50e sets the photometric area for the exposure control unit 50d.

  In the present embodiment, the system controller 50 further includes a metadata generation unit 50a, a stereo image generation unit 50b, and an image file generation unit 50c.

  The metadata generation unit 50a serving as the accompanying data generation unit generates various accompanying data (hereinafter referred to as metadata) related to the captured image and supplies the generated data to the image file generation unit 50c. For example, the metadata generation unit 50a generates shooting date information and various information (stereoscopic image information) related to a stereoscopic image when the captured image is a stereoscopic image. As the stereoscopic image information, for example, there is an optimum trimming position which is trimming information. The information of the optimal trimming position is information indicating an effective image frame of each monocular image, and indicates an area used for recognition of the fused image at the time of reproduction. Further, as the stereoscopic image information, there is information on the maximum pop-out amount that defines the maximum value of the depth feeling of the stereoscopic image.

  The stereo image generation unit 50b generates a stereo image based on a plurality of input monocular images. For example, when a single image in which a plurality of monocular images are arranged on the imaging surface of the CCD 34 is formed, the stereo image generation unit 50b generates a stereo image using the input image as it is. When a plurality of monocular images are captured and input separately, for example, the stereo image generation unit 50b generates a stereo image based on each input monocular image.

  The image file generation unit 50c can convert the stereo image generated by the stereo image generation unit 50b into an electronic image file of a predetermined format and output it. When the stereo image is an integrated image, the image file generation unit 50c converts one integrated image into an image format of, for example, the JPEG standard or the TIFF standard to form a single image file. In addition, when the stereo image is configured by a plurality of monocular images, the image file generation unit 50c configures one image file by assigning each monocular image to each page in the TIFF format, for example.

  In this case, the image file generation unit 50c compresses the stereo image as necessary, and converts the stereo image into a digital image file having a predetermined format to which accompanying data (metadata) is added.

  FIG. 2 is a block diagram showing a specific configuration of the image file generation unit 50c in FIG. FIG. 3 is an explanatory diagram showing an example of an image file generated by the image file generation unit 50c.

  Image data of a stereo image is input to the compression unit 71 of the image file generation unit 50c. The compression unit 71 performs a predetermined compression process on the image data of the stereo image and outputs it to the data synthesis unit 72. Metadata is also input to the data synthesizer 72, and the data synthesizer 72 synthesizes image data and metadata.

  In the present embodiment, the image file generation unit 50 c includes a metadata size setting unit 73 and an image file size setting unit 74. The metadata size setting unit 73 can output information on the image size (3D image size) (number of pixels) of the stereoscopic image to the data synthesis unit 72. The image file size setting unit 74 can output information on the image size (number of pixels) of the image based on the image data to the data synthesis unit 72. Note that the metadata size setting unit 73 and the image file size setting unit 74 set information on the image size of an image based on the same image data. Therefore, when the image data is data about a stereoscopic image. The outputs of the setting units 73 and 74 are information having the same image size.

  The data synthesizing unit 72 synthesizes each input data and gives it to the format unit 75. The format unit 75 arranges each input data according to a predetermined image format and outputs it as one image file.

  As shown in FIG. 3, the image file has a header part and an image data part. In the image data portion, the image data compressed by the compression unit 71 is arranged. In the header portion, information on the file name and image size of the image file is arranged, and metadata is also arranged. The image size information is set by the image file size setting unit 74.

  The metadata includes metadata identification information, information on the shooting date and time of an image, and information on a stereoscopic image. Note that the shooting date / time information is information included in metadata generated by the metadata generation unit 50 a and input to the data synthesis unit 72.

  The stereoscopic image information includes various pieces of information regarding the stereoscopic image when the image data arranged in the image data portion is a stereoscopic image. Information on the optimal trimming position and information on the maximum pop-out amount are included in the metadata generated by the metadata generation unit 50 a and input to the data composition unit 72. The information on the 3D image size is information set by the metadata size setting unit 73 and indicates the image size of the stereoscopic image based on the image data. Here, as described above, the 3D image size information recorded as metadata and the image size information recorded in the header portion of the image file have the same contents.

  The file format in FIG. 3 is an example of an image file. In FIG. 3, the example in which the 3D image size is arranged as a part of the stereoscopic image information has been described. However, the file format is arranged in another area in the metadata. Also good.

  In the present embodiment, the controller 50 is described as being 3D-compatible. However, if the controller 50 is not 3D-compatible, the description area of the stereoscopic image information in the metadata area is not included. For example, a predetermined initial value is set or no information is described.

  Next, the operation of the embodiment thus configured will be described with reference to FIG. FIG. 4 is a flowchart showing an operation flow of the image file generation unit 50c.

  Now, it is assumed that the stereo shooting mode is instructed by the stereo changeover switch 54. The subject optical image incident through the stereo adapter 10 forms an image on the imaging surface of the CCD color imaging device 34 through the photographing lens group 21, the exposure control mechanism 22, the half mirror 31, and the filter system 32. The CCD color imaging device 34 outputs data of one image including the left and right monocular images L and R. An image signal from the CCD color image sensor 34 is input to the controller 50 via the preprocess circuit 36.

  The stereo image generation unit 50b generates an integrated image in which monocular images are arranged on the left and right based on the input image signal. The metadata generation unit 50a generates shooting date / time information and stereoscopic image information related to the generated integrated image.

  In step S1 in FIG. 4, the image file generation unit 50c performs a predetermined compression process on the generated integrated image (stereo image). In the next step S2, the metadata size setting unit 73 in the image file generation unit 50c generates image size information (3D image size) of the stereoscopic image stored in the image data unit. In step S3, the image file size setting unit 74 in the image file generation unit 50c generates information on the image size of the image stored in the image data unit. That is, in this case, the 3D image size set by the metadata size setting unit 73 and the image size set by the image file size setting unit 74 have the same value.

  The image file generation unit 50c combines the stereo image, metadata, 3D image size and image size information by the data combining unit 72 (step S4), and the format unit 75 generates an image file having the data format shown in FIG. Generate (step S5).

  The image file generated by the image file generation unit 50 c is given to the digital process circuit 39. The digital process circuit 39 can display the integrated image on the display screen of the LCD 40 based on the input electronic image file. In addition, the digital process circuit 39 can provide the input electronic image file to the memory card 42 via the card IF 41 for recording.

  As described above, in the present embodiment, the image file generated by the image file generation unit includes the 3D image size of the same image in addition to the image size information of the image based on the image data in the image file. , Placed in the metadata. The information on the image size of the stereoscopic image is arranged separately from the information on the image size of the header portion in the image file, and these pieces of information can be managed separately. When an image file is created (updated) by a device or software that does not support 3D, the 3D image size of the metadata related to stereoscopic image information is not generated or updated. Thus, it can be determined whether or not the image file has been created (updated) by a 3D incompatible device or software.

  Further, in the above-described embodiment, the example in which the image file is generated by the image processing apparatus including the image file generation unit has been described. However, the image file generation unit may be realized by software having the same function. it is obvious. That is, an image file similar to that in FIG. 2 can be generated by using a computer capable of executing a program having the same function as that in the flowchart of FIG.

  When the image file generation unit 50c is configured by software, for example, the image file size setting unit 74 is configured by a computer operation system or the like, and the image size in the image file is automatically set when the file is created (updated). It is common to be described in a manual manner.

  In the first embodiment, the example applied to the electronic camera has been described. However, the first embodiment can also be applied to a single image processing apparatus that processes an image captured by the electronic camera. A similar function can be achieved by a program such as a personal computer to be processed.

  In the above embodiment, an example has been described in which a single image in which monocular images are arranged on the left and right sides is acquired by a camera, and a stereo image based on this image is generated. For example, two cameras separately It is obvious that each monocular image obtained by photographing can be input separately and can be applied to an apparatus that generates one stereo image from each monocular image.

(Second Embodiment)
FIG. 5 is a block diagram showing an image processing / editing apparatus according to the second embodiment of the present invention, and FIG. 6 is a flowchart showing a processing flow of image file generation. The present embodiment is applied to an image file editing apparatus. Note that this embodiment can also be realized by software having the same function as that of the apparatus shown in FIG.

  The decoding unit 81 receives an image file having the data format shown in FIG. The decoding unit 81 decodes the input image file and extracts various data included in the image file. The decoding unit 81 gives the image data in the image file to the image expansion unit 82 and gives the metadata to the metadata processing unit 84.

  The image decompression unit 82 decompresses input image data to obtain image data before compression. This image data is given to the image processing / editing unit 83.

  The image processing / editing unit 83 processes / edits the input image data in accordance with the user's processing / editing operation (step S11), and the processed / edited image data is sent to the image file generating unit 86. Output. Note that the image processing / editing unit 83 can be used for processing / editing processing with reference to metadata about image data to be processed / edited.

  The metadata processing unit 84 updates the metadata based on the processing / editing process of the image processing / editing unit 83. For example, when the processing / editing process of the image processing / editing unit 83 is a trimming process for a stereoscopic image, the metadata processing unit 84 indicates an optimal trimming position indicating an optimal cutout range corresponding to the trimming process. And information on the maximum pop-up amount are updated.

  The image file generation unit 86 combines the image data from the image processing / editing unit 83 and the metadata from the metadata processing unit 84 to generate an image file in the file format of FIG. The size update unit 87 in the image file generation unit 86 has the same functions as the metadata size setting unit 73 and the image file size setting unit 74 in FIG. 2, and is processed and edited by the image processing / editing unit 83. Information on the image size of the image is generated (step S12). The image file generation unit 86 rewrites the image size information (see FIG. 3) in the image file in the file format with the information generated by the size update unit 87. Further, the image file generation unit 86 rewrites the 3D image size information (see FIG. 3) in the metadata in the file format with the information generated by the size update unit 87 (step S13). In this way, the image file generation unit 86 generates and outputs an image file in which the image size in the image file and the 3D image size in the metadata are changed to the same information (step S14).

  As described above, in the present embodiment, the metadata is updated along with the processing / editing of the image, and the information on the image size changed along with the processing / editing is changed to the image size and metadata in the image file. Write to the middle 3D image size. As a result, as long as the image is processed and edited using the editing apparatus according to the present embodiment, the image size in the image file and the information on the 3D image size in the metadata match.

(Third embodiment)
FIG. 7 is a block diagram showing an image file reproducing apparatus according to the third embodiment of the present invention, and FIG. 8 is a flowchart showing a processing flow of the image file. This embodiment is applied to an image file playback apparatus. Note that this embodiment can also be realized by software having the same function as the apparatus of FIG.

  The decoding unit 91 receives an image file having the file format shown in FIG. The decoding unit 91 decodes the input image file and takes out various data included in the image file. The decoding unit 91 gives the image data in the image file to the image expansion unit 92, and gives the metadata to the switching unit 96, the metadata invalidation unit 95, and the size determination unit 94.

  The image decompression unit 92 decompresses input image data to obtain image data before compression. The image data is given to the reproduction processing unit 93.

  The size determination unit 94 acquires information on the image size (see FIG. 3) in the image file and information on the 3D image size in the metadata (step S21). The size determination unit 94 determines whether or not the image size in the acquired image file matches the 3D image size in the metadata (step S22). The size determination unit 94 outputs the determination result to the switching unit 96 and the warning unit 97.

  The metadata invalidation unit 95 invalidates the input metadata (step S24). For example, the metadata invalidation unit 95 invalidates the metadata regarding the stereoscopic image by deleting the parameter or setting it to an initial value.

  Based on the determination result of the size determination unit 94, the switching unit 96 selectively supplies either the metadata from the decoding unit 91 or the metadata from the metadata invalidation unit 95 to the reproduction processing unit 93. For example, when the switching unit 96 is given a determination result indicating that the image sizes match, the switching unit 96 gives the metadata from the decoding unit 91 to the reproduction processing unit 93 to indicate that the image sizes do not match. When the determination result is given, the reproduction process is prohibited (step S24).

  The reproduction processing unit 93 performs predetermined reproduction processing on the image data using the metadata from the switching unit 96 (step S25). In addition, when the determination result indicating that the image sizes do not match is given from the size determination unit 94, the reproduction processing unit 93 displays the metadata information set by the metadata invalidation unit 95 as the reproduction processing unit 93. You may give to.

  When the determination result from the size determination unit 94 indicates that the image sizes do not match, the warning unit 97 issues a warning to the user, for example, by a warning display or the like.

  Thus, in the present embodiment, when the image sizes do not match for an image file whose image size is described independently in the file header and metadata, the image data and the metadata Therefore, the reproduction process is prohibited or the reproduction process using the invalidated metadata is executed. As a result, when an image is processed / edited by a device or software that does not support 3D, it is possible to prevent the reproduction process from being performed using metadata that is not correlated with the image data.

  In each of the above-described embodiments, the stereo system is described as a binocular system corresponding to both the left and right eyes. However, the present invention can be similarly applied to a general multi-lens stereo system having three or more eyes. Of course.

  Furthermore, it is clear that the present invention can be applied to moving images.

1 is a block diagram showing an electronic camera in which an image processing apparatus according to a first embodiment of the present invention is incorporated. The block diagram which shows the specific structure of the image file generation part 50c in FIG. Explanatory drawing which shows an example of the image file produced | generated by the image file production | generation part 50c. The flowchart which shows the operation | movement flow of the image file production | generation part 50c. FIG. 5 is a block diagram showing an image processing / editing apparatus according to the second embodiment of the present invention. The flowchart which shows the processing flow of image file generation. The block diagram which shows the image file reproducing | regenerating apparatus which concerns on the 3rd Embodiment of this invention. The flowchart which shows the processing flow of an image file. Explanatory drawing for demonstrating the problem of background art.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Camera body, 5 ... Lens unit, 10 ... Stereo adapter, 11, 12, 13, 14 ... Mirror, 21 ... Shooting lens group, 22 ... Exposure control mechanism, 25 ... Exposure control driver, 31 ... Half mirror, 32 ... Filters 34... CCD color imaging device 35. CCD driver 36. Preprocess circuit 39. Digital process circuit 40. LCD display unit 41 Card interface (IF) 42 Memory card 45 AF sensor module 46 ... Separator lens, 47 ... AF sensor, 50 ... System controller (CPU), 50a ... Metadata generation unit, 50b ... Stereo image generation unit, 50c ... Image file generation unit, 50d ... Exposure control unit, 50e ... Photometry area Setting unit, 51 ... non-volatile memory (EEPROM), 52 ... operation switch ( W), 53 ... operation display unit, 54 ... stereo switching switch, 60 ... personal computer.
Agent Patent Attorney Susumu Ito

Claims (8)

Stereo image data generating means for generating stereo image data based on a plurality of monocular images of the same subject obtained with a predetermined parallax;
Accompanying data generation means for generating accompanying data related to the stereo image data;
Stereo image size information generating means for generating information on the image size of the stereo image data;
Image file generation means for generating an image file by combining the stereo image data generated by the stereo image data generation means and the accompanying data generated by the accompanying data generation means, the stereo image size information Image file generating means for generating an image file by adding information on the image size generated by the generating means to the inside and outside of the accompanying data, and converting the information into a predetermined file format;
An image processing apparatus. The stereo image data based on a plurality of monocular images of the same subject obtained with a predetermined parallax and the accompanying data related to the stereo image data are combined, and information about the image size of the stereo image data is further added to the accompanying image Image processing / editing means for performing image processing / editing based on stereo image data in an image file generated by converting the data into a predetermined file format by adding the data inside and outside, respectively,
Ancillary data updating means for updating the ancillary data based on the content of the image processing / editing with respect to the ancillary data before the image processing / editing is performed;
Update the information about the image size of the stereo image data added to the inside and outside of the accompanying data before the image processing / editing based on the contents of the image processing / editing. Stereo image size information updating means for
Image file generating means for generating an image file by synthesizing the stereo image data processed and edited by the image processing / editing means and the accompanying data updated by the accompanying data updating means, Image file generation means for adding data relating to the image size updated by the image size information update means to the inside and outside of the accompanying data, converting the data to a predetermined file format, and generating an image file;
An image processing / editing apparatus. The stereo image data based on a plurality of monocular images of the same subject obtained with a predetermined parallax and the accompanying data related to the stereo image data are combined, and information about the image size of the stereo image data is further added to the accompanying image Reproduction means for reproducing the image file generated by adding the data to the inside and outside of the data and converting the data into a predetermined file format,
Determining means for determining a match between information relating to the image size in the accompanying data reproduced by the reproducing means and information relating to the image size outside the accompanying data;
Control means for prohibiting the continuation of the regenerating operation by the regenerating means when the determining means determines that there is a mismatch,
An image file reproduction apparatus having Control means for controlling to invalidate accompanying data instead of prohibiting the continuation of the reproduction operation when it is determined that the determination means does not match;
The image file reproducing device according to claim 3 , comprising:   5. The image file reproduction device according to claim 3, further comprising a warning unit that issues a warning when the determination unit determines that there is a mismatch. A stereo image generation step of generating stereo image data based on a plurality of monocular images of the same subject obtained with a predetermined parallax;
An accompanying data generation step of generating accompanying data relating to the stereo image data;
Stereo image size information generating step for generating information on the image size of the stereo image data;
The stereo image size information is a step of generating an image file by synthesizing the stereo image data generated in the stereo image data generating step and the accompanying data generated by the accompanying data generating means. An image file generation step of adding information on the image size generated by the generation means to the inside and outside of the accompanying data and converting the information to a predetermined file format to generate an image file;
An image processing method. The stereo image data based on a plurality of monocular images of the same subject obtained with a predetermined parallax and the accompanying data related to the stereo image data are combined, and information about the image size of the stereo image data is further added to the accompanying image An image processing / editing step for performing image processing / editing based on stereo image data in an image file generated by converting the data into a predetermined file format by adding it to the inside and outside of the data,
An accompanying data update step for updating the accompanying data based on the contents of the image processing / editing with respect to the accompanying data before the image processing / editing is performed,
Update the information about the image size of the stereo image data added to the inside and outside of the accompanying data before the image processing / editing based on the contents of the image processing / editing. A stereo image size information update step, an image for generating an image file by combining the stereo image data processed / edited by the image processing / editing unit and the accompanying data updated by the accompanying data update unit An image file generating step for generating an image file by adding data relating to the image size updated by the stereo image size information updating means to the inside and outside of the accompanying data and converting the data into a predetermined file format When,
An image processing / editing method. The stereo image data based on a plurality of monocular images of the same subject obtained with a predetermined parallax and the accompanying data related to the stereo image data are combined, and information about the image size of the stereo image data is further added to the accompanying image A reproduction step of reproducing an image file generated by converting the data into a predetermined file format by adding the data inside and outside, respectively;
A determination step of determining a match between information on the image size in the accompanying data reproduced by the reproducing step and information on the image size outside the accompanying data;
If the determination step determines that there is a discrepancy, prohibiting the continuation of the reproduction operation, invalidating the accompanying data, and performing at least one of a warning;
A method for reproducing an image file.

Images