JP2011139126A - Imaging apparatus and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus for generating image data according to a plurality of scene modes while reducing a data size and a processing time, and also to provide a program therefor. <P>SOLUTION: The imaging apparatus includes: an imaging unit for imaging a subject to generate an RAW image; a feature extraction unit for extracting a feature image including at least one feature portion from the RAW image; an image processing unit for processing the RAW image based on first scene information prescribing processing to the RAW image and processing the feature image based on second scene information prescribing processing to the feature image; a quantization unit for performing discrete cosine conversion processing and quantization processing to a RAW processing image and a feature processing image; a differential data generation unit for generating differential data indicating a difference between image data subjected to RAW processing image processing and feature image data subjected to feature processing image processing for each of feature image data; and an image compression unit for irreversibly compressing the image data. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and a program.

  In an imaging apparatus such as a digital still camera, a plurality of scene modes each defining processing for a RAW image obtained by imaging are stored, and when imaging is performed, a user has set (or previously set) Some image data can be generated according to the set scene mode by performing image processing according to the set scene mode.

  Under such circumstances, a technique for generating image data corresponding to a plurality of scene modes has been developed. By recording the image data corresponding to one scene mode and the difference data corresponding to the difference between the image data and the image data corresponding to another scene mode, and using the image data and the difference data, As a technique for generating image data corresponding to a corresponding scene mode, for example, Patent Document 1 and Patent Document 2 can be cited. Further, as a technique for generating image data corresponding to a scene mode by recording RAW image data and performing image processing on the RAW image data using a parameter corresponding to the scene mode, for example, Patent Document 3 and Patent Reference 4 is cited.

JP 2007-60153 A JP 2008-85882 A JP 2007-33038 A JP 2004-264945 A

  In the conventional technique for generating image data corresponding to the scene mode corresponding to the difference data by using the image data and the difference data (hereinafter sometimes referred to as “conventional technique 1”), the difference in the intermediate data. Is recorded as difference data. Therefore, when the conventional technique 1 is used, it is possible to generate image data different from the image data by using the image data and the difference data. However, in the conventional technique 1, since the difference in the entire image is recorded as difference data, the data size (data capacity) of the difference data is increased. Further, in the conventional technique 1, since the difference in the intermediate data is recorded as the difference data, new image data obtained from the image data and the difference data is an irreversibly compressed image such as JPEG (Joint Photographic Experts Group). In the case of data, the image indicated by the new image data is different from an image (image corresponding to the same scene mode) indicated by the image data that has been subjected to image processing without taking a difference and is irreversibly compressed. turn into.

  In addition, a conventional technique for generating image data corresponding to a scene mode by performing image processing on the RAW image data using a parameter corresponding to the scene mode (hereinafter sometimes referred to as “conventional technique 2”). Since the RAW image data is processed with parameters according to the scene mode, it is possible to generate image data according to the scene mode. However, since the RAW image data has a very large data size, when the conventional technique 2 is used, the storage capacity of the recording medium that stores the RAW image data is reduced. In addition, since image processing for RAW image data requires a certain amount of time, the conventional technique 2 cannot generate image data according to the scene mode unless the RAW image data itself is used. Therefore, when the conventional technique 2 is used, image processing takes time.

  Therefore, even if the conventional technique 1 and the conventional technique 2 are used, it is not desired to generate image data corresponding to a plurality of scene modes while reducing the data size and the processing time.

  The present invention has been made in view of the above problems, and an object of the present invention is to generate image data corresponding to a plurality of scene modes while reducing the data size and the processing time. It is an object of the present invention to provide a new and improved imaging apparatus and program capable of enabling

  In order to achieve the above object, according to a first aspect of the present invention, an imaging unit that images a subject and generates a RAW image, and a feature image including at least one feature part is extracted from the RAW image. The feature extraction unit performs image processing according to the first scene information for the RAW image based on first scene information that defines processing for the RAW image, and defines the processing for the feature image. An image processing unit that performs image processing on the feature image according to the second scene information based on the second scene information for each of the feature images extracted by the unit, and a RAW processing image obtained by subjecting the RAW image to image processing, A quantization unit that performs a discrete cosine transform process and a quantization process on the feature-processed image obtained by performing image processing on the feature image, and a RAW process in the quantization unit. A difference data generation unit that generates difference data for each feature image data, indicating difference between the image data obtained by processing the image and the feature image data obtained by processing the feature processed image in the quantization unit; An imaging apparatus including an image compression unit that performs irreversible compression is provided.

  In such a configuration, image data obtained by performing image processing corresponding to one scene mode (first scene mode) on a RAW image is irreversibly compressed, and the RAW image and the RAW image are converted into another scene mode (first image mode). Difference data with the feature image extracted corresponding to the second scene mode) is generated. That is, in such a configuration, the image data recorded on the recording medium can be image data having a data size smaller than that of the irreversibly compressed RAW image, and the difference data recorded on the recording medium is The image data that has been subjected to the image processing corresponding to the first scene mode and the image data that has been subjected to the image processing corresponding to the second scene mode are used as data indicating only the difference (that is, the difference data It is possible to make the data size smaller than the case of taking the difference of the whole image). Furthermore, since the difference data generated by such a configuration is data based on the difference between the RAW image and the feature image, the image indicated by the image data obtained by adding the difference data to the irreversibly compressed image data is The RAW image is the same image as the image indicated by the image data obtained by directly performing the image processing corresponding to the scene mode corresponding to the feature image used for generating the difference data. In other words, image data directly processed for each scene mode by using lossy-compressed image data and difference data (image data with reduced data size and difference data) generated by such a configuration. It is possible to obtain an image similar to the image indicated by.

  Further, the feature extraction unit may extract the feature image in a minimum processing unit of irreversible compression in the image compression unit.

  The second scene information corresponding to the feature image data used to generate the difference data for the RAW image by adding the compressed image data obtained by irreversibly compressing the image data and the difference data. An image restoration unit that generates image data that has been subjected to image processing according to the above may be further provided.

  The image processing unit includes a plurality of processing units that sequentially and selectively perform different processes on the input image, and the first image when the input image is the RAW image. Based on the scene information, a specific processing unit among the plurality of processing units performs processing according to the first scene information, and when the input image is the feature image, the second scene A process management unit that causes a specific processing unit of the plurality of processing units to perform processing according to the second scene information corresponding to the feature image based on the information may be provided.

  Further, the image processing apparatus may further include a setting unit that sets the first scene information, or the first scene information and at least one or more second scene information based on a user operation.

  In order to achieve the above object, according to a second aspect of the present invention, an imaging unit that images a subject and generates a RAW image, and a feature image including at least one feature part from the RAW image are provided. Based on the feature extraction unit to extract and the first scene information that defines the processing for the RAW image, the image processing according to the first scene information for the RAW image is performed, and the processing for the feature image is defined. An image processing unit that performs image processing according to the second scene information for the feature image based on the second scene information for each feature image extracted by the feature extraction unit, and a RAW process in which the RAW image is subjected to image processing A quantization unit that performs discrete cosine transform processing and quantization processing on the image and the feature-processed image obtained by performing image processing on the feature image, and R in the quantization unit A difference data generation unit that generates, for each feature image data, difference data indicating a difference between image data obtained by processing a W-processed image and feature image data obtained by processing a feature-processed image in the quantization unit; A program for generating image data based on image data irreversibly compressed by an imaging device including an image compression unit for irreversibly compressing data and difference data generated by the imaging device, the computer comprising: The compressed image data obtained by irreversibly compressing the image data and the difference data are added, and the second scene information corresponding to the feature image data used to generate the difference data is generated for the RAW image. A program is provided for functioning as means for generating image data subjected to image processing.

  In such a program, difference data based on the difference between the RAW image generated in the image pickup apparatus having the above configuration and the feature image is added to the image data irreversibly compressed by the image pickup apparatus having the above configuration. The image indicated by the image data is the same as the image indicated by the image data obtained by directly performing image processing according to the scene mode corresponding to the feature image used for generating the difference data for the RAW image. . Therefore, by using such a program, it is possible to use irreversibly compressed image data and difference data (image data and difference data with a reduced data size) generated by the imaging apparatus having the above configuration for each scene mode. It is possible to obtain an image similar to the image indicated by the image data subjected to direct image processing.

  According to the present invention, it is possible to generate image data corresponding to a plurality of scene modes while reducing the data size and the processing time.

It is a flowchart which shows an example of the scene information setting process in the imaging device which concerns on embodiment of this invention. It is a flowchart which shows an example of the data generation process in the imaging device which concerns on embodiment of this invention. It is a block diagram which shows an example of a structure of the imaging device which concerns on embodiment of this invention. It is a block diagram which shows an example of a structure of the image process part with which the imaging device which concerns on embodiment of this invention is provided.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

(Processing in Imaging Device According to Embodiment of Present Invention)
[Outline of Processing in Imaging Device According to Embodiment of Present Invention]
As described above, even when the conventional technique 1 and the conventional technique 2 are used, image data corresponding to a plurality of scene modes cannot be generated while reducing the data size and the processing time. Therefore, the imaging apparatus according to the embodiment of the present invention (hereinafter, referred to as “imaging apparatus 100”) extracts a feature image including a feature portion corresponding to a scene mode from a RAW image (an image before image processing), and RAW. Difference data based on the difference between the image and the feature image is generated. For example, when the scene mode related to feature image extraction is “portrait” indicating image processing suitable for a case where the subject is a person, the imaging apparatus 100 captures the image of the face portion of the subject in the RAW image. Are extracted as feature images. Therefore, since the imaging apparatus 100 does not use the difference in the entire image as difference data unlike the conventional technique 1, the data size of the difference data can be made smaller than the difference data according to the conventional technique 1.

  Here, the scene mode according to the embodiment of the present invention abstractly represents image processing performed by the imaging processing apparatus 100. Even if the content of the image processing is shown, the user who uses the imaging apparatus 100 cannot always understand the content of the image processing. Therefore, the imaging apparatus 100 assists the user in selecting a scene mode, for example, by showing the user a scene mode that abstracts image processing. In addition, as the scene mode according to the embodiment of the present invention, for example, the above “portrait”, “landscape” indicating image processing suitable when the subject is a landscape, or an image suitable when the subject is a child “Child” indicating processing, “Auto” indicating general-purpose image processing independent of the subject, and the like. Note that the imaging apparatus 100 can be used in various ways depending on the subject and the imaging environment, such as “snow”, “beach”, “macro”, “sunset”, “dawn”, “backlight”, and “fireworks”. It is possible to set the scene mode.

  The imaging apparatus 100 is obtained by imaging using first scene information that defines processing for a RAW image corresponding to a scene mode set for imaging (hereinafter referred to as “first scene mode”). Process the RAW image. In addition, the imaging apparatus 100 includes second scene information that defines processing for a feature image corresponding to another scene mode (hereinafter, referred to as “second scene mode”) set in addition to the first scene mode. To process feature images. For example, when a plurality of second scene modes are set by the user, the imaging apparatus 100 is set using a plurality of second scene information corresponding to each of the set second scene modes. A plurality of feature images corresponding to each of the plurality of second scene modes are processed. Hereinafter, information that defines processing for an image may be collectively referred to as “scene information”.

  The scene information according to the embodiment of the present invention includes, for example, the type of image processing to be executed, parameters for each image processing, and information indicating a feature portion for extracting a feature image. Table 1 shows an example of information indicating feature portions for extracting feature images according to the embodiment of the present invention.

  The imaging apparatus 100 performs irreversible compression on the image data after performing image processing according to the scene mode set for imaging on the RAW image. Therefore, the image capturing apparatus 100 does not store the RAW image data itself in a recording medium (for example, a storage unit described later) as in the conventional technique 2, so that the data size of the image data stored in the storage medium is the conventional technique 2. It can be made smaller than when using. That is, when the imaging apparatus 100 is used, unlike the conventional technique 2, the storable capacity of the recording medium that stores the image data is not excessively compressed.

  In addition, the imaging apparatus 100 adds difference between image data that has been irreversibly compressed (hereinafter sometimes referred to as “compressed image data”) and difference data that is based on the difference between the RAW image and the feature image. Image data (hereinafter, also referred to as “restored image data”) that has been subjected to image processing corresponding to the scene mode corresponding to the feature image used for data generation is generated. Here, since the difference data is data based on the difference between the RAW image and the feature image, the image indicated by the restored image data obtained by adding the difference data to the compressed image data is the difference data of the RAW image. The same image as the image indicated by the image data directly subjected to the image processing corresponding to the scene mode corresponding to the feature image used for generation is obtained. Therefore, when using the imaging apparatus 100, the image obtained as a result of using the difference data is not different from the image obtained as a result of image processing without using the difference data, as in the case of using the conventional technique 2.

  Furthermore, since the imaging apparatus 100 generates restored image data based on the compressed image data and the difference data, the restored image data is generated in a shorter time than the conventional technique 1 that performs image processing on RAW image data. Is possible.

  The imaging apparatus 100 according to the embodiment of the present invention can generate image data corresponding to a plurality of scene modes while performing the above-described processing while reducing the data size and the processing time. It can be.

[Specific Example of Processing in Imaging Device According to Embodiment of Present Invention]
Next, the process in the imaging device 100 will be described more specifically. The imaging apparatus 100 generates image data corresponding to a plurality of scene modes while reducing the data size and the processing time by performing the following processes (1) to (3), for example. Enable

(1) Scene information setting process The imaging apparatus 100 sets the first scene information corresponding to the first scene mode that is a scene mode for imaging and the second scene mode corresponding to the second scene mode that is another scene mode. 2 Set the scene information. Here, for example, the imaging apparatus 100 performs the process (1) based on an operation signal corresponding to a user operation transmitted from an operation unit (described later) included in the imaging apparatus 100. FIG. 1 is a flowchart showing an example of scene information setting processing in the imaging apparatus 100 according to the embodiment of the present invention.

  The imaging apparatus 100 sets a scene mode for imaging (first scene mode) (S100: first scene information setting process). In step S100, for example, the imaging apparatus 100 selects scene information corresponding to a scene mode set by the user from a plurality of scene information stored in advance, and sets the selected scene information as first scene information. To do. Further, when the first scene information is not set, the imaging apparatus 100 newly sets the scene information corresponding to the set scene mode as the first scene information, and the first scene information is already set. In this case, the scene information corresponding to the set scene mode is updated and set.

  When the first scene mode is set in step S100, the imaging apparatus 100 determines whether to set another scene mode (S102). The imaging apparatus 100 determines to set another scene mode, for example, when an operation signal based on a user operation indicates setting of another scene mode.

  If it is not determined in step S102 that another scene mode is set, the imaging apparatus 100 ends the process (1) (scene information setting process).

  If it is determined in step S102 that another scene mode is set, the imaging apparatus 100 sets the scene mode (second scene mode) selected by the user (S104. Second scene information setting process). ). In step S104, for example, the imaging apparatus 100 selects scene information corresponding to a scene mode set by the user from a plurality of scene information stored in advance, and sets the selected scene information as second scene information. To do. Further, the imaging apparatus 100 newly sets the scene information corresponding to the set scene mode as the second scene information regardless of whether the second scene information is not set or is already set. Note that the imaging apparatus 100 can also delete the preset second scene information (corresponding to the cancellation of the second scene mode) based on the user operation in step S104.

  When the second scene mode is set in step S104, the imaging apparatus 100 repeats the processing from step S102.

  The imaging apparatus 100 sets the first scene information, or the first scene information and the second scene information, for example, by performing the process shown in FIG.

(2) Data Generation Processing The imaging device 100 generates image data or image data and difference data when imaging is performed. FIG. 2 is a flowchart showing an example of data generation processing in the imaging apparatus 100 according to the embodiment of the present invention.

  The imaging apparatus 100 determines whether to perform imaging (S200). For example, the imaging apparatus 100 determines to perform imaging when an operation signal based on pressing of a shutter button (an example of a user operation) is detected. If it is not determined in step S200 that imaging is to be performed, the imaging apparatus 100 does not proceed with processing until it is determined that imaging is to be performed.

  If it is determined in step S200 that imaging is to be performed, the imaging apparatus 100 performs imaging in an imaging unit (described later) to generate a RAW image, and RAW based on the set first scene information. The image is processed (S202).

  The imaging apparatus 100 determines whether the second scene information is set (S204). 2 illustrates that the imaging apparatus 100 performs the process of step S204 after the process of step S202, the imaging apparatus 100 may perform the process of step S202 and the process of step S204 independently. Good. That is, the imaging apparatus 100 can also perform the process of step S202 and the process of step S204 in synchronization.

  If it is not determined in step S204 that the second scene information has been set, the imaging apparatus 100 uses the RAW image processed based on the first scene information in step S202 (hereinafter referred to as “RAW processed image”). On the other hand, DCT (Discrete Cosine Transformation) processing and quantization processing are performed (S216). Then, the imaging apparatus 100 performs a process in step S218 described later.

  When it is determined in step S204 that the second scene information has been set, the imaging apparatus 100 has a feature corresponding to the second scene information from the RAW image based on the set second scene information. A feature image including a portion is extracted (S206). In addition, the imaging apparatus 100 adds, for example, meta information indicating a position in the RAW image to the extracted feature image.

  Here, the imaging apparatus 100 extracts a feature image in a minimum processing unit (MCU: Minimum Coded Unit) for irreversible compression in step S216 described later. As described above, the imaging apparatus 100 simplifies the generation of restored image data using the image data (compressed image data) that has been irreversibly compressed in step S216 described later and the difference data generated in step S212 described later. Can do. Note that in the case where feature images are extracted in units of the irreversible compression minimum processing, the imaging apparatus 100 may perform processing to be described later using, for example, a plurality of feature images within a predetermined range as one feature image group.

  In addition, the imaging apparatus 100 performs the process of step S206 for each set second scene information. That is, for example, when a plurality of pieces of second scene information are set in the scene information setting process shown in FIG. 1, feature images corresponding to the set pieces of second scene information are respectively extracted.

  When the feature image is extracted in step S206, the imaging apparatus 100 processes the feature image based on the second scene information corresponding to each of the extracted feature images (S208).

  The imaging apparatus 100 performs DCT processing and quantum processing on the RAW processed image processed in step S202 and the feature image processed based on the second scene information in step S208 (hereinafter referred to as “feature processed image”). (S210).

  When the DCT process and the quantization process are performed in step S210, the imaging apparatus 100 indicates a difference between the image data obtained by processing the RAW processed image in step S210 and the feature image data obtained by processing the feature processed image in step S210. Difference data is generated for each feature image data (S212). Here, the imaging apparatus 100 generates difference data by subtracting feature image data from image data, for example.

  When difference data is generated in step S212, the imaging apparatus 100 records the generated difference data (S214). Here, the imaging apparatus 100 records in a storage medium (described later) provided in the imaging apparatus 100 or a recording medium such as an external recording medium. Note that the imaging apparatus 100 stores the difference data in a recording area unique to the manufacturer of the imaging apparatus 100 in which, for example, image data corresponding to the first scene mode (compressed image data recorded in step S218 described later) is recorded. Alternatively, the difference data may be recorded as a separate file from the image data.

  In addition, the imaging apparatus 100 may record the compressed difference data by performing a reversible compression process on the difference data. As described above, the imaging apparatus 100 can further reduce the data size of the difference data. The imaging apparatus 100 performs, for example, Huffman encoding processing and compresses difference data.

  When the process of step S212 or the process of step S216 is performed, the imaging apparatus 100 records the image data with lossy compression (S218). Here, the imaging apparatus 100 irreversibly compresses image data, for example, in the JPEG format or the JPEG2000 format, and records the data in a recording medium such as a storage unit (described later) or an external recording medium provided in the imaging apparatus 100.

  The imaging apparatus 100 performs, for example, the process shown in FIG. 2 to set the first scene information set in the scene information setting process shown in FIG. 1 or the first scene information and the scene information setting process shown in FIG. Based on the set second scene information, compressed image data and difference data are generated and recorded on a recording medium.

(3) Restoration processing When the image capturing apparatus 100 detects an image restoration command corresponding to the second scene mode based on a user operation, the compressed image data generated in the processing (data generation processing) of (2) By adding the difference data corresponding to the second scene mode, restored image data corresponding to the second scene mode corresponding to the difference data is generated.

  More specifically, when detecting the restoration command, the imaging apparatus 100 reads compressed image data and difference data corresponding to the restoration command from the recording medium. Here, when the difference data is compressed by the Huffman encoding process, the imaging apparatus 100 performs the Huffman decoding process to restore the difference data. Then, the imaging apparatus 100 generates restored image data by adding the difference data to the minimum processing unit portion of the compressed image data corresponding to the difference data. The imaging apparatus 100 generates a JPEG image (an example of an image indicated by the restored image data) by performing Huffman coding on the restored image data.

  The imaging apparatus 100 performs, for example, the above processing (1) (scene information setting processing) to (3) (restoration processing). Here, the imaging apparatus 100 sets the first scene information and the second scene information by the process (1) (scene information setting process), respectively. Further, in the process (2) (data generation process), the imaging apparatus 100 extracts a feature image corresponding to the second scene mode from the RAW image, and generates difference data based on the difference between the RAW image and the feature image. To do. Further, in the process (2) (data generation process), the imaging apparatus 100 performs image processing on the RAW image according to the first scene mode, and then irreversibly compresses the image data. In addition, the imaging apparatus 100 generates restored image data by adding the compressed image data and the difference data in the process (3) (restoration process).

  Therefore, for example, the imaging device 100 performs the processing (1) (scene information setting processing) to (3) (restoration processing), thereby achieving the effects described in the outline of the processing in the imaging device 100 described above. Can do.

(Image pickup device according to an embodiment of the present invention)
Next, an example of the configuration of the imaging apparatus 100 capable of performing the above-described processing (1) (scene information setting processing) to (3) (restoration processing) will be described. FIG. 3 is a block diagram illustrating an example of the configuration of the imaging apparatus 100 according to the embodiment of the present invention.

  The imaging apparatus 100 includes a setting unit 102, a storage unit 104, an imaging unit 106, a feature extraction unit 108, an image processing unit 110, a quantization processing unit 112, a compression processing unit 114, and an image restoration unit 116. Is provided.

  In addition, the imaging apparatus 100 includes, for example, a CPU (Central Processing Unit) and various processing circuits, and a control unit (not shown) that controls the entire imaging apparatus 100, and a program used by the control unit (not shown). ROM (Read Only Memory, not shown) in which control data such as calculation parameters are recorded, and RAM (Random Access Memory, not shown) for temporarily storing a program executed by a control unit (not shown). An operation unit (not shown) operable by the user, a display unit (not shown), a communication unit (not shown) for communicating with an external device, and the like may be provided.

  Examples of the operation unit (not shown) include operation input devices such as a keyboard and a mouse, buttons, direction keys, and combinations thereof. Note that the operation unit (not shown) may have a function of receiving an input from an external device such as a remote controller. The display unit (not shown) is a display unit included in the imaging apparatus 100 and displays various display screens. As a screen displayed on the display unit (not shown), for example, an image (moving image or still image; the same shall apply hereinafter) indicated by the image data stored in the storage unit 104 or the processing of (3) above. Examples include a screen on which an image indicated by the restored data is displayed. Examples of the display unit (not shown) include a liquid crystal display and an organic EL display. A communication unit (not shown) is a communication unit included in the imaging apparatus 100 and serves to perform wired / wireless communication with an external apparatus via a network (or directly). Examples of the communication unit (not shown) include a LAN (Local Area Network) terminal and a transmission / reception circuit, an IEEE 802.15.1 port and a transmission / reception circuit, and the like.

  The setting unit 102 plays a role of leading the process (1) (scene information setting process). More specifically, the setting unit 102 performs the process shown in FIG. 1 based on an operation signal according to a user operation transmitted from an operation unit (not shown), for example, and performs the first scene information or the first One scene information and second scene information are set in the storage unit 104. In the imaging apparatus 100, for example, a control unit (not shown) may serve as the setting unit 102.

  The storage unit 104 is a storage unit included in the imaging device 100. Examples of the storage unit 104 include a magnetic recording medium such as a hard disk and a non-volatile memory such as a flash memory.

  The storage unit 104 stores various data such as scene information, compressed image data, difference data corresponding to the compressed image data, and applications. Here, in FIG. 3, it corresponds to the first scene information 150, the second scene information A152, the second scene information B154, the compressed image data 156 corresponding to the first scene information 150, and the second scene information 152A. In the example, difference data A158 and difference data B160 corresponding to the second scene information B154 are stored in the storage unit 104.

  The imaging unit 106 is configured by an imaging element such as a shutter or a CCD (Charge Coupled Device), for example, and performs imaging based on an operation signal corresponding to a user operation transmitted from an operation unit (not shown). Then, the imaging unit 106 outputs a RAW image obtained as a result of imaging to the feature extraction unit 108.

  The feature extraction unit 108, the image processing unit 110, the quantization processing unit 112, and the compression processing unit 114 serve to lead the processing (data generation processing) of (2). Here, the imaging apparatus 100 includes, for example, each of the feature extraction unit 108, the image processing unit 110, the quantization processing unit 112, and the compression processing unit 114 as a processing circuit that performs each process, and these processing circuits are integrated. You may provide as a circuit (namely, one processing circuit).

  The feature extraction unit 108 extracts a feature image from the RAW image based on the second scene information stored in the storage unit 104. Then, the feature extraction unit 108 outputs the RAW image and the feature image to the image processing unit 110. Note that when the second scene information is not stored in the storage unit 104 (that is, when the second scene mode is not set), the feature extraction unit 108 does not perform processing related to the feature image extraction. . When the imaging apparatus 100 is configured to directly input the RAW image output from the imaging unit 106 to the image processing unit 110, the feature extraction unit 108 transmits the feature image to the image processing unit 110. Output.

  The image processing unit 110 sequentially and selectively performs different processing on the input image (RAW image, one or more feature images), and processes the processed image (RAW processed image, feature processed image). Output. More specifically, the image processing unit 110 performs image processing corresponding to the first scene information for the RAW image based on the first scene information. Further, when a feature image is input, the image processing unit 110 performs image processing corresponding to the second scene information for the feature image based on the second scene information corresponding to the feature image stored in the storage unit 104. I do.

  FIG. 4 is a block diagram illustrating an example of the configuration of the image processing unit 110 included in the imaging apparatus 100 according to the embodiment of the present invention. The image processing unit 110 includes a noise reduction processing unit 130, a white balance processing unit 132, The color interpolation processing unit 134, the gamma processing unit 136, the color processing unit 138, and the contour enhancement processing unit 140 are provided as processing units. In addition, the image processing unit 110 includes a processing management unit 142 that controls each processing unit and causes each processing unit to perform processing selectively.

  The noise reduction processing unit 130 reduces noise included in an image by filter processing using a filter such as a low-pass filter or a band-pass filter. For example, the white balance processing unit 132 applies a gain set in advance for each color of RGB (Red Green Blue) to the image, and amplifies the pixel value of each pixel (pixel). For example, the color interpolation processing unit 134 generates RGB of all pixels from the Bayer array. For example, the gamma processing unit 136 performs non-linear transformation on the image to ensure visual linearity. The color processing unit 138 corrects the color tone of the image, for example. The contour enhancement processing unit 140, for example, detects an edge portion from the image and enhances the lightness and darkness of the image by increasing the luminance of the detected edge portion.

  When the input image is a RAW image, the processing management unit 110 causes each processing unit to selectively perform processing based on the first scene information. When the processing management unit 110 selectively causes each processing unit to perform processing based on the first scene information on the input RAW image, the RAW processing image output from the image processing unit 110 becomes a RAW image. On the other hand, the image is processed according to the first scene mode set.

  In addition, when the input image is a feature image, the process management unit 110 selectively causes each processing unit to perform a process based on the second scene information corresponding to the feature image. When the process management unit 110 selectively causes each processing unit to perform processing based on the corresponding second scene information on the input feature image, the feature processing image output from the image processing unit 110 is: The image is subjected to image processing according to the second scene mode set for the feature image. Table 2 shows an example of the extracted content of the feature image when each processing unit is performed on the feature image. In Table 2, noise reduction is indicated as “NR” and white balance is indicated as “WB”.

  For example, with the configuration of FIG. 4, the image processing unit 110 sequentially and selectively performs different processing on an input image and outputs a processed image. Note that the imaging apparatus 100 may be configured to include, for example, a plurality of processing units illustrated in FIG. 4 corresponding to the number of images (RAW images and feature images) that can be input.

  With reference to FIG. 3 again, an example of the configuration of the imaging apparatus 100 according to the embodiment of the present invention will be described. The quantization processing unit 112 performs DCT processing and quantization processing on each input processed image.

  The compression processing unit 114 generates difference data and compresses image data, and records the compressed image data and difference data in the storage unit 104. Further, the compression processing unit 114 may further compress the generated difference data.

  More specifically, the compression unit 114 includes a difference data generation unit 118, a first compression processing unit 120 (image compression unit), and a second compression processing unit 122. Here, FIG. 4 illustrates a configuration in which the compression processing unit 114 includes the second compression processing unit 122 that compresses the difference data, but a configuration in which the second compression processing unit 122 is not included may be employed.

  The difference data generation unit 118 generates difference data for each feature image data based on the image data processed from the RAW processing image output from the quantization unit 112 and the feature image data processed from the feature processing image. . Then, the difference data generation unit 118 outputs the image data to the first compression processing unit 120, and outputs the generated one or more difference data to the second compression processing unit 122. Although not shown in FIG. 4, the difference data 118 includes a volatile memory such as SDRAM (Synchronous Dynamic Random Access Memory), and one or more image data output from the quantization unit 112. The difference data may be generated after storing the feature image data.

  The first compression processing unit 120 performs irreversible compression on the input image data, and records the compressed image data that has been irreversibly compressed in the storage unit 104. Further, the second compression processing unit 122 performs Huffman compression (an example of lossless compression) on each input difference data, and records the compressed difference data in the storage unit 104.

  The imaging apparatus 100 includes the feature extraction unit 108, the image processing unit 110, the quantization processing unit 112, and the compression processing unit 114, thereby performing the process (2) (data generation process).

  The image restoration unit 116 plays a role of performing the processing (restoration processing) of (3) above. More specifically, when the image restoration unit 116 detects a restoration command based on an operation signal corresponding to a user operation transmitted from an operation unit (not shown), for example, the compressed image data corresponding to the restoration command And the difference data are read from the storage unit 104. Further, the image restoration unit 116 performs Huffman decoding processing to restore the difference data, and adds the difference data to the minimum processing unit portion of the image data corresponding to the difference data to generate restored image data. The image restoration unit 116 generates a JPEG image (an example of an image indicated by the restored image data) by performing Huffman coding on the restored image data. Note that the image restoration unit 116 may display an image indicated by the generated restored image data on a display unit (not shown), or may record the generated restored image data in the storage unit 104.

  Further, the image restoration unit 116 displays an image indicated by the compressed image data stored in the storage unit 104 based on an operation signal according to a user operation transmitted from an operation unit (not shown). Display).

  The imaging apparatus 100 includes the image restoration unit 116, so that not only the image indicated by the compressed image data corresponding to the set first scene mode but also the image corresponding to the set second scene mode, for example, Provide to users based on user operations. The imaging apparatus 100 further includes a display control unit (not shown) that causes the display unit (not shown) to display the image indicated by the compressed image data or the image indicated by the restored image data generated by the image restoration unit 116. May be. In the imaging apparatus 100, a control unit (not shown) can also serve as the image restoration unit 116.

  Further, in the imaging apparatus 100, for example, a control unit (not shown) executes a program for causing a computer to function as an image restoration unit corresponding to the image restoration unit 116, thereby performing the process (restoration process) of (3). Can also be done.

  The imaging apparatus 100 performs the above-described processing (1) (scene information setting processing) to (3) (restoration processing) with the configuration illustrated in FIG. 3, for example. Therefore, the imaging apparatus 100 can generate image data corresponding to a plurality of scene modes while reducing the data size and the processing time.

  As described above, the imaging apparatus 100 according to the embodiment of the present invention extracts the feature image corresponding to the second scene mode from the RAW image, and generates difference data based on the difference between the RAW image and the feature image ( Process (2) (data generation process)). Therefore, since the imaging apparatus 100 does not use the difference in the entire image as difference data unlike the conventional technique 1, the image processing for the difference data can be shortened compared to the case of using the conventional technique 1, and the difference data The data size can be made smaller than the difference data according to the conventional technique 1. Further, since the imaging apparatus 100 does not require development processing for feature extraction as in the case of using the conventional technique 1, the processing time can be further reduced as compared with the case of using the conventional technique 1.

  In addition, the image capturing apparatus 100 performs image processing according to the first scene mode on the RAW image, and then irreversibly compresses the image data (processing (2) (data generation processing)). Here, the difference data is data based on the difference between the RAW image and the feature image. Therefore, the image indicated by the restored image data obtained by adding the difference data to the compressed image data generated by the imaging apparatus 100 corresponds to the scene mode corresponding to the feature image used for generating the difference data with respect to the RAW image. The image is the same as the image indicated by the image data directly subjected to the image processing. That is, when the imaging apparatus 100 is used, an image obtained as a result of using difference data as in the case of using the conventional technique 2 is not different from an image obtained as a result of image processing without using difference data.

  In addition, the imaging apparatus 100 performs irreversible compression on the image data after performing image processing on the RAW image according to the first scene mode set for imaging. Therefore, the imaging apparatus 100 does not store the RAW image data itself in the storage unit 104 as in the conventional technique 2, and therefore, the data size of the image data stored in the storage unit 104 is larger than that in the case of using the conventional technique 2. Can be small. That is, when the imaging apparatus 100 is used, the storable capacity in the storage unit 104 is not excessively compressed as in the conventional technique 2.

  Furthermore, the imaging apparatus 100 generates restored image data by adding the compressed image data and the difference data (process (3) (restoration process)). Therefore, the imaging apparatus 100 uses the conventional technique 1 because it is not necessary to perform an IDCT (Inverse Discrete Cosine Transform) process on the difference for restoring the difference data as in the case of using the conventional technique 1. The restored image data can be generated in a shorter time than the case. Further, since the imaging apparatus 100 does not generate restored image data by performing image processing on the RAW image data as in the case of using the conventional technique 2, the image capturing apparatus 100 restores in a shorter time than in the case of using the conventional technique 2. Image data can be generated.

  Therefore, the imaging apparatus 100 can generate image data corresponding to a plurality of scene modes while reducing the data size and the processing time.

[Modification of Imaging Device 100]
The imaging apparatus 100 may not include the imaging unit 106 and may be configured to process a RAW image transmitted from an imaging unit as an external device. In the above case, the imaging apparatus according to the modification of the embodiment of the present invention serves as an image processing apparatus. Even in the above case, the imaging apparatus (image processing apparatus) according to the modified example of the embodiment of the present invention has the above-described processes (1) (scene information setting process) to (3) (restoration process). Therefore, the same effect as that of the imaging device 100 according to the embodiment of the present invention can be obtained.

  As described above, the imaging apparatus has been described as an embodiment of the present invention. However, the above-described embodiment of the present invention is, for example, an imaging apparatus such as a digital still camera and a digital video camera, a portable communication apparatus such as a mobile phone, and a PC. It can be applied to various devices such as computers such as (Personal Computer) and servers (Server), and portable game machines.

(Program according to an embodiment of the present invention)
A program for causing a computer to execute the process (2) (data generation process) can generate image data corresponding to a plurality of scene modes while reducing the data size and the processing time. It can be.

  Further, a program for causing a computer to function as an image restoration unit corresponding to the image restoration unit of the imaging apparatus according to the embodiment of the present invention (a program for causing a computer to execute the process (3) (restoration process)). Thus, based on the compressed image data and the difference data according to the embodiment of the present invention, it is possible to generate restored image data corresponding to the scene mode corresponding to the difference data.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to the example which concerns. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood.

  For example, in the above, a program (computer program) for causing a computer to execute the process (2) (data generation process), and an image restoration unit corresponding to the image restoration unit of the imaging apparatus according to the embodiment of the present invention Although it has been shown that a program (computer program) for functioning as a means is provided, the embodiment of the present invention can further provide a storage medium storing the program.

  The configuration described above shows an example of the embodiment of the present invention, and of course, the technical features of the present invention.

DESCRIPTION OF SYMBOLS 100 Imaging device 102 Setting part 104 Storage part 106 Imaging part 108 Feature extraction part 110 Image processing part 112 Quantization processing part 114 Compression processing part 116 Restoration part 118 Difference data generation part 120 1st compression processing part 122 2nd compression processing part 130 Noise reduction processing unit 132 White balance processing unit 134 Color interpolation processing unit 136 Gamma processing unit 138 Color processing unit 140 Outline enhancement processing unit

Claims (6)

An imaging unit that images a subject and generates a RAW image;
A feature extraction unit that extracts a feature image including at least one feature portion from the RAW image;
Based on the first scene information that defines the process for the RAW image, the image extraction unit that performs the image processing according to the first scene information for the RAW image and defines the process for the feature image is extracted. An image processing unit that performs image processing according to the second scene information for the feature image based on the second scene information for each feature image;
A quantization unit that performs a discrete cosine transform process and a quantization process on a RAW processed image obtained by performing image processing on the RAW image and a feature processed image obtained by performing image processing on the feature image;
Difference data generation for generating, for each feature image data, difference data indicating a difference between image data obtained by processing a RAW processed image in the quantization unit and feature image data obtained by processing a feature processed image in the quantization unit And
An image compression unit for irreversibly compressing the image data;
An imaging apparatus comprising:   The imaging apparatus according to claim 1, wherein the feature extraction unit extracts the feature image in a minimum processing unit of irreversible compression in the image compression unit.   The compressed image data obtained by irreversibly compressing the image data and the difference data are added, and the second scene information corresponding to the feature image data used for generating the difference data for the RAW image is added. The imaging apparatus according to claim 2, further comprising an image restoration unit that generates image data that has been subjected to image processing. The image processing unit
A plurality of processing units that sequentially and selectively perform different processes on an input image;
When the input image is the RAW image, based on the first scene information, a specific processing unit of the plurality of processing units performs processing according to the first scene information, and the input When the image to be processed is the feature image, a process corresponding to the second scene information corresponding to the feature image is performed on a specific processing unit among the plurality of processing units based on the second scene information. A process management unit that performs
The imaging apparatus according to any one of claims 1 to 3, further comprising:   4. The apparatus according to claim 1, further comprising a setting unit configured to set the first scene information, or the first scene information and at least one or more second scene information based on a user operation. The imaging apparatus of Claim 1. An imaging unit that images a subject and generates a RAW image;
A feature extraction unit that extracts a feature image including at least one feature portion from the RAW image;
Based on the first scene information that defines the process for the RAW image, the image extraction unit that performs the image processing according to the first scene information for the RAW image and defines the process for the feature image is extracted. An image processing unit that performs image processing according to the second scene information for the feature image based on the second scene information for each feature image;
A quantization unit that performs a discrete cosine transform process and a quantization process on a RAW processed image obtained by performing image processing on the RAW image and a feature processed image obtained by performing image processing on the feature image;
Difference data generation for generating, for each feature image data, difference data indicating a difference between image data obtained by processing a RAW processed image in the quantization unit and feature image data obtained by processing a feature processed image in the quantization unit And
An image compression unit for irreversibly compressing the image data;
A program that generates image data based on image data that has been irreversibly compressed by an imaging device including the difference data generated by the imaging device,
Computer
The compressed image data obtained by irreversibly compressing the image data and the difference data are added, and the second scene information corresponding to the feature image data used for generating the difference data for the RAW image is added. Means for generating image data that has undergone image processing;
Program to function as.

Images