JP5082141B2 - Image processing system, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing system, an image processing method, and a program. The present invention particularly relates to an image processing system and an image processing method for processing an image, and a program for the image processing system.

  There is known an image processing apparatus that can switch between a normal compression mode and a RAW compression mode (see, for example, Patent Document 1). In this image processing apparatus, in the normal compression mode, the raw data is interpolated to compensate for missing signal components in units of pixels, and in the RAW compression mode, the signal processing unit that bypasses the interpolation processing and the output of the signal processing unit are image-compressed. .

Also, a video image processing apparatus for creating a print separation version without using an intermediate medium such as a film from a video image is known (for example, see Patent Document 2). In this video image processing apparatus, a video signal is converted into a digital signal and stored, a plurality of portions to be interpolated are selected from the stored image, and each selected portion is in accordance with the nature of the image. Apply different interpolation processes.
Japanese Patent No. 3864748 Japanese Examined Patent Publication No. 6-59106

  However, in the RAW compression mode according to the technique disclosed in Patent Document 1, the same compression processing is performed on the entire image area. Further, according to the technique disclosed in Patent Document 2, all color areas have color signals in all image areas. For this reason, according to the technique of the said patent document 1 and the patent document 2, the calculation amount in image processing will become large.

  In order to solve the above-described problem, according to a first aspect of the present invention, an image processing system includes an image acquisition unit that acquires a RAW image expressed in a RAW format, and a RAW image using a RAW image. A compressed RAW image expressed in the RAW format by compressing the RAW image with a different intensity between the feature area in the RAW image and the area other than the feature area in the RAW image, and a feature area detection unit that detects the feature area A compression unit that generates a compressed RAW image.

  According to a second aspect of the present invention, there is provided an image processing method, an image acquisition stage for acquiring a RAW image expressed in a RAW format, and a feature region detection for detecting a feature region in the RAW image using the RAW image. And a compression step of generating a compressed RAW image represented in a RAW format by compressing the RAW image with different intensities in a feature region in the RAW image and a region other than the feature region in the RAW image. .

  According to a third aspect of the present invention, there is provided a program for an image processing system, wherein a computer uses an RAW image to acquire a feature region in a RAW image using an image acquisition unit that acquires a RAW image represented in RAW format. A feature region detection unit for detecting, a compression that generates a compressed RAW image represented in the RAW format by compressing the RAW image with different intensities in the feature region in the RAW image and the region other than the feature region in the RAW image. Function as a part.

  According to the fourth aspect of the present invention, there is provided an image processing system, an image acquisition unit that acquires a RAW image expressed in a RAW format, and a feature region detection that detects a feature region in the RAW image using the RAW image. And a color processing unit that calculates values of a plurality of color components in each of a plurality of pixels in the RAW image based on pixel values of one or more pixels in the RAW image. The values of a plurality of color components are calculated using different methods for the pixels in the feature region and the pixels in the region other than the feature region in the RAW image.

  According to a fifth aspect of the present invention, there is provided an image processing method, an image acquisition stage for acquiring a RAW image represented in a RAW format, and a feature region detection for detecting a feature region in the RAW image using the RAW image. A color processing stage that calculates values of a plurality of color components in each of a plurality of pixels in the RAW image based on pixel values of one or more pixels in the RAW image, and the color processing stage includes: The values of the plurality of color components are calculated by different methods for pixels in the feature region and pixels in a region other than the feature region in the RAW image.

  According to a sixth aspect of the present invention, there is provided a program for an image processing system, wherein a computer uses an RAW image to acquire a feature region in a RAW image by using an RAW image. A feature region detection unit to detect, a color processing unit that calculates values of a plurality of color components in each of a plurality of pixels in a RAW image based on pixel values of one or more pixels in the RAW image, and a feature in the RAW image The pixel in the region and the pixel in the region other than the feature region in the RAW image function as a color processing unit that calculates the values of the plurality of color components by different methods.

  The above summary of the invention does not enumerate all necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 shows an example of an image processing system 10 according to an embodiment. The image processing system 10 can function as a monitoring system as described below.

  The image processing system 10 includes a plurality of imaging devices 100a to 100d (hereinafter collectively referred to as imaging devices 100) that capture the monitoring target space 150, an image processing device 120, a communication network 110, an image processing device 170, an image DB 175, and A plurality of display devices 180a-d (hereinafter collectively referred to as display devices 180) are provided. Note that the image processing device 170 and the display device 180 are provided in a space 160 different from the monitoring target space 150.

  The imaging device 100a images the monitoring target space 150. The imaging device 100a includes an imaging unit 102a and a captured image compression unit 104a. The imaging device 100a may be, for example, a single-plate color camera, and generates a RAW image RAW image generated according to the amount of light received by each of the plurality of light receiving elements included in the imaging unit 102a. Note that the imaging apparatus 100a may generate a plurality of RAW images by continuously capturing images. Then, the imaging device 100a transmits a RAW moving image including a plurality of compressed RAW images as moving image constituent images to the image processing device 120. Note that the imaging apparatus 100a may compress the RAW moving image by the captured image compression unit 104a and transmit the compressed RAW moving image to the image processing apparatus 120.

  Note that the imaging device 100b, the imaging device 100c, and the imaging device 100d have the same functions and operations as the imaging device 100a, respectively. Therefore, description of the functions and operations of the imaging device 100b, the imaging device 100c, and the imaging device 100d is omitted.

  The image processing device 120 acquires a RAW moving image from the imaging device 100. The image processing apparatus 120 performs various kinds of image processing on the RAW moving image in the RAW format.

  For example, the image processing apparatus 120 uses the acquired RAW moving image to select a plurality of feature regions having different types of features, such as a region where a person 130 is captured, a region where a moving body 140 such as a vehicle is captured, and the like. To detect. Then, the image processing apparatus 120 compresses the RAW moving image by reducing the image quality of the region other than the feature region in the RAW format. A compressed RAW moving image which is a compressed moving image in the RAW format is generated. The image processing device 120 transmits the compressed RAW moving image to the image processing device 170.

  Note that the image processing apparatus 120 generates feature area information including information for specifying the feature area. Then, the image processing device 120 attaches the feature area information to the compressed RAW moving image and transmits the feature region information to the image processing device 170 through the communication network 110.

  The image processing apparatus 170 receives the RAW moving image associated with the feature area information from the image processing apparatus 120. Then, the image processing apparatus 170 records the received RAW moving image in the image DB 175. As described above, the image DB 175 holds the compressed RAW image. Note that the image DB 175 may include a non-volatile recording medium such as a hard disk, and may hold a compressed RAW image in the recording medium. The image DB 175 can function as an image holding unit in the present invention.

  Further, the image processing apparatus 170 receives an instruction to display the video of the monitoring target space 150 on the display device 180. When the image processing apparatus 170 accepts the instruction, the image processing apparatus 170 reads the compressed RAW moving image recorded in the image DB 175, decompresses it using the associated feature area information, and performs the synchronization process. To the display device 180. At this time, the image processing apparatus 170 performs the synchronization process in the feature region with higher accuracy than the region other than the feature region.

  The feature area information is text data including the position of the feature area, the size of the feature area, the number of feature areas, identification information for identifying the captured image in which the feature area is detected, or the like, or compressed or encrypted into the text data. It may be data that has been processed. Then, the image processing apparatus 170 identifies a RAW image that satisfies various search conditions based on the position of the feature region included in the feature region information, the size of the feature region, the number of feature regions, and the like. Then, the image processing apparatus 170 may perform synchronization processing on the identified RAW image and provide it to the display apparatus 180.

  As described above, according to the image processing system 10, since the feature region is recorded in association with the RAW moving image, it is possible to search and find a RAW image that matches a predetermined condition in the RAW moving image at high speed. Further, according to the image processing system 10, only a RAW image that meets a predetermined condition can be decoded, so that a partial moving image that meets the predetermined condition can be provided promptly in response to a reproduction instruction.

  As described above, the image processing apparatus 120 performs feature region detection and compression processing in the RAW format. A RAW image contains one color component value per pixel. Therefore, the image processing apparatus 120 only needs to compress image information with a smaller number of colors compared to a case where an image including a plurality of color component values per pixel is handled. For this reason, the image processing apparatus 120 can perform compression with a smaller amount of computation, and as a result, the compressed RAW moving image can be quickly supplied to the image processing apparatus 170. In addition, since the imaging apparatus 100 only needs to transmit a RAW moving image to the image processing apparatus 120 without performing a synchronization process, the RAW moving image can be quickly transmitted to the image processing apparatus 120.

  Note that the RAW image includes a smaller number of colors than an image including a plurality of color component values per pixel, and therefore the data is small if the color depth is the same. Therefore, according to the image processing system 10, images are exchanged in the RAW format on the communication path between the imaging apparatus 100 and the image processing apparatus 120 and on the communication path between the image processing apparatus 120 and the image processing apparatus 170. In some cases, the amount of image data in these communication paths can be significantly reduced.

  Note that the image processing apparatus 120 generates feature area information including information that identifies the feature area detected from the RAW image. Then, the image processing device 120 attaches the feature area information to the compressed RAW moving image and transmits the feature region information to the image processing device 170 through the communication network 110.

  The image processing apparatus 170 receives the compressed RAW moving image associated with the feature area information from the image processing apparatus 120. Then, the image processing apparatus 170 generates a display moving picture by decompressing and synchronizing the received compressed RAW moving picture using the associated feature area information, and supplies the generated display moving picture to the display apparatus 180. To do. The display device 180 displays the moving image for display supplied from the image processing device 170.

  The image processing apparatus 170 may record the compressed moving image data in the image DB 175 in association with the feature area information associated with the compressed RAW moving image data. Then, in response to a request from the display device 180, the image processing device 170 reads the compressed moving image data and the feature region information from the image DB 175, decompresses the read compressed moving image data using the feature region information, and displays it. A moving image may be generated and supplied to the display device 180.

  The feature area information is text data including the position of the feature area, the size of the feature area, the number of feature areas, identification information for identifying the captured image in which the feature area is detected, or the like, or compressed or encrypted into the text data. It may be data that has been processed. Then, the image processing apparatus 170 identifies captured images that satisfy various search conditions based on the position of the feature region, the size of the feature region, the number of feature regions, and the like included in the feature region information. Then, the image processing device 170 may decode the identified captured image and provide the decoded captured image to the display device 180.

  As described above, according to the image processing system 10, since the representative feature region is recorded in association with the moving image, it is possible to quickly search and find a picked-up image group that meets a predetermined condition in the moving image. Also, according to the image processing system 10, only a captured image group that meets a predetermined condition can be decoded, so that a partial moving image that meets the predetermined condition can be displayed promptly in response to a reproduction instruction.

  FIG. 2 shows an example of a block configuration of the image processing apparatus 120. The image processing apparatus 120 includes an image acquisition unit 250, a feature region detection unit 203, a detection condition acquisition unit 204, a compression control unit 210, a compression unit 230, an association processing unit 206, and an output unit 207. The image acquisition unit 250 includes a compressed moving image acquisition unit 201 and a compressed moving image expansion unit 202.

  The compressed moving image acquisition unit 201 acquires a compressed moving image. Specifically, the compressed moving image acquisition unit 201 acquires encoded captured moving image data generated by the imaging device 100. Note that the captured moving image data may be obtained by compressing the captured moving image represented in the RAW format.

  The compressed moving image expansion unit 202 expands the captured moving image data acquired by the compressed moving image acquisition unit 201 and generates a plurality of captured images included in the captured moving image. Note that the captured image in the present embodiment is represented in the RAW format unless otherwise specified. And the captured image in the following description may be an example of the RAW image in this invention.

  Specifically, the compressed moving image decompression unit 202 decodes the encoded captured moving image data acquired by the compressed moving image acquisition unit 201, and generates a plurality of captured images included in the captured moving image.

  The captured image included in the captured moving image may be a frame image and a field image. The captured image in the present embodiment may be an example of a moving image constituent image in the present invention. In this way, the image acquisition unit 250 acquires a plurality of moving images captured by each of the plurality of imaging devices 100.

  The plurality of captured images obtained by the compressed moving image decompression unit 202 are supplied to the feature region detection unit 203 and the compression unit 230. The feature region detection unit 203 detects a feature region from a moving image including a plurality of captured images. Specifically, the feature region detection unit 203 detects a feature region from each of the plurality of captured images. As described above, the feature region detection unit 203 detects the feature region in the captured image using the captured image represented in the RAW format. Note that the captured moving image described above may be an example of a moving image in the following description.

  For example, the feature region detection unit 203 detects an image region whose image content changes in a moving image as a feature region. Specifically, the feature region detection unit 203 may detect an image region including a moving object as a feature region. Note that the feature region detection unit 203 may detect a plurality of feature regions having different types of features from each of the plurality of captured images. Note that the type of feature may be an index of the type of object such as a person and a moving object. The type of the object may be determined based on the degree of coincidence of the shape of the object or the color of the object. As described above, the feature region detection unit 203 may detect a plurality of feature regions having different types of included objects from a plurality of captured images.

  For example, the feature region detection unit 203 extracts an object that matches a predetermined shape pattern with a matching degree equal to or higher than a predetermined matching degree from each of the plurality of picked-up images, and in the picked-up image including the extracted object. The region may be detected as a feature region having the same feature type. Note that a plurality of shape patterns may be determined for each type of feature. As an example of the shape pattern, a shape pattern of a human face can be exemplified. Different face patterns may be determined for each of a plurality of persons. Thereby, the feature region detection unit 203 can detect different regions each including a different person as different feature regions. The feature region detection unit 203 includes, in addition to the person's face, a part of the human body such as the head of the person or the hand of the person, or at least a part of a living body other than the human body. Can be detected as a feature region. The living body includes a specific tissue existing inside the living body such as a tumor tissue or a blood vessel inside the living body. In addition to the living body, the feature region detection unit 203 may detect, as the feature region, a region in which a card such as money, a cash card, a vehicle, or a license plate of the vehicle is captured.

  In addition to pattern matching by template matching or the like, the feature region detection unit 203 detects feature regions based on learning results by machine learning (for example, Adaboost) described in, for example, Japanese Patent Application Laid-Open No. 2007-188419. It can also be detected. For example, an image feature amount extracted from a predetermined subject image and an image feature amount extracted from a subject image other than the predetermined subject are extracted from the predetermined subject image. Learn the features of the image features. Then, the feature region detection unit 203 may detect a region from which an image feature amount having a feature that matches the learned feature is extracted as a feature region. Thereby, the feature region detection unit 203 can detect a region where a predetermined subject is imaged as a feature region.

  Note that the feature region detection unit 203 may detect the feature region by the above-described method based on the pixel value of a pixel having a specific color component value in the captured image. For example, the detection condition acquisition unit 204 acquires color components of pixel values to be used when the feature region detection unit 203 detects the feature region from the captured image. The detection condition acquisition unit 204 may acquire the color component from the outside of the image processing apparatus 120. The feature region detection unit 203 may detect the feature region based on the pixel value of the pixel having the value of the color component acquired by the detection condition acquisition unit 204 among the pixels included in the captured image.

  As described above, the feature region detection unit 203 detects a plurality of feature regions from a plurality of captured images included in each of the plurality of moving images. Then, the feature region detection unit 203 supplies information indicating the detected feature region to the compression control unit 210. Note that the information indicating the feature region includes coordinate information of the feature region indicating the position of the feature region, type information indicating the type of the feature region, and information for identifying the captured moving image in which the feature region is detected.

  The compression control unit 210 controls the compression processing of the moving image by the compression unit 230 according to the feature region based on the information indicating the feature region acquired from the feature region detection unit 203. The compression unit 230 compresses the captured image by reducing the resolution of the region other than the feature region in the moving image constituent image included in the moving image. The compression unit 230 compresses the captured image expressed in the RAW format with different intensities in the feature region in the captured image and the region other than the feature region in the captured image, thereby compressing the captured image expressed in the RAW format. A compressed RAW image is generated.

  Note that the compression unit 230 generates a compressed RAW image by compressing the images of the plurality of feature regions in the captured image with different intensities according to the feature types of the feature regions. Specifically, the compression unit 230 compresses each of the plurality of captured images with different intensities in the feature region in each of the plurality of captured images and the region other than the feature region in each of the plurality of captured images. A plurality of compressed RAW images each represented in the RAW format are generated. In this way, the compression unit 230 compresses each image area in the RAW image with an intensity corresponding to the importance.

  The association processing unit 206 associates information specifying the feature area detected from the captured image with the captured image. Specifically, the association processing unit 206 associates information for specifying a feature area detected from a captured image with a compressed RAW moving image including the captured image as a moving image constituent image. Then, the output unit 207 outputs the compressed RAW moving image in which the feature area is associated by the association processing unit 206 to the image processing apparatus 170. The image processing apparatus 170 records the compressed RAW moving image in the image DB 175.

  FIG. 3 shows an example of a block configuration of the compression unit 230. The compression unit 230 includes an image dividing unit 232, a plurality of fixed value conversion units 234a-c (hereinafter may be collectively referred to as a fixed value conversion unit 234), and a plurality of compression processing units 236a-d (hereinafter referred to as compression processing). Part 236 in some cases).

  The image dividing unit 232 acquires a plurality of captured images from the image acquisition unit 250. As described above, the captured image is represented in the RAW format.

  Then, the image dividing unit 232 divides the plurality of captured images into a feature region and a background region other than the feature region. Specifically, the image dividing unit 232 divides the plurality of captured images into each of a plurality of feature areas and a background area other than the feature areas. As described above, the image dividing unit 232 divides each of the plurality of captured images into the feature region and the background region. Then, the compression processing unit 236 compresses the feature region image, which is the feature region image, and the background region image, which is the background region image, with different strengths. Specifically, the compression processing unit 236 compresses a feature area moving image including a plurality of feature area images and a background area moving image including a plurality of background area images with different strengths.

  Specifically, the image dividing unit 232 generates a feature area moving image for each of a plurality of feature types by dividing a plurality of captured images. Then, the fixed value unit 234 fixes the pixel values of the regions other than the feature regions of the respective feature types for each of the feature region images included in the plurality of feature region moving images generated for each feature type. To do. Specifically, the fixed value converting unit 234 sets pixel values in regions other than the feature region to predetermined pixel values.

  Then, the compression processing unit 236 compresses a plurality of feature area moving images for each feature type. For example, the compression processing unit 236 performs MPEG compression on a plurality of feature area moving images for each feature type. A specific example of the compression processing in the RAW format in the compression processing unit 236 will be described later.

  Note that the fixed value unit 234a, the fixed value unit 234b, and the fixed value unit 234c are a feature region moving image of the first feature type, a feature region moving image of the second feature type, and a third feature, respectively. The type of feature area animation is fixed. Then, the compression processing unit 236a, the compression processing unit 236b, and the compression processing unit 236c are the feature region moving image of the first feature type, the feature region moving image of the second feature type, and the feature of the third feature type. Compress area video.

  Note that the compression processing units 236a-c compress the feature area moving image with a predetermined strength according to the type of feature. For example, the compression processing unit 236 may convert the feature area moving image to a different resolution determined in advance according to the type of the feature, and compress the converted feature area moving image. In addition, when compressing the feature region moving image by MPEG encoding, the compression processing unit 236 may compress the feature region moving image with different quantization parameters determined in advance according to the feature type.

  The compression processing unit 236d compresses the background area moving image. Note that the compression processing unit 236d may compress the background region moving image with a strength higher than the strength of any of the compression processing units 236a-c. The feature area moving image and the background area moving image compressed by the compression processing unit 236 are supplied to the association processing unit 206.

  Since regions other than the feature region have been fixed values by the fixed value unit 234, when the compression processing unit 236 performs predictive encoding by MPEG encoding or the like, the region other than the feature region may be connected to the predicted image. The amount of difference between the images can be significantly reduced. Therefore, the compression rate of the feature area moving image can be significantly increased.

  In this figure, each of the plurality of compression processing units 236 included in the compression unit 230 compresses a plurality of feature region images and a background region image. However, in another embodiment, the compression unit 230 performs one compression process. A single compression processing unit 236 may compress a plurality of feature region images and background region images with different intensities. For example, a plurality of feature region images and a background region image are sequentially supplied to one compression processing unit 236 in a time-sharing manner, and the one compression processing unit 236 differs from the plurality of feature region images and the background region image. You may compress sequentially by intensity.

  In addition, the one compression processing unit 236 quantizes the image information of the plurality of feature regions and the image information of the background region with different quantization coefficients, respectively, thereby converting the images of the plurality of feature regions and the images of the background region. They may be compressed with different strengths. Also, an image obtained by converting a plurality of feature region images and a background region image into images of different image quality is supplied to one compression processing unit 236, and the one compression processing unit 236 includes a plurality of feature region images and Each image in the background area may be compressed. Further, as described above, one compression processing unit 236 quantizes with a different quantization coefficient for each region, or one compression processing unit 236 compresses an image converted into a different image quality for each region. The compression processing unit 236 may compress one image, or may compress each of the images divided by the image dividing unit 232 as described with reference to FIG. When one compression processing unit 236 compresses one image, the image dividing unit 232 does not need to perform the dividing process and the fixed value converting unit 234 does not perform the fixing process. The dividing unit 232 and the fixed value unit 234 may not be provided.

  FIG. 4 shows an example of a block configuration of the image processing apparatus 170. The image processing apparatus 170 includes an association analysis unit 302, an expansion control unit 310, an expansion unit 320, a color processing unit 360, a synthesis unit 330, an output unit 340, and an instruction acquisition unit 350.

  The image acquisition unit 301 acquires the compressed moving image compressed by the compression unit 230. Specifically, the image acquisition unit 301 acquires a compressed moving image including a plurality of feature area moving images and a background area moving image. More specifically, the image acquisition unit 301 acquires a compressed moving image with feature area information attached thereto. The compressed moving image includes a plurality of compressed RAW images that are compressed captured images. In addition, each of the plurality of feature area moving images and the background area moving image includes a plurality of compressed RAW images. As described above, the image acquisition unit 301 acquires the compressed RAW image compressed by the compression unit 230.

  Then, the association analysis unit 302 separates the compressed moving image into a plurality of feature region moving images, a background region moving image, and feature region information, and supplies the plurality of feature region moving images and background region moving images to the decompressing unit 320. The association analysis unit 302 also analyzes the feature region information and supplies the feature region position and the feature type to the extension control unit 310. The decompression control unit 310 controls the decompression processing by the decompression unit 320 according to the position of the feature region and the feature type acquired from the association analysis unit 302. For example, the decompression control unit 310 decompresses each region of the moving image indicated by the compressed moving image to the decompressing unit 320 according to the compression method in which the compression unit 230 compresses each region of the moving image according to the position of the feature region and the type of feature. Let

  Hereinafter, the operation of each component included in the extension unit 320 will be described. The decompression unit 320 includes a plurality of decoders 322a-d (hereinafter collectively referred to as decoders 322). The decoder 322 decodes one of the plurality of encoded characteristic area moving images and background area moving images. Specifically, the decoder 322a, the decoder 322b, the decoder 322c, and the decoder 322d decode the first feature region moving image, the second feature region moving image, the third feature region moving image, and the background region moving image, respectively. In this manner, the decompression unit 320 creates a decompressed RAW image, which is a decompressed captured image, by decompressing the compressed RAW image included in the feature region moving image or the background region moving image. As described above, the expansion unit 320 generates an expanded RAW image by expanding the compressed RAW image held in the image DB 175.

  The color processing unit 360 calculates values of a plurality of color components in each of a plurality of pixels in the image region indicated by the expanded RAW image based on pixel values of one or more pixels in the expanded RAW image. Specifically, the color processing unit 360 calculates the values of a plurality of color components using different methods for the pixels in the feature region in the expanded RAW image and the pixels in the region other than the feature region in the expanded RAW image.

  More specifically, the color processing unit 360 may calculate the values of a plurality of color components with higher accuracy in the pixels in the feature region in the expanded RAW image than in the pixels in the region other than the feature region in the expanded RAW image. For example, the color processing unit 360 calculates the values of a plurality of color components using the pixel values of a larger number of pixels in the pixels in the feature region in the expanded RAW image than in the pixels in the region other than the feature region in the expanded RAW image. It's okay. In addition, the color processing unit 360 calculates the values of a plurality of color components in the pixels included in the feature area in the expanded RAW image using the plurality of pixel values in the expanded RAW image, and the feature area in the expanded RAW image The values of a plurality of color components in the pixels included in the other area may be represented by the same color component values in other pixels in the expanded RAW image.

  When a plurality of feature regions are detected, the color processing unit 360 determines the values of the plurality of color components in the pixels included in the plurality of feature regions in the expanded RAW image according to the feature types of the feature regions. May be calculated using different methods. In this case, the color processing unit 360 may calculate the values of the plurality of color components in the pixels included in the plurality of feature areas in the expanded RAW image with accuracy according to the feature type of the feature area.

  The synthesizing unit 330 synthesizes a plurality of feature area moving images and background area moving images obtained by the expansion by the expansion unit 320 to generate one display moving image. Specifically, the combining unit 330 generates one display moving image by combining the image of the feature region on the captured image included in the plurality of feature region moving images with the captured image included in the background region moving image.

  The output unit 340 outputs the feature area information and the display moving image acquired from the association analysis unit 302 to the display device 180. The above operation may be performed when an external instruction is acquired.

  For example, the instruction acquisition unit 350 acquires an instruction to output a compressed RAW image held in the image DB 175. The instruction to output includes an instruction to display and an instruction to transmit.

  Then, when the instruction acquisition unit 350 acquires an instruction, the expansion unit 320 generates an expanded RAW image by expanding the compressed RAW image held in the image DB 175. Then, when the instruction acquisition unit 350 acquires the instruction, the color processing unit 360 converts the values of the plurality of color components in each of the plurality of pixels in the image region indicated by the expanded RAW image into one or more pixels in the expanded RAW image. It calculates based on the pixel value of. Then, when the instruction acquisition unit 350 acquires an instruction, the output unit outputs an output image having the values of the plurality of color components calculated by the color processing unit 360 as the pixel values in each of the plurality of pixels. As described above, when the instruction acquisition unit 350 acquires an instruction, the output unit 340 outputs an output image having a plurality of color component values in each of a plurality of pixels in the image region indicated by the expanded RAW image.

  As described above, the image processing apparatus 170 can synchronize and supply an image held in the RAW format to the display apparatus 180 when an instruction is acquired. For this reason, the storage capacity of the image DB 175 can be reduced. Further, the image processing apparatus 170 performs low-precision synchronization processing such as copying neighboring pixels for the background region, and performs high-precision synchronization processing only on the feature region. For this reason, the image processing apparatus 170 can quickly provide a moving image for display to the display device 180 even when the image DB 175 records a moving image in the RAW format.

  Note that the combining unit 330 may perform super-resolution processing on the image of the feature region in the captured image obtained by the color processing unit 360 simultaneously, and then combine the image with the region other than the feature region. Examples of such super-resolution processing include super-resolution processing based on principal component analysis as described in JP-A-2006-350498, or subject movement as described in JP-A-2004-88615. The super-resolution processing based on can be illustrated.

  Note that the synthesis unit 330 may perform super-resolution processing for each object included in the feature region in the captured image obtained by the color processing unit 360 being synchronized. For example, when the feature region includes a human face image, the synthesis unit 330 performs super-resolution processing for each face part (for example, eyes, nose, mouth, etc.) as an example of the object. In this case, the synthesizing unit 330 stores learning data such as a model described in JP-A-2006-350498 for each face part (for example, eyes, nose, mouth). Then, the synthesis unit 330 may perform super-resolution processing on the image of each face part using the learning data selected for each face part included in the feature region.

  As described above, the synthesis unit 330 can reconstruct the image of the feature region using principal component analysis (PCA). As an image reconstruction method by the synthesizer 330 and a learning method for the image reconstruction, in addition to learning / image reconstruction by principal component analysis (PCA), local preserving projection (LPP), Linear Discriminant Analysis (LDA), Independent Component Analysis (ICA), Multidimensional Scaling (MDS), Support Vector Machine (Support Vector Regression), Neural Network, Hidden Markov Model Theory , Maximum posterior probability estimation, iterative backprojection, wavelet transform, locally linear embedding bedding: LLE), Markov random field (Markov random field: MRF) method can be used such as.

  In addition to the model as described in Japanese Patent Application Laid-Open No. 2006-350498, the learning data includes low frequency components and high frequency components of the object image respectively extracted from a large number of sample images of the object. Good. Here, the low-frequency component of the object image is further clustered by the K-means method or the like for each object type, and a representative low-frequency component (for example, a centroid value) is determined for each cluster. It's okay.

  Then, the synthesis unit 330 extracts a low frequency component from the image of the object included in the feature region in the captured image. The synthesizing unit 330 then selects a cluster in which a value suitable for the extracted low-frequency component is determined as a representative low-frequency component from among the low-frequency component clusters extracted from the sample image of the extracted object type object. Is identified. Then, the synthesis unit 330 identifies a cluster of high frequency components associated with the low frequency component included in the identified cluster. In this way, the synthesizer 330 can identify a cluster of high frequency components that are correlated with the low frequency components extracted from the objects included in the captured image. Then, the synthesis unit 330 may convert the image of the object into a higher quality image with higher frequency components that represent the specified cluster of higher frequency components. For example, the synthesis unit 330 may add the high-frequency component selected for each object with a weight according to the distance from the center of each object to the processing target position on the face to the object image. Note that the representative high-frequency component may be generated by closed-loop learning. As described above, since the synthesis unit 330 selects and uses desired learning data for each object from learning data generated by learning for each object, the image of the object is imaged with higher accuracy. May be possible.

  FIG. 5 shows an example of another block configuration of the compression unit 230. The compression unit 230 in this configuration compresses a plurality of captured images by a spatial scalable encoding process corresponding to the type of feature.

  The compression unit 230 in this configuration includes an image quality conversion unit 510, a difference processing unit 520, and an encoding unit 530. The difference processing unit 520 includes a plurality of inter-layer difference processing units 522a-d (hereinafter collectively referred to as inter-layer difference processing units 522). Encoding section 530 includes a plurality of encoders 532a-d (hereinafter collectively referred to as encoders 532).

  The image quality conversion unit 510 acquires a plurality of captured images from the image DB 175. In addition, the image quality conversion unit 510 acquires information specifying the feature region detected by the feature region detection unit 203 and information specifying the type of feature in the feature region. Then, the image quality conversion unit 510 duplicates the captured image, and generates captured images of the number of types of features in the feature area. Then, the image quality conversion unit 510 converts the generated captured image into an image having a resolution corresponding to the type of feature.

  For example, the image quality conversion unit 510 has a captured image converted to a resolution corresponding to the background area (hereinafter referred to as a low resolution image), and a captured image converted to the first resolution corresponding to the type of the first feature ( Called a first resolution image), a captured image converted to a second resolution corresponding to the second feature type (referred to as a second resolution image), and a third resolution corresponding to the third feature type. A captured image (referred to as a third resolution image) converted to is generated. Here, it is assumed that the first resolution image has a higher resolution than the low resolution image, the second resolution image has a higher resolution than the first resolution image, and the third resolution image has a higher resolution than the second resolution image.

  Then, the image quality conversion unit 510 converts the low resolution image, the first resolution image, the second resolution image, and the third resolution image into the inter-layer difference processing unit 522d, the inter-layer difference processing unit 522a, and the inter-layer difference processing unit 522b, respectively. And to the inter-layer difference processing unit 522c. The image quality conversion unit 510 supplies a moving image to each of the inter-layer difference processing units 522 by performing the above-described image quality conversion processing on each of the plurality of captured images.

  Note that the image quality conversion unit 510 may convert the frame rate of the moving image supplied to each of the inter-layer difference processing unit 522 in accordance with the feature type of the feature region. For example, the image quality conversion unit 510 may supply, to the inter-layer difference processing unit 522d, a moving image having a lower frame rate than the moving image supplied to the inter-layer difference processing unit 522a. In addition, the image quality conversion unit 510 may supply a moving image having a lower frame rate than the moving image supplied to the inter-layer difference processing unit 522b to the inter-layer difference processing unit 522a, and a frame lower than the moving image supplied to the inter-layer difference processing unit 522c. The rate movie may be supplied to the inter-tier difference processing unit 522b. Note that the image quality conversion unit 510 may convert the frame rate of the moving image supplied to the inter-layer difference processing unit 522 by thinning out the captured image according to the feature type of the feature region.

  The inter-layer difference processing unit 522d and the encoder 532d predictively encode a background area moving image including a plurality of low-resolution images. Specifically, the inter-layer difference processing unit 522 generates a difference image from a predicted image generated from another low-resolution image. Then, the encoder 532d quantizes the transform coefficient obtained by converting the difference image into a spatial frequency component, and encodes the quantized transform coefficient by entropy coding or the like. Note that such predictive encoding processing may be performed for each partial region of the low-resolution image.

  Further, the inter-layer difference processing unit 522a predictively encodes the first feature region moving image including the plurality of first resolution images supplied from the image quality conversion unit 510. Similarly, the inter-layer difference processing unit 522b and the inter-layer difference processing unit 522c each predictively encode a second feature area moving image including a plurality of second resolution images and a third feature area moving image including a plurality of third resolution images. To do. Hereinafter, specific operations of the inter-layer difference processing unit 522a and the encoder 532a will be described.

  The inter-layer difference processing unit 522a decodes the first resolution image encoded by the encoder 532d, and expands the decoded image to an image having the same resolution as the first resolution. Then, the inter-layer difference processing unit 522a generates a difference image between the enlarged image and the low resolution image. At this time, the inter-layer difference processing unit 522a sets the difference value in the background area to zero. Then, the encoder 532a encodes the difference image in the same manner as the encoder 532d. Note that the encoding processing by the inter-layer difference processing unit 522a and the encoder 532a may be performed for each partial region of the first resolution image.

  Note that, when the first resolution image is encoded, the inter-layer difference processing unit 522a calculates the code amount predicted when the difference image with the low resolution image is encoded, and the other first resolution image. The amount of code predicted when the difference image between the generated prediction image and the prediction image is encoded is compared. In the case where the latter code amount is smaller, the inter-layer difference processing unit 522a generates a difference image from the predicted image generated from the other first resolution image. In addition, the inter-layer difference processing unit 522a, when it is predicted that the code amount is smaller when encoded without taking the difference from the low-resolution image or the predicted image, It is not necessary to take a difference between them.

  Note that the inter-layer difference processing unit 522a does not have to set the difference value in the background area to zero. In this case, the encoder 532a may set the encoded data for difference information in an area other than the feature area to zero. For example, the encoder 532a may set the conversion coefficient after conversion to a frequency component to zero. Note that the motion vector information when the inter-layer difference processing unit 522d performs predictive encoding is supplied to the inter-layer difference processing unit 522a. The inter-layer difference processing unit 522a may calculate a motion vector for a predicted image using the motion vector information supplied from the inter-layer difference processing unit 522d.

  Note that the operations of the inter-layer difference processing unit 522b and the encoder 532b are that the second resolution image is encoded, and the first resolution after encoding by the encoder 532a when the second resolution image is encoded. The operations of the inter-layer difference processing unit 522b and the encoder 532b are substantially the same as the operations of the inter-layer difference processing unit 522a and the encoder 532a, except that the difference from the image may be taken. To do. Similarly, the operations of the inter-layer difference processing unit 522c and the encoder 532c are that the third resolution image is encoded, and when the third resolution image is encoded, after the encoding by the encoder 532b. Except for the fact that a difference from the second resolution image may be obtained, the operation is substantially the same as the operation of the inter-layer difference processing unit 522a and the encoder 532a, and thus the description thereof is omitted.

  As described above, the image quality conversion unit 510 generates, from each of the plurality of captured images, a low image quality image with a low image quality and a feature region image with higher image quality than the low image quality at least in the feature region. Then, the difference processing unit 520 generates a feature region difference image indicating a difference image between the feature region image in the feature region image and the feature region image in the low-quality image. Then, the encoding unit 530 encodes the feature region difference image and the low quality image.

  In addition, the image quality conversion unit 510 generates a low-quality image with reduced resolution from a plurality of captured images, and the difference processing unit 520 includes a feature region image in the feature region image and a feature region image in the low-quality image. A feature region difference image between the image and the image enlarged is generated. Further, the difference processing unit 520 has a spatial frequency component in which the difference between the feature region image and the enlarged image in the feature region is converted into the spatial frequency region, and the data amount of the spatial frequency component is in the region other than the feature region. A reduced feature area difference image is generated.

  As described above, the compression unit 230 performs hierarchical encoding by encoding image differences between a plurality of layers having different resolutions. As is clear from this, a part of the compression method by the compression unit 230 of this configuration is H.264. It is clear that a compression scheme according to H.264 / SVC is included. Note that when the image processing apparatus 170 decompresses such a layered compressed moving image, the moving image data of each layer is decoded, and the difference is taken for the region encoded by the inter-layer difference. The captured image having the original resolution can be generated by the addition process with the captured image decoded in the hierarchy.

  The example of the compression process by the compression unit 230 has been described above with reference to FIGS. 3 and 5. Note that in super-resolution processing based on principal component analysis as described in Japanese Patent Laid-Open No. 2006-350498 described with reference to FIG. 4, an image of an object is represented by a principal component vector and a weighting coefficient. The data amounts of these weighting coefficients and principal component vectors are significantly smaller than the data amount of pixel data included in the object image itself. Therefore, the compression unit 230 may calculate the weighting coefficient from the image of the object included in the feature region in the compression processing for compressing the feature region image in the plurality of captured images acquired from the imaging unit 102. That is, the compression unit 230 can compress an image of an object included in the feature region by representing the principal component vector and the weighting coefficient. Then, the output unit 207 may transmit the principal component vector and the weighting coefficient to the image processing apparatus 170. In the image processing device 170, an image of an object included in the feature region can be reconstructed using the principal component vector and the weighting coefficient acquired from the image processing device 120. The image processing apparatus 120 is included in the feature region using a model that represents an object with various feature parameters in addition to a model based on principal component analysis as described in JP-A-2006-350498. Needless to say, an image of an object can be compressed.

  FIG. 6 shows an example of a pixel array 600 of a captured image expressed in the RAW format. Gnm (n = 1-6, m = 1-6) in this figure indicates a pixel having an output value corresponding to the light intensity of the green color component. Rnm (n = 1 to 5, m = 1 to 6) indicates a pixel having an output value corresponding to the light intensity of the red color component. Bnm (n = 1 to 6, m = 1 to 5) indicates a pixel having an output value corresponding to the light intensity of the red color component. In this way, each pixel has an output value corresponding to the intensity of any one of R, G, and B color components.

  Although the primary color pixel array 600 has been described with reference to this drawing, the captured image may have a complementary color pixel array. Further, the captured image may include pixels having output values corresponding to the light intensities of the four color components.

  FIG. 7 shows an example of the feature region. The feature region detection unit 203 detects a feature region from each of the captured image 700-1, the captured image 700-1, and the captured image 700-3 (hereinafter collectively referred to as the captured image 700) included in the moving image.

  For example, the feature region detection unit 203 uses the captured images 700-1 to 700-3 as moving regions 710-1 to 710-3 (hereinafter collectively referred to as a moving region 710) including moving images, and human body images. Body region 720-1 to -3 (hereinafter collectively referred to as body region 720), and head region 730-1 to 730-3 (hereinafter collectively referred to as head region 730) including an image of a person's head. ) Is detected. In each captured image 700, the region other than the moving region 710, the body region 720, and the head region 730 is the background region described above.

  Note that the feature region detection unit 203 may detect the feature region based on the pixel value of a pixel having a predetermined color component value included in the captured image 700. For example, the feature region detection unit 203 may detect the feature region by image recognition based on the pixel value of a pixel having a predetermined color component value included in the captured image 700. More specifically, the feature region detection unit 203 may detect a feature region by image recognition based on the G signal.

  FIG. 8 shows an example of a combination of pixels selected from the feature region and the background region in the compression unit 230. In the following description, an example of pixels selected from the movement region 710, the body region 720, the head region 730, and the background region described in FIG. 7 will be described using the pixel array 600 described in FIG.

  When compressing the image of the head region 730, the compression processing unit 236a detects pixels indicated by the G plane 810a and the G plane 810b (hereinafter collectively referred to as the G plane 810), the B plane 811, and the R plane 812. Select and compress. The G plane 810a includes G11, G13, G15..., And the G plane 810b includes G22, G24, G26. The B plane 811 includes B21, B23, B25. The R plane 812 includes R12, R14, R16. As described above, the compression processing unit 236a selects and compresses all pixels from the pixel array included in the head region 730.

  The compression processing unit 236b selects pixels indicated by the G plane 820, the B plane 821, and the R plane 822 when compressing the image of the head region 720. Note that the G plane 820, the B plane 821, and the R plane 822 may be the same as the G plane 810a, the B plane 811, and the R plane 812.

  In addition, the compression processing unit 236c selects pixels indicated by the G plane 830, the B plane 831, and the R plane 832 when compressing the image of the moving area 710. The G plane 830 includes G11, G15,..., The B plane 831 includes B21, B25..., And the R plane 832 includes R12, R16.

  The compression processing unit 236d selects pixels indicated by the G plane 840, the B plane 841, and the R plane 842 when compressing the image of the background area. The G plane 840 includes G11..., The B plane 841 includes B21..., And the R plane 842 includes R12.

  Note that the compression processing unit 236 may compress the image signals generated by the G plane 810, the G plane 820, the G plane 830, and the G plane 840 as Y signals. In addition, the compression processing unit 236 processes the image signals generated by the B plane 811, the B plane 821, the B plane 831, and the B plane 841 as Cb signals, so that the R plane 812, the R plane 822, the R plane 832, The image signal generated by the R plane 842 may be processed as a Cr signal.

  Note that the compression processing unit 236 may perform inter-frame compression of a plurality of images obtained by selecting pixels indicated by each plane from each of the captured images 700 for each plane. Therefore, the compression processing unit 236 can be configured using an encoder that performs MPEG encoding.

  FIG. 9 shows an example of an image signal obtained after the synchronization processing by the color processing unit 360. As shown in the figure, the color processing unit 360 generates image signals indicated by the G plane 900, the B plane 901, and the R plane 902 by the synchronization process. For example, the color processing unit 360 calculates a pixel indicated by a shade in the drawing using a pixel value of a pixel located in the vicinity of the pixel. As a result, the color processing unit 360 generates an image signal having pixel values of each color in all pixels.

  In the drawing, shaded pixels indicate pixels for which the color processing unit 360 should calculate pixel values when the inside of the head region 730 is synchronized. Needless to say, for the body region 720, the moving region 710, and the background region, the color processing unit 360 may calculate pixel values for more pixels.

  For example, the color processing unit 360 calculates pixel values for the G12 and G13 pixels in the moving region 710. Note that the above-described synchronization processing by the color processing unit 360 includes substantial image enlargement processing in the body region 720, the moving region 710, and the background region. As described above, since the color processing unit 360 performs the synchronization process and the image enlargement process at the same time, the storage capacity of the image DB 175 can be reduced, and the processing speed of the image processing apparatus 170 may be improved.

  FIG. 10 shows an example of a weighting coefficient used when the color processing unit 360 performs the background region synchronization processing. For the G signal, the color processing unit 360 calculates the pixel value by weighting the pixel value with the weighting shown in FIG. Also, the color processing unit 360 calculates pixel values by weighting the pixel values with the weights shown in FIG.

  Note that the above method is an example of a low-accuracy synchronization process, and the color processing unit 360, in addition to the above, uses the nearest neighbor method or linear interpolation (or cubic spline) to calculate pixel values in the background region. It may be calculated. Note that the color processing unit 360 may calculate the pixel values of the moving region 710, the body region 720, and the head region 730 with higher accuracy than the background region by using more pixel values.

  In addition, the color processing unit 360 may perform synchronization processing using image signals having a larger number of colors or a larger number of gradations in the feature region. In addition, the color processing unit 360 may perform synchronization processing using image signals obtained by image sensors having different sensitivities in the feature region. In this case, the color processing unit 360 can perform synchronization processing using a method described in Japanese Patent Application Laid-Open No. 2004-336469. In addition, the color processing unit 360 may perform demosaicing processing in the feature region according to the difference in the total-variation energy terms of the color signal, the color difference signal, and the color sum signal.

  FIG. 11 shows an example of an image processing system 20 according to another embodiment. The configuration of the image processing system 20 in the present embodiment is the image described with reference to FIG. 1 except that the imaging devices 100a-d have image processing units 804a-d (hereinafter collectively referred to as image processing units 804). The configuration of the processing system 10 is the same.

  The image processing unit 804 has components other than the image acquisition unit 250 among the components included in the image processing apparatus 120. The functions and operations of the constituent elements included in the image processing unit 804 are replaced with that the constituent elements included in the image processing device 120 process the captured moving image obtained by the decompression processing by the compressed moving image decompression unit 202. The functions and operations of the components included in the image processing apparatus 120 may be substantially the same except that the captured moving image captured by the imaging unit 102 is processed. Note that the captured moving image may include a RAW-format captured image as a moving image constituent image. Also in the image processing system 20 having such a configuration, substantially the same effect as that described in relation to the image processing system 10 in FIGS. 1 to 10 can be obtained.

  FIG. 12 shows an example of the hardware configuration of the image processing device 120 and the image processing device 170. The image processing device 120 and the image processing device 170 include a CPU peripheral unit, an input / output unit, and a legacy input / output unit. The CPU peripheral section includes a CPU 1505, a RAM 1520, a graphic controller 1575, and a display device 1580 that are connected to each other by a host controller 1582. The input / output unit includes a communication interface 1530, a hard disk drive 1540, and a CD-ROM drive 1560 that are connected to the host controller 1582 by the input / output controller 1584. The legacy input / output unit includes a ROM 1510, a flexible disk drive 1550, and an input / output chip 1570 connected to the input / output controller 1584.

  The host controller 1582 connects the RAM 1520, the CPU 1505 that accesses the RAM 1520 at a higher transfer rate, and the graphic controller 1575. The CPU 1505 operates according to the contents of the programs stored in the ROM 1510 and the RAM 1520 and controls each unit. The graphic controller 1575 acquires image data generated by the CPU 1505 or the like on a frame buffer provided in the RAM 1520 and displays the image data on the display device 1580. Alternatively, the graphic controller 1575 may include a frame buffer that stores image data generated by the CPU 1505 or the like.

  The input / output controller 1584 connects the host controller 1582 to the hard disk drive 1540, the communication interface 1530, and the CD-ROM drive 1560, which are relatively high-speed input / output devices. The hard disk drive 1540 stores programs and data used by the CPU 1505. The communication interface 1530 is connected to the network communication device 1598 to transmit / receive programs or data. The CD-ROM drive 1560 reads a program or data from the CD-ROM 1595 and provides it to the hard disk drive 1540 and the communication interface 1530 via the RAM 1520.

  The input / output controller 1584 is connected to the ROM 1510, the flexible disk drive 1550, and the relatively low-speed input / output device of the input / output chip 1570. The ROM 1510 stores a boot program that is executed when the image processing device 120 and the image processing device 170 are started, or a program that depends on the hardware of the image processing device 120 and the image processing device 170. The flexible disk drive 1550 reads a program or data from the flexible disk 1590 and provides it to the hard disk drive 1540 and the communication interface 1530 via the RAM 1520. The input / output chip 1570 connects various input / output devices via the flexible disk drive 1550 or a parallel port, serial port, keyboard port, mouse port, and the like.

  A program executed by the CPU 1505 is stored in a recording medium such as the flexible disk 1590, the CD-ROM 1595, or an IC card and provided by the user. The program stored in the recording medium may be compressed or uncompressed. The program is installed in the hard disk drive 1540 from the recording medium, read into the RAM 1520, and executed by the CPU 1505. The program executed by the CPU 1505 causes the image processing apparatus 120 to function as each component included in the image processing apparatus 120 described with reference to FIGS. 1 to 11, and the image processing apparatus 170 is changed to FIGS. It is made to function as each component which image processing apparatus 170 which was explained related has.

  The program shown above may be stored in an external storage medium. As the storage medium, in addition to the flexible disk 1590 and the CD-ROM 1595, an optical recording medium such as a DVD or PD, a magneto-optical recording medium such as an MD, a tape medium, a semiconductor memory such as an IC card, or the like can be used. Further, a storage device such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet is used as a recording medium, and is provided to the image processing device 120 and the image processing device 170 as a program via the network. Also good. As described above, the computer controlled by the program functions as the image processing device 120 and the image processing device 170.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

1 is a diagram illustrating an example of an image processing system 10 according to an embodiment. 2 is a diagram illustrating an example of a block configuration of an image processing apparatus 120. FIG. 3 is a diagram illustrating an example of a block configuration of a compression unit 230. FIG. 2 is a diagram illustrating an example of a block configuration of an image processing apparatus 170. FIG. It is a figure which shows an example of the other block structure of the compression part 230. FIG. It is a figure which shows an example of the pixel arrangement | sequence 600 of the captured image represented with a RAW format. It is a figure which shows an example of a characteristic area. It is a figure which shows an example of the combination of the pixel selected from the characteristic area and the background area in the compression part 230. FIG. It is a figure which shows an example of the image signal obtained after the synchronous process by the color process part 360. FIG. It is a figure which shows an example of the weighting coefficient used when the color process part 360 carries out the synchronous process of a background area | region. It is a figure which shows an example of the image processing system 20 which concerns on other embodiment. 2 is a diagram illustrating an example of a hardware configuration of an image processing device 120 and an image processing device 170. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image processing system 20 Image processing system 100 Image pick-up apparatus 102 Image pick-up part 104 Image pick-up image compression part 110 Communication network 120 Image processing apparatus 130 Person 140 Mobile body 150 Monitoring object space 160 Space 170 Image processing apparatus 175 Image DB
180 Display device 201 Compressed video acquisition unit 202 Compressed video expansion unit 203 Feature area detection unit 204 Detection condition acquisition unit 206 Association processing unit 207 Output unit 210 Compression control unit 230 Compression unit 232 Image division unit 234 Fixed value conversion unit 236 Compression processing Unit 250 image acquisition unit 301 image acquisition unit 302 association analysis unit 310 expansion control unit 320 expansion unit 322 decoder 330 synthesis unit 340 output unit 350 instruction acquisition unit 360 color processing unit 510 image quality conversion unit 520 difference processing unit 522 inter-layer difference Processing unit 530 Encoding unit 532 Encoder 600 Pixel array 700 Captured image 710 Moving region 720 Body region 730 Head region 810 G plane 811 B plane 812 R plane 820 G plane 821 B plane 822 R plane 830 G plane 831 B plane 832 R Pre Down 840 G plane 841 B plane 842 R plane 900 G plane 901 B plane 902 R plane 804 the image processing section 1505 CPU
1510 ROM
1520 RAM
1530 Communication interface 1540 Hard disk drive 1550 Flexible disk drive 1560 CD-ROM drive 1570 Input / output chip 1575 Graphic controller 1580 Display device 1582 Host controller 1584 Input / output controller 1590 Flexible disk 1595 CD-ROM
1598 Network communication device

Claims (19)

An image acquisition unit for acquiring a RAW image represented in a RAW format;
A feature region detection unit that detects a feature region in the RAW image using the RAW image;
A compressed RAW image that is a compressed RAW image expressed in the RAW format by compressing the RAW image with different intensities in the feature region in the RAW image and a region other than the feature region in the RAW image. a compression unit which generates,
An expansion unit that generates an expanded RAW image that is the expanded RAW image by expanding the compressed RAW image;
A color processing unit that calculates values of a plurality of color components in each of a plurality of pixels in an image region indicated by the expanded RAW image based on pixel values of one or more pixels in the expanded RAW image. ,
The color processing unit calculates values of the plurality of color components in different manners for pixels in the feature region in the expanded RAW image and pixels in regions other than the feature region in the expanded RAW image. > Image processing system. The image processing system according to claim 1, wherein the image acquisition unit acquires the RAW image in which a captured image is represented in the RAW format. Wherein the color processing unit, in the pixel in the characteristic region from the pixels in the region other than the feature region in the decompressed RAW image in the decompressed RAW image, claim 1 to calculate the value of the plurality of color components with higher accuracy or the image processing system according to 2. The feature region detection unit detects a plurality of feature regions in the RAW image using the RAW image,
Said compression portion, an image of a plurality of the characteristic region in the RAW image, by compressing each with different intensities depending on the type of characteristics of the characteristic region, according to claim 1 or 2 to generate the compressed RAW image Image processing system. 5. The color processing unit according to claim 4 , wherein the color processing unit calculates values of the plurality of color components in pixels included in the plurality of feature regions in the expanded RAW image in different manners according to the feature types of the feature regions. Image processing system. The image according to claim 5 , wherein the color processing unit calculates values of the plurality of color components in pixels included in the plurality of feature regions in the expanded RAW image with accuracy according to the type of feature of the feature region. Processing system. The color processing unit uses the pixel values of the plurality of color components using pixel values of a larger number of pixels in pixels in the feature region in the expanded RAW image than in pixels in the region other than the feature region in the expanded RAW image. The image processing system according to claim 3 , wherein: The color processing unit calculates values of the plurality of color components in pixels included in the feature region in the expanded RAW image using a plurality of pixel values in the expanded RAW image, and the features in the expanded RAW image The image processing system according to claim 3 , wherein values of the plurality of color components in pixels included in a region other than the region are represented by values of the same color component in other pixels in the expanded RAW image. The image processing system according to claim 3 , wherein the feature region detection unit detects the feature region based on a pixel value of a pixel having a predetermined color component value included in the RAW image. A detection condition acquisition unit that acquires a color component of a pixel value to be used when the feature region detection unit detects the feature region from the RAW image;
The feature region detecting unit, among the pixels included in the RAW image, according to claim 9, based on the pixel values of the pixels having a value of a color component which the detection condition acquisition unit has acquired, it detects the feature area Image processing system. The image acquisition unit acquires a moving image including a plurality of the RAW images as moving image constituent images,
The feature region detection unit detects feature regions in the plurality of RAW images from each of the plurality of RAW images,
The compression unit compresses each of the plurality of RAW images with different intensities in the feature region in each of the plurality of RAW images and in a region other than the feature region in each of the plurality of RAW images, The image processing system according to any one of claims 1 to 10, wherein a plurality of the compressed RAW images each represented in a RAW format are generated. An image holding unit for holding the compressed RAW image;
The image processing system according to any one of claims 1 to 11 , wherein the expansion unit generates the expanded RAW image by expanding the compressed RAW image held in the image holding unit. . An instruction acquisition unit for acquiring an instruction to output the compressed RAW image held in the image holding unit;
An output unit that outputs an output image having a plurality of color component values in each of a plurality of pixels in an image region indicated by the decompressed RAW image when the instruction acquisition unit acquires the instruction;
The expansion unit generates the expanded RAW image by expanding the compressed RAW image held in the image holding unit when the instruction acquisition unit acquires the instruction,
The color processing unit calculates values of the plurality of color components in each of a plurality of pixels in an image region indicated by the decompressed RAW image when the instruction acquisition unit acquires the instruction,
When the instruction acquisition unit acquires the instruction, the output unit outputs the output image having the values of the plurality of color components calculated by the color processing unit as pixel values in the plurality of pixels, respectively. The image processing system according to claim 12 .   A program for an image processing system, comprising:
  The program for functioning as an image processing system as described in any one of Claims 1-13. An image acquisition stage for acquiring a RAW image represented in RAW format;
A feature region detection step of detecting a feature region in the RAW image using the RAW image;
A compressed RAW image that is a compressed RAW image expressed in the RAW format by compressing the RAW image with different intensities in the feature region in the RAW image and a region other than the feature region in the RAW image. and the compression stage to generate,
Generating a decompressed RAW image that is the decompressed RAW image by decompressing the compressed RAW image;
A color processing step of calculating values of a plurality of color components in each of a plurality of pixels in an image region indicated by the expanded RAW image based on pixel values of one or more pixels in the expanded RAW image. ,
In the color processing step, values of the plurality of color components are calculated in different manners for pixels in the feature region in the expanded RAW image and pixels in regions other than the feature region in the expanded RAW image. > Image processing method. An image acquisition unit for acquiring a RAW image represented in a RAW format;
A feature region detection unit that detects a feature region in the RAW image using the RAW image;
A color processing unit that calculates values of a plurality of color components in each of a plurality of pixels in the RAW image based on pixel values of one or more pixels in the RAW image;
The image processing system, wherein the color processing unit calculates values of the plurality of color components in different manners for a pixel in the feature region in the RAW image and a pixel in a region other than the feature region in the RAW image. Wherein the color processing unit, in the pixel in the characteristic region from the pixels in the region other than the feature region in the RAW image in the RAW image, according to claim 16 for calculating a value of said plurality of color components with higher accuracy Image processing system.   A program for an image processing system, comprising:
  The program for functioning as an image processing system of Claim 16 or 17. An image acquisition stage for acquiring a RAW image represented in RAW format;
A feature region detection step of detecting a feature region in the RAW image using the RAW image;
A color processing stage that calculates values of a plurality of color components in each of a plurality of pixels in the RAW image based on pixel values of one or more pixels in the RAW image,
The image processing method in which the color processing step calculates the values of the plurality of color components in different manners for pixels in the feature region in the RAW image and pixels in regions other than the feature region in the RAW image.

Images